PyMedPhys Activity Report -- The new PyMedPhys GUI

[PyMedPhys]

PyMedPhys GUI

A tool to compare Linac deliveries directly to the TPS plan

Within our clinics at RCCC, NBCC, and SASH we have developed a new tool for comparing the Linac delivery directly to the plan recorded within Monaco. We have a server set up at RCCC, and we use SSH tunnels, dns aliases and nginx to make it so that at all three sites if http://physics-server/delivery is typed into a browser the user is greeted by a MU Density comparison tool. Some demo data has been created, and this GUI we use internally can be run on your own machine by running the following within a command prompt:

pip install pymedphys==0.25.1

pymedphys gui

Of note, in order to roll out this GUI within our centre we have implemented automated GUI testing with Cypress (https://www.cypress.io/). This means that for every code change undergone within the PyMedPhys repo a machine spins up and runs the GUI to make sure that key tasks within the GUI have not been broken by those code changes. For example, for the 0.25.1 release, you can watch a video online of those automated tests running by going to https://dashboard.cypress.io/projects/tgt8f6/runs/57/specs and then pressing the video play button as in the screenshot below:

If you would be interested in using this GUI in your centre please reach out.

Some notes on local configuration

For those who are interested, to stop using the demo data, and instead use data from your own clinic, you will need to create a `config.toml` file within the directory `~/.pymedphys`. To see the `config.toml` file used in our own centre see:

https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml

Of note, if you would like to have your configuration file in either a shared directory, or under git version control you can specify a "redirect field" within your root config file. For example, the file found at `~/.pymedphys/config.toml` on our server looks like the following:

redirect = 'C:\Users\PExIT\git\pymedphys\examples\site-specific\cancer-care-associates\config.toml'

Under the hood, `pymedphys gui` uses streamlit (https://www.streamlit.io/). A brilliant tool for GUI creation aimed at science programmers. If you would like to customise how `pymedphys gui` runs on your machine you can do this through streamlit's configuration file. For example, on our server we have the following file at `~/.streamlit/config.toml`:

[server]

port = 38894

headless = true

baseUrlPath = "/delivery/"

[browser]

serverAddress = "physics-server"

serverPort = 80

gatherUsageStats = false

New Releases

Since the last activity report the following releases have occurred:

• 0.22.0 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.22.0
• 0.23.0 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.23.0
• 0.24.0 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.24.0
• 0.24.1 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.24.1
• 0.24.2 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.24.2
• 0.24.3 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.24.3
• 0.25.0 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.25.0
• 0.25.1 -- https://github.com/pymedphys/pymedphys/releases/tag/v0.25.1

[LipTeck Chew]

Hi Simon,

I have installed pymedphys and ImageMagick both in 64 bits Windows 7 and 64 bits Ubuntu 20.04 LTS Desktop versions running good in demo mode. I am now trying to setup in the actual clinical environment. Our department is using full Elekta Solution: Mosaiq v2.64, Monaco v5.11.03 with 2 Agility Head Linacs: 1 VersaHD and 1 Infinity.

1. I am confused with iComLogFiles directory and the iComLogFiles\live directory described

https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml

May I know what the differences are?

What are the recommendation for the location of logfiles, live and grouped patient directory: local or NAS?

 

2. In the example of listeners at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, both batch scripts consolidate the logfiles from two linacs' NSS to D:\iComLogFilesThen, in the patient archiving (grouping) it does something using D:\iComLogFiles\live and Z:\iCom\live to produce something at D:\iComLogFiles\patients

May I know what is “Z:\iCom\live” ? 

Is Z:\ a local drive or network drive?

Can a single computer listen to both NSS simultaneously or we need to deploy one listener per computer for each NSS?  

3. In the process of grouping patients at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, there is a line: “This is done by setting up a .bat file to run on machine boot”. 

Could you elaborate further? 

Is the machine here referring to the linacs or physics-server/pc? 

Do you mean patient archiving (grouping) is done daily in the morning?

 

3.       Regarding the command line tool at https://docs.pymedphys.com/ref/cli/logfile.html: pymedphys logfile orchestration [-h] [-m MOSAIQ_SQL] [-l LINAC_DETAILS] data_directory (orchestration.bat)

May I know whether this data_directory referring to ~.\pymedphys\data or D:\iComLogFiles\patients or the TRF directory?

Which data directory should we configure?

Do we need to map network drive to each separate NSS with the rtduser?

 

4.       How do we enable reading all clinics or designated clinics appear under the \\FocalData\Installation directory described at https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml? We have several folders for different groups of doctors.

 

5.       Do we need the D:\iComLogFiles\patients for the gui to read trf ? What are the configurations required just to make the trf portion work? Would the trf files from two NSS override each other?

Thank you for your patience :)

warm regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

I’m keen to send you some nice responses to this. Currently quite under the pump, might have to respond to you tonight when I get home from work.

 

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/c038999e-5dc5-479b-aa58-4355ed687f27%40googlegroups.com.

[Simon Biggs]

Hi LipTeck,

 

As a first point, the iCOM command line tool recently underwent a small revamp. See https://github.com/pymedphys/pymedphys/releases/tag/v0.28.0 for more details. The GUI itself has undergone a change too, see https://github.com/pymedphys/pymedphys/releases/tag/v0.29.0.

 

As such, I would recommend updating to pymedphys==0.29.1 before implementing the feedback below.

 

Also, given this is going down the path of implementing the iCOM listener please be aware of the following bug that appears to plague all iCOM listener tools (commercial tools included) https://github.com/pymedphys/pymedphys/issues/849.

 

Cheers,

Simon

 

 

 

 

Hi Simon,

 

I have installed pymedphys and ImageMagick both in 64 bits Windows 7 and 64 bits Ubuntu 20.04 LTS Desktop versions running good in demo mode. I am now trying to setup in the actual clinical environment. Our department is using full Elekta Solution: Mosaiq v2.64, Monaco v5.11.03 with 2 Agility Head Linacs: 1 VersaHD and 1 Infinity.

 

 

1. I am confused with iComLogFiles directory and the iComLogFiles\live directory described

https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml

May I know what the differences are?

Both the live and the patients directories within the iCOM directory are for storing iCOM data. The live directory is a simple .txt dump of the most recent iCOM stream records. These files a regularly overwritten. The patients directory is where iCOM stream data is grouped and indexed by patient.

What are the recommendation for the location of logfiles, live and grouped patient directory: local or NAS?

I would have the iCOM files be stored on the same machine that is running the iCOM listener, and then have these files shared via the network so that they are accessible by the machine that is running pymedphys gui.

2. In the example of listeners at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, both batch scripts consolidate the logfiles from two linacs' NSS to D:\iComLogFilesThen, in the patient archiving (grouping) it does something using D:\iComLogFiles\live and Z:\iCom\live to produce something at D:\iComLogFiles\patients

May I know what is “Z:\iCom\live” ? 

Is Z:\ a local drive or network drive?

Z:\ is a network drive shared via intranet from another site. We run this tool for 4 Linacs at 3 sites.

Can a single computer listen to both NSS simultaneously or we need to deploy one listener per computer for each NSS?  

A single computer can listen to both NSS’. We have three computers doing it since there is one computer at each site and I don’t want the internet connection to cause the dropout issues discussed above.

 

3. In the process of grouping patients at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, there is a line: “This is done by setting up a .bat file to run on machine boot”. 

Could you elaborate further? 

Is the machine here referring to the linacs or physics-server/pc? 

Do you mean patient archiving (grouping) is done daily in the morning?

Actually, if you install pymedphys 0.29.1, this archiving tool is no longer needed (and the CLI interface has been removed). It is now undergone automatically by the listener process already. No need to manage any of that part yourself any longer.

 

3.       Regarding the command line tool at https://docs.pymedphys.com/ref/cli/logfile.html: pymedphys logfile orchestration [-h] [-m MOSAIQ_SQL] [-l LINAC_DETAILS] data_directory (orchestration.bat)

May I know whether this data_directory referring to ~.\pymedphys\data or D:\iComLogFiles\patients or the TRF directory?

That data directory is actually where ever you would like it to be... I personally put it on the D drive of a server’s local machine where that D drive points to a rather large spinning disk storage. The TRF files end up taking quite a lot of space.

Which data directory should we configure?

In your config.toml point to the same directory as written within the logfile orchestration tool.

Do we need to map network drive to each separate NSS with the rtduser?

No, don’t map the NSS network drive, but do save the rtduser password within the Windows credential manager:

 

Instead of writing “tunnel-sash-9002”, instead write in the IP address of the NSS on both the network address section, and the username section.

4.       How do we enable reading all clinics or designated clinics appear under the \\FocalData\Installation directory described at https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml? We have several folders for different groups of doctors.

Unfortunately this is not currently directly supported at the moment, I might be able to make this possible. I do wonder if it might “just work” if you treat each clinic as its own “site” within the config file, as in create a new site for each clinic, and have all the configuration be the same except change the clinic path. It would be nice to directly support this use case though.

5.       Do we need the D:\iComLogFiles\patients for the gui to read trf ? What are the configurations required just to make the trf portion work?

Nope, you don’t need that patient directory if you’re going to be just using TRF. However, now that patient indexing is included within the listener maybe this is not as much of an issue?

Would the trf files from two NSS override each other?

They shouldn’t. The TRF files are organised by IP address. Let me know if you have any issues here though.

Thank you for your patience :)

My pleasure :) Things are quite hectic at the moment, thanks for baring with response delays :). Feel free to let me know if you have any more issues.

warm regards,

LipTeck

[LipTeck Chew]

Hi Simon,

I am excited to see the development is getting faster and I have upgraded pymedphys to v0.29.1 successfully.

I have used the nssm tool to create the iCom listeners for two NSS according to https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb without patient_archiving since it is now built-in. I have screenshot the tasks manager on both listeners. Hope this is a good indication of correct implementation. (iCom listening services.png)

Would these listeners automatically start when the linacs are turned on?

I have set the credential ok but there was time zone error when running the orchestration script. Please see get TRF error screenshots. FYI, I am using "Asia/Singapore" for this field in the cvs files and config.toml

I have managed to setup multiple clinics by duplicating the [[site]], [site.monaco], [site.mosaiq], [site.linac] , using site names as clinic names and clinic folders in FocalData\Installation directory.

Hope to get the TRF tool and iCom going :)

regards,

LipTeck

[Simon Biggs]

Great to see the progress :). What does your config_mosaiq_sql.csv (https://docs.pymedphys.com/ref/cli/logfile.html#config-mosaiq-sql-csv) look like?

After running the iCOM listener are the directories "live" and "patient" appearing within the directory you are pointing to?

Regarding your question about the services starting, no you'll need to initially press "start", but once you've done that it should automatically be running whenever the Linac reboots.

--
You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/35511b23-3edf-4765-a3ec-58fa81ce719d%40googlegroups.com.

[LipTeck CHEW]

Hi Simon

I have attached my config_mosaiq_sql.csv and the getTRF_error.txt  We are looking at the possibility of getting the new TRF files nightly to a NAS. Suppose we have a cronjob to keep adding the fresh TRF files to D:\pymedphys\TRF, will the GUI able to recognise the files or we still need to run the orchestration CLI?

I have manually created the live and patients directories within D:\pymedphys\iComLogFiles\live and D:\pymedphys\patients earlier on. Nevertheless, I do have the listeners_log.txt for both NSS in the live directory after clicking on the "start" for both listeners. Please see attached screenshot. Yes, I think it should work when the linacs are turned on.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

When you run the orchestration CLI command there is a file created called something like "to_be_indexed". Any trf file placed in there will be picked up by the orchestration CLI and indexed as per its standard processes. Also, if ever a duplicate TRF file is found it is discarded instead of causing duplicate data entries. So, if you are able to have TRF files nightly, you could have that process running, placing the files within "to_be_indexed" then, once that was finished you can then run the orchestration CLI and it will index them for you, as well as pull files from any diagnostic backups in case there is an issue with your nightly pull.

In order for the GUI to recognise them, they do need to be indexed according to the orchestration CLI. But as long as you run both that should work just fine.

For the timezone issue, I might need to get more details off of you. Just to cover our bases before diving in too deep could you tell me what version of pandas you are using?

Cheers,

Simon

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPH5k7vm-AtrKrPXRJh6YajBCaZAis6E-vKHMxK69h89Q%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Thanks for a detail explanation on the Treatment Record File (TRF) orchestration CLI. 

Running “conda list pandas” shows that it’s version 1.0.3 Build pypi_0 channel pypi 

Regards 

Lip Teck

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CAALqB0BWYiAV0nRYjSfBD6eui-5Ww_CJVHD_XxWMDq4btdwBow%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

The iCom listeners are not working with error message attached.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

Instead of running the command as a service could you run:

```

pymedphys --debug icom listen ip.address.goes.here D:/path/to/file/saving/location

```

And then let me know what happens

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPx0ms9KBNkR_mE6hPamsNbJD77m49zrh5o_dqDefUuSQ%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Please see the attached text file.

regards,

LipTeck

[LipTeck CHEW]

Hi Simon

We have just upgraded NSS from v3 to v4 on 10-May-2020. Hope it is not NSS version dependent.

regards,

LipTeck

[Simon Biggs]

That debug file is perfect :). All good, we've got integrity 4 also.

Could you run:

pip install pymedphys==0.30.0dev4

and let me know if that issue has been fixed :).

Cheers,

Simon

On Mon, 1 Jun 2020 at 16:39, LipTeck CHEW <liptec...@gmail.com> wrote:

Hi Simon

We have just upgraded NSS from v3 to v4 on 10-May-2020. Hope it is not NSS version dependent.

regards,

LipTeck

On Mon, 1 Jun 2020 at 14:29, LipTeck CHEW <liptec...@gmail.com> wrote:

Hi Simon

Please see the attached text file.

regards,

LipTeck

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkODbrKcGoi3PE1Ur2nmX6NdPHSQ3yOGMbFSAKb2edKFuA%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

I have managed to install but:

1. the linacs were shutdown so the services are not able to run. I'll check that again tomorrow

2. the gui was not able to launch

Please see details attached.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

So that bug never sees the light of day again would I be able to get a bit of data from you? Should it be fixed, would you be able to run a warm up beam or service mode beam (one that doesn’t have any patient data in it) and then, within the patients directory find the corresponding .xz file (under that particular dummy patient name). And I would be keen for you to upload this file for use within PyMedPhys’ automated testing suite. This will make sure any iCOM features that are particular to that particular file will always be verified as being able to be decoded with each code submission to PyMedPhys.

 

In preparation for uploading I’d like you to take a scan of the contents (so that you are happy with what you are uploading). To read the file, you will need to first extract it.

 

If you run “ipython” in the directory where the .xz file exists then you can run the following code within ipython:

 

In [1]: import lzma

In [2]: import pathlib

In [3]: icom_paths = pathlib.Path('.').glob('*.xz')

 

In [4]: for icom_path in icom_paths:
   ...:     with lzma.open(icom_path, "r") as f:
   ...:         contents = f.read()
   ...:
   ...:     with open(icom_path.with_suffix(".txt"), "wb") as f:
   ...:         f.write(contents)

 

This will then convert all of the xz files to .txt files. You can then open the .txt files with something like VSCode (https://code.visualstudio.com/). When you open the text file with VSCode you will need to press “Do you want to open it anyway?”:

 

 

There will be lots of “boxes” and weird characters, but all patient identification information will appear in sections that look like the following:

 

Once you’re confident that the xz file doesn’t have any patient information within it would you then be able to use the following link to upload the .xz file to the Zenodo repository:

 

https://zenodo.org/deposit/new?c=pymedphys

 

If you have issues with Zenodo, feel free to email that .xz file on this mailing list and I will upload it.

 

Thanks LipTeck,

Cheers,

Simon

 

 

 

From: pyme...@googlegroups.com [mailto:pyme...@googlegroups.com] On Behalf Of LipTeck CHEW
Sent: Monday, 1 June 2020 8:08 PM
To: Simon Biggs
Cc: PyMedPhys
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkNmLBMu4vgqAZB60HCHps0KEq_Vwbyfu0NHf-vho_bmZw%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

We have rebooted the physicspc this morning.

Updates on the icom listen --debug 

- the listeners are running but not generating the xz files

- this is confirmed with the _log.txt are updating the time stamp

- the gui is now able to launch. I suspect the physicspc could be running on low resources

- please see attached

Updates on TRF:

- We have a new cronjob running every minute to get the new TRF files from both NSS to live folders in our NAS:

\\Fph-nas01\nas1\VHD\TRF\Live

\\Fph-nas01\nas1\INF\TRF\Live

I just need a robocopy script at the physicspc to bring them incrementally to a single directory:

D:\pymedphys\TRF\to_be_indexed

If you have a new development version, we can install and test during the lunch time today SG time 1230-1330.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

That's pretty cool with the TRF files. It'll be nice to see it all up and running. Once all this Covid stuff is done maybe I should come visit :). I do have an uncle in Singapore...

Glad to hear the GUI is now launching, that was confusing me as to what the issue was there. I have produced a new development version, the only thing changed that affects your icom setup is that I have made it produce more debugging information to help me see what's going on within the logs you're providing.

Would you be able to run the following and then provide the logs for running the listener in debug mode:

pip install pymedphys==0.30.0dev5

Cheers :),

Simon

--
You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMxcVypmC8UXUbYp7i-4oKiups32tXr3xdsd8w3ws3_CA%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Yes, sure!

Yes, now we can see there are some text files generated in

D:\pymedphys\iComlogfiles\live\172.31.255.2

Let me know when we can run the orchestration CLI. I can manually copy over some trf over to

D:\pymedphys\TRF

and test this CLI.

Cheers,

LipTeck

[Simon Biggs]

What issues were you having with the orchestration CLI again?

--
You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkNsztG2%2B0vWT16AKZs6wCNX2BNcgB6BLh2K9Y4Txx3X5w%40mail.gmail.com.

[Simon Biggs]

Also, I'd be quite keen to be able to include one of those .txt files within the PyMedPhys testing suite if you're in a position to share it :).

Cheers :),

Simon

[LipTeck CHEW]

Hi Simon

I have tried resuming the iCom Listening Services manually during the day but they have continued to stay in the "pause" mode. Please see the windows popup message.

regards,

LipTeck 

[LipTeck CHEW]

Hi Simon

Is there any solution to the timezone error? Singapore is at the GMT +8. Are there alternatives to replace "Asia/Singapore" to enable the CLI to run first?

Thanks :)

regards,

LipTeck

---------- Forwarded message ---------
From: LipTeck CHEW <liptec...@gmail.com>
Date: Tue, 2 Jun 2020 at 14:36
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

To: Simon Biggs <m...@simonbiggs.net>

Hi Simon

Please see attached orchestration error text file.

regards,

LipTeck

[Simon Biggs]

When you created the service did you get it to create logs of its output? What does the iCOM logs say?

--
You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkOJ24b7YPHqQdvd%3Dj1x4P_A62LR7vjq3hDBe4O0u_nPeg%40mail.gmail.com.

[Simon Biggs]

Hi LipTeck,

 

I can reproduce your error with the file you provided by running the following:

 

 

It appears the issue is due to the spaces within the csv file.

 

I have reattached the config file you provided just with those spaces removed. Let me know if that fixes your issues.

 

-->

 

I’ll need to update the docs (https://docs.pymedphys.com/ref/cli/logfile.html#config-mosaiq-sql-csv). It seems my best intentions to make the displayed csv files be human readable resulted in an issue. I suspect I might go down the path of using the `.pymedphys/config.toml` file for this configuration also. I think that might be a more robust and less segmented configuration choice.

 

Make sure to keep an eye on the release notes (https://docs.pymedphys.com/release-notes.html) whenever you upgrade just to be abreast of what parts of your set up might be adjusted. Because the version number is still 0.*.* PyMedPhys is still going to be going through quite a few adjustments before its configuration API (and other APIs) stabilise.

 

Cheers,

Simon

[LipTeck CHEW]

Hi Simon

re: iCom Listening

For now,  I have opened two anaconda prompt and run the iComVHD.bat and iComINF.bat on each. It seems that it is generating txt log files with filenames from 000 to 255 in both directories

D:\pymedphys\iComLogFiles\Live\172.31.255.2

D:\pymedphys\iComLogFiles\Live\172.31.255.52

I can see that they are growing :) and I'll test the gui to read it later.

I think the reason the iCom listening services not running properly was due to the python environment / path wasn't there when nssm run the batch scripts.

How do we fix this ?

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

Have a look at the following file:

 

https://github.com/CCA-Physics/physics-server/blob/8dbe7d6c14c4cc8f940bcaf9d72e149074a279fe/NBCC/icom/4299_listening.bat

 

There, within the BAT file, I am adding python and pymedphys to the path before calling pymedphys.

 

Cheers,

Simon

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Wednesday, 3 June 2020 10:54 AM
To: Simon Biggs
Cc: Simon Biggs; PyMedPhys
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon

[LipTeck CHEW]

Hi Simon

re: config_mosaiq_sql.csv and config_linac_details.csv

I have removed the all the space before and after the comma in these files.

re: orchestration

It has managed to copy the SDD zip files out but looks like they have gone to the "already_indexed" folders but the "to_be_indexed" folders are empty. Please see attached text file and screenshot.

By the way, the orchestration command has created two folders: 3316 and 4181 for the linacs. Do I need to seperately robocopy the minutely new trf files into those folders?

 

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

Could you install the following development release and try again:

 

pip install pymedphys==0.30.0dev6

 

In your robo copy, you can just dump trf files anywhere within “to_be_indexed” it’ll deal with that. No need to split them up by linac.

 

Cheers,

Simon

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]

Sent: Wednesday, 3 June 2020 11:11 AM
To: Simon Biggs
Cc: PyMedPhys

Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon

[LipTeck CHEW]

Hi Simon

Yes, the iCom listeners are running finally after adding the path to python in the batch scripts. This is confirmed with the new time stamps of the text files 000-255.

Yay!

regards,

LipTeck

[LipTeck CHEW]

Hi Simon

I have installed the v0.30.0dev6

Ran the script looks ok but now I need a user to access Mosaiq db_datareader at Mosaiq SQL.

I have just requested it and pending reply.

regards,

LipTeck

[LipTeck CHEW]

Hi Simon

1. How do we configure the icom patients directory since the grouping feature is already built-in?

2. Gui unable to resolve a plan whether the start angle was 180 or -180
File "c:\users\physics\miniconda3\lib\site-packages\pymedphys\_utilities\transforms\bipolar.py", line 43, in convert_IEC_angle_to_bipolar
    ), "Unable to automatically determine whether angle is 180 or -180"

3. Is the gui able to read Monaco wedge plans?

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

Make sure to change the 'icom' config item to look like the following:

https://github.com/pymedphys/pymedphys/blob/d33d72a0a0258bacdf3e713a65822e4565f20f26/examples/site-specific/cancer-care-associates/config.toml#L67-L72

If you only have one patient directory just create a list with one item in it.

With not being able to resolve a plan, you'll need to potentially anonymise that one and send it through to me to have a see.

Regarding wedge plans, no not at the moment :(. If we do manage to get a few service contracts though I expect development time might be able to be sent that way ;)

Cheers,

Simon

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMXj9_mVG75m%2BZsD4%3DnW1-a3F4ZctLxM7WKDgHwTvsrBw%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

I have just corrected in the config.toml

[icom]

patient_directories=['D:\pymedphys\patients']

The gui is running ok without complain on the patient_directories.

Would the listeners know the icom patient_directories which has just updated?

The text files 000-255 are running ok but without anything generated in the 

D:\pymedphys\patients

regards,

LipTeck

 

[Simon Biggs]

I expect the path you're after is D:\pymedphys\iComLogFiles\patients

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMWLxRT3p0TFY77sGdTbvcUF3zHWA1UB5-C-Srm4c5GPQ%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Re: iCom logging 

After setting the patients folder to

D:\pymedphys\iComLogFiles\patients

I have stopped and started the listeners. 

Observing the iCom log, the only quick beam delivery at VersaHD managed to create a folder named

Deliver Quick Beam_Deliver Quick Beam,

with ×z files inside but not from Infinity linac that treated a patient. 

Is there anything missing?

Re: Mosaiq db reader user account 

Did you have to request one from Elekta or simply create this user within the Mosaiq application for this purpose?

Thank you. 

Regards 

LipTeck 

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CAALqB0Djd%3DdFuoS3uvX7TtWL8VYqBSFefsCmxQV10zo7xeArtA%40mail.gmail.com.

[Simon Biggs]

I think you'll need to run the listener in debug mode for the infinity and delivery a beam and then see what the logs say. Make sure to turn off the service listener when you are running the listener yourself via the terminal.

With Mosaiq access, Mosaiq comes with a default user called "reportsuser" if you ask Elekta for the default password for that one that will work. But yes, at one of our sites we simply made a "physics" user, and gave it the appropriate data reader permissions. So either of those options will work.

Sounds like we're almost there! :).

Cheers,

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkNaJjmK8TNO8ypNOHZgA79hJb_fT%2BQwy7yZVhH_MztFVQ%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon 

I’ve noticed the listener was able to show the patient name in debug mode. The live files were updating, but they were not grouped into the patients folder. 

However, beam on in the service mode of TCS managed to create folders in it with xz files. 

Regards 

LipTeck 

[Simon Biggs]

That’s interesting. Are you able to deliver with Mosaiq your morning warm up beams or something like that? Or deliver a beam with a dummy patient, and then provide me the debug logs during that delivery?

 

By the way, in order for the file to be archived the beam needs to finish, and then the beam needs to be exited out of within Mosaiq.

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Thursday, 4 June 2020 1:38 PM
To: Simon Biggs
Cc: PyMedPhys; Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon 

[LipTeck CHEW]

Hi Simon

Is it just the output of this CML

pymedphys --debug icom listen 172.31.255.2 D:\pymedphys\ >debug0406.txt

regards,

LipTeck

[Simon Biggs]

Yup, if that's the Linac that's not showing patient values. I'm interested as to why you're not pointing to your iCOM directory as you were with your previous command...

[LipTeck CHEW]

Hi Simon

The CLI did not manage to write to a text file but I managed to copy the last few lines.

Base on debug at VersaHD 172.31.255.2,

We could see when the morning qa patient were loaded, but patient name became blank during beam on, and re-appear after beam-off, blank again after unloaded. No folders created in patients folder. Loading a 5x5 quick beam has shown same behaviour but a folder was created in the patients folder.

During patients treatment at both linacs, we could see names on and off in the debug CLI but no folders created.

regards,

LipTeck

On Thu, 4 Jun 2020 at 16:42, Simon Biggs <m...@simonbiggs.net> wrote:

All good, probably can't help just now. I'll let you send through the logs on the mailing list, and see how we go. It does feel like we're getting there. Lots of hurdles though. Thanks for hanging in there :).

On Thu., 4 Jun. 2020, 6:31 pm LipTeck CHEW, <liptec...@gmail.com> wrote:

Sorry, just saw this. Yes, I am available.

On Thu, 4 Jun 2020 at 14:57, Simon Biggs <m...@simonbiggs.net> wrote:

Actually can we just quickly sort this out on a video call? Could you jump on https://whereby.com/pymedphys ?

[Simon Biggs]

Hi LipTeck,

Could you provide a copy of the debug log when the beam turns off and the treatment is closed within Mosaiq.

Cheers,

Simon

---

From: pyme...@googlegroups.com <pyme...@googlegroups.com> on behalf of LipTeck CHEW <liptec...@gmail.com>
Sent: 04 June 2020 18:52
To: Simon Biggs <m...@simonbiggs.net>; PyMedPhys <pyme...@googlegroups.com>

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkOr0kVuDiBfiOgxWnZugi_b%3D1mBG%3D-vuc5L2SQF1d0Sbw%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

We have to do this on the Infinity Linac 172.31.255.52

It seems that the moment the morning qa patient was loaded, the patient name appeared. During beam on, it disappeared but appeared again and then disappeared, marked by two lines before and after beam on.

No folder created in patients directory.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

I have created a new dev version where the only changes that affect icom listening is that I have added a bucket more debug logging messages in and around the patient archiving logic. If you could install the new dev version and then provide me the CLI output logs that encompass two different morning run up beams (ones without patient info).

 

Install new version with:

 

pip install pymedphys==0.30.0dev7

 

Then run your icom listen in debug mode. That would be really helpful.

 

We’re getting there :).

 

Thanks LipTeck :),

Simon

[LipTeck CHEW]

Hi Simon

We warm up the machine VersaHD and Infinity then performed measurement at VersaHD followed by Infinity again.

Please see attached sequence and manually copied debug logs.

regards,

LipTeck

[Simon Biggs]

Did you run it in debug mode with the latest version?

[LipTeck CHEW]

Yes, It’s v 0.30.0dev7

 with —debug —verbose but still unable to write to a text file. I had to manually copy from the CLI and paste to a notepad file. 

[Simon Biggs]

Ahh, can you not run –verbose, and just run –debug.

[LipTeck CHEW]

The streams were going very fast. Manually, that's the best I could do to capture a few lines.

[Simon Biggs]

Were you actually beaming on during those logs?

[Simon Biggs]

Actually, I see that you were beaming on as there are multiple iCOM streams per second, that, in my experience, only occurs during beam on time. The issue appears to be it’s not picking up the Patient ID. When you get a line like the following:

 

DEBUG:root:Saved stream to D:\pymedphys\iComLogFiles\live\172.31.255.2\054.txt

INFO:root:IP: 172.31.255.2 | Timestamp: 2020-06-0510:11:38 | Patient ID: None | Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

 

Can you stop the logging, and send me that .txt file? I would like to be able to upload that stream to be an item tested within PyMedPhys. I need to get to the bottom of why it can’t read the Patient ID within that file.

 

Cheers,

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/SYCP282MB05412FE99DD3692ED39D9D4095860%40SYCP282MB0541.AUSP282.PROD.OUTLOOK.COM.

[Simon Biggs]

Oh wow, okay. Can you try deliver a beam that does have an ID? I may need to see if there is a way I can handle “no-id” beams. Currently the software will not be able to handle a beam that does not have an ID defined. (It groups beams by Patient ID...).

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Friday, 5 June 2020 1:45 PM
To: Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

I have just realised there was no id. Please see screenshot attached.

However, our patients has FP numbers with prefix FP and 8 digits. 

e.g. 

FP12345678

FP00005678

FP12340000

 

 

 

On Fri, 5 Jun 2020 at 11:25, Simon Biggs <sbi...@riverinacancercare.com.au> wrote:

What is the actual patient id for that plan?

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Friday, 5 June 2020 1:19 PM
To: Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon

 

054.txt attached.

 

regards,

LipTeck

 

 

 

On Fri, 5 Jun 2020 at 11:04, Simon Biggs <sbi...@riverinacancercare.com.au> wrote:

All good, I’m not after the CLI log, actually importantly, the .txt file that is saved within the live directory:

 

D:\pymedphys\iComLogFiles\live\172.31.255.2

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Friday, 5 June 2020 1:02 PM
To: Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon,

Yes, We've loaded the morning QA patient, beam on, a 6x followed by a 10x. The lines went really fast. 

 

I saw the format in the format below. Omitting the lines on the 000-255 files:

 

Mosaiq beam loaded:

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

 

Beam on:

| Patient Name: None | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

 

during beam on:

| Patient Name: None | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

 

beam off:

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

| Patient Name: Morning QA  2018, Versa | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

| Patient Name: None | Machine ID: VersaHD

 

it's too fast to catch the complete log

 

regards,

LipTeck

[LipTeck CHEW]

Sorry that the morning qa patients were created within Mosaiq without any ID. However, 

All our patients have 3 IDs in our Mosaiq 

1. FP:

2. Id #:

3. RTN1:

Among these three IDs, only FP is unique.

Could you map FP to Patient ID and then group?

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/SYCP282MB05414D85600D2D9C67B252B295860%40SYCP282MB0541.AUSP282.PROD.OUTLOOK.COM.

[LipTeck CHEW]

Sorry that the morning QA patients has no ID. However, all of our patients have three IDs but the only unique ID is the FP. 

1. FP:

2. Id #:

3. RTN1:

Are you able to map Patient ID = FP?

[Simon Biggs]

Unfortunately I can only make use of what the iCOM stream provides. If you open up those text files with VSCode you can search for whether or not any of those IDs appear within that file.

[LipTeck CHEW]

Hi Simon 

I have used a test patient named TG119 with FP and the iCom log is finally working ok and a folder was generated in the patients folder. 

I’ve added FP information to the morning QA patient and also started running the iCom listeners services. Hope to be able to share more later today. 

Regards 

LipTeck 

On Sat, 6 Jun 2020 at 11:17 AM, LipTeck CHEW <liptec...@gmail.com> wrote:

Hi Simon

re: pymedphys icom listen

Sorry that we were occupied with a few new cases.

I share with you one of the Mobius elog file in year 2019 for a pelvis vmat case. I have edited the PatientName, PatientID, and TreatmentName. I have excluded the ebin file that has the actual patient information.

The elog file name begins with time stamp followed by patient Sirname and then separated by comma without space with first name and then one space with FP number in bracket. It is able to read the FP  as the PatientID. Therefore, I think this information is available in the iCom. 

It is interesting that the Mobius log use the time stamp as the start of the filename and then adds a bracket to the PatientID to end the filename:

"2019-07-03, 11-18-45, Sirname,Firstname (FP12345678).txt"

Perhaps in this way, it is able to handle blank fields. So, a Morning QA 2018, Versa patient without FP would look like this:

"yyyy-mm-dd, tt-hh-ss,  Morning QA 2018,Versa ().txt"

Generally Mobius logs everything but only logs that have PatientID, FieldID, and TreatmenName are useful for the MobiusFX. Perhaps combining PatientID, FieldID and TreatmentName together with something to create a ReferenceID for regrouping will make the icom listener much more robust.

re: pymedphys orchestration

The new user account for Mosaiq db_reader will be provided on next Tueday 9-June-2020.

have a good weekend!

regards,

LipTeck

[Simon Biggs]

Brilliant, that’s exactly what I was hoping to see :).

 

Cheers,

[LipTeck CHEW]

Hi Simon

re: icom listen

The morning qa patients did not appear in the log. No successful grouping today. The only thing that has been successfully logged was the TG119, TG119 with ID FP: 000007 and Deliver Quick Beam_Deliver Quick Beam. I can see xz files accumulating inside this folder whenever we use quick beam at the TCS. There is one similarity: they are simple beams with open fields with short delivery time less than 1 minute. I attached the icom_listen.zip file that has the information below:

I think the icom listen CLI has been reading the FP number and necessary information but what are the other criteria that were not met causing it not to group? None of the patients were grouped. The difference of the actual patient treatment with the above TG119, TG119 and Deliver Quick Beam_Deliver Quick Beam is patient setup imaging and the total time of all possible beams under one prescription. Is there a limit of time between fields or total time for each prescription? 

Regarding the 000-255 files. Could you explain how the data is logged? Since the information is coming at 0.25 sec, how is the data recorded in these 256 files? What happens after 256 or 64 sec?

re: orchestration

We have managed to get an account to access the Mosaqi SQL but not able to index the trf files (see attached txt). 

Note the last line of the CLI:

FileNotFoundError: [WinError 3] The system cannot find the path specified: '

D:\\pymedphys\\TRF\\to_be_indexed\\20_05_26 00_02_07 Z 06_6MeV.trf' -> 

'D:\\pymedphys\\TRF\\no_mosaiq_record_found\\20_05_26 00_02_07 Z 06_6MeV.trf'

There was '\\' in each sub-directory. Is that a bug?

Thank you.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

Thank you for those testing files. Are you okay if I upload those to Zenodo for use within PyMedPhys' test suite?

Regarding the "FileNotFoundError" can you install via the following and then try again?

pip install pymedphys==0.30.0dev8

This hopefully will remove that "FileNotFoundError", but there is a bigger issue at play. The way Mosaiq is used to identify the trf log file is that it is aligned via timestamp. However, if it is not kept in check the timestamp of the linac can drift and no longer be in agreement to the Mosaiq server. Under this identification method, the Linac's time needs to agree with the Mosaiq sequencer time to within 1 minute. Given you're getting the following error:

pymedphys._mosaiq.delivery.NoMosaiqEntries: No Mosaiq entries were found for 06/6MeV at 2020-05-26 08:02:07

I suspect the Linac time is not currently aligned with your Mosaiq server. Please let me know if that is the case, and then if so see if you can get your engineer to adjust the Linac time to agree with the Sequencer.

Cheers,

Simon

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPbsKVFokEc3npnbUwLGqO4iFYEQJrAQj9DDTTrrGOHDg%40mail.gmail.com.

[Simon Biggs]

Regarding your other questions:

I think the icom listen CLI has been reading the FP number and necessary information but what are the other criteria that were not met causing it not to group? None of the patients were grouped. The difference of the actual patient treatment with the above TG119, TG119 and Deliver Quick Beam_Deliver Quick Beam is patient setup imaging and the total time of all possible beams under one prescription. Is there a limit of time between fields or total time for each prescription? 

I would need to see the debug logs during the deliveries to be able to get an understanding of what was going on here. Data is recorded/not recorded entirely based upon whether or not that "patient_id" parameter is found. See the following code:

https://github.com/pymedphys/pymedphys/blob/a94ec69297ce482e5ade33e2bc32a1b144bc0967/pymedphys/_icom/patients.py#L150-L186

Essentially if the previous run had no patient id, and this stream has one, then start a fresh record and begin recording. If the previous run had a patient id, and the current run does not, then presume the delivery has finished and save all of the recorded streams to disk.

Regarding the 000-255 files. Could you explain how the data is logged? Since the information is coming at 0.25 sec, how is the data recorded in these 256 files? What happens after 256 or 64 sec?

Those 256 files are not actually currently used. The data for each patient is stored within program memory. The live folder was designed in that way to facilitate other tools listening to changes within that directory and responding to it, without needing to spin up another listener. One use case I have in mind for those live files is a live feedback iView movie frame tagging tool, to allow live feedback on coll angle, gantry angle, etc and automatically tag the frames of an iView movie for use within a dynamic Winston Lutz Arc.

Cheers,

Simon

[LipTeck CHEW]

Hi Simon 

Thanks for your explanation on the iCom listening process. Hope there’s a solution or workaround. 

Regards 

LipTeck 

[Simon Biggs]

My pleasure :) I do have a favour to ask. Once this is all over, would you be interested in collating your installation notes into a document? I can then upload that document to help guide others through what you’ve achieved :).

 

Let me know if you’re interested.

 

Cheers,

Simon

 

 

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Wednesday, 10 June 2020 9:24 AM
To: Simon Biggs
Cc: PyMedPhys; Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon 

[LipTeck CHEW]

Yes, of course. Actually, I have started journaling it. 

I have another idea is to use the raspberry pi 4B 8GB as a server. 

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/SYCP282MB0541A6E67CEBA4D5D558C74795820%40SYCP282MB0541.AUSP282.PROD.OUTLOOK.COM.

[Simon Biggs]

That would be quite cool. In the past I have thought it would be quite neat to put together pre-built raspberry pi’s for people, maybe have a little “PyMedPhys” shop (it could sell shirts too :) ). If you like that idea, it might be interesting setting something up together.

[LipTeck CHEW]

Sounds like a good idea. The upcoming regional meeting is the AOCMP and SEACOMP at Phuket this October. Hope it’s in time to send in a paper. 😃

[LipTeck CHEW]

Hi Simon 

Looking at the debug lines, I can see that the patient’s name is there but not the ID. 

I suspect the code is not reading the ID if there’s alphabet in it. Our TG119, TG119 has FP: 000007 but all our patients has FP that begins with prefix “FP” eg

FP12345678

FP87654321

etc. 

Hope this is the issue. 

Regards 

LipTeck 

[Simon Biggs]

Are you able to make a dummy patient that has the same ID format as your clinical patients and reproduce the effect in logs that you are able to share?

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Wednesday, 10 June 2020 11:16 AM
To: Simon Biggs
Cc: PyMedPhys; Simon Biggs
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon 

[LipTeck CHEW]

Yes, I can do that.

[LipTeck CHEW]

Hi Simon 

I’ve used the morning QA, TG119 and quick beam as test patients. 

Results:

1. Morning QA: patient ID was none with Both FP as FPMQA02018 and then 1533162018, delivered in QA mode

2. TG119: patient ID was detected as 000007, delivered in QA mode, grouping ok

3. Quick beam: patient ID was Deliver Quick Beam, grouping ok

Looks like my theory was incorrect. We’re back to a junction.  🤷‍♂️

Regards 

LipTeck 

[LipTeck CHEW]

Hi Simon 

re: orchestration

After running this command, it ended without a break. 

We have managed to sync VersaHD. We’ll sync Infinity this Thursday evening. Does that mean all previous trf not useable? Could you relax the code indexing so that these files can be used ? Is there a manual way to fix these trf file so that they could be indexed?

The to_be_indexed folder is now empty. 

The no_field_label_in_logfile folder seems to have captured mainly the service beams such as quick beam, kV flexmap, etc. Would a CBCT produce a trf file?

The no_mosaiq_record_found folder seems to have captured our morning linac warm up beams, output beams, and patients setup beam trf. 

Any comments?

Regards 

LipTeck 

[Simon Biggs]

The code at the following location is the part that would need to be adjusted:

https://github.com/pymedphys/pymedphys/blob/32fc4275251659ceef68bdb49aab7da6245696d5/pymedphys/labs/managelogfiles/index.py#L127

Potentially some of those options could be provided as options via the command line interface. I could make an offset and buffer parameter within the CLI that can be used to adjust the alignment. ... Might even help to have a "also-search-previously-unindexed" to flag to also search within the "no Mosaiq entry found" directory...

I'd need to make those changes though. Do you know what the offset was between the Linac and Mosaiq before?

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkOaOGEnUeA_7vcyGqX2jb62JVuAnA4RVjHCZ%3DOCOvxXHg%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

The time difference was about 1 minute and 35 seconds. 

Regards 

LipTeck 

[Simon Biggs]

You could open that python file within your pymedphys install on your computer and change that 240 to about 600 or something and that might fix it. As a stop gap work around...

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkP9KdTpYPrR%3DdnVURzMiTWN5Q-7sL%2BtL66gS%2BK9Jh8R8g%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Please see the attached prefix_investigation.zip file.

regards,

LipTeck

[LipTeck CHEW]

Just to share the iCom interlock. 

On Fri, 29 May 2020 at 9:56 AM, Simon Biggs <sbi...@riverinacancercare.com.au> wrote:

Hi LipTeck,

 

As a first point, the iCOM command line tool recently underwent a small revamp. See https://github.com/pymedphys/pymedphys/releases/tag/v0.28.0 for more details. The GUI itself has undergone a change too, see https://github.com/pymedphys/pymedphys/releases/tag/v0.29.0.

 

As such, I would recommend updating to pymedphys==0.29.1 before implementing the feedback below.

 

Also, given this is going down the path of implementing the iCOM listener please be aware of the following bug that appears to plague all iCOM listener tools (commercial tools included) https://github.com/pymedphys/pymedphys/issues/849.

 

Cheers,

Simon

 

 

 

 

Hi Simon,

 

I have installed pymedphys and ImageMagick both in 64 bits Windows 7 and 64 bits Ubuntu 20.04 LTS Desktop versions running good in demo mode. I am now trying to setup in the actual clinical environment. Our department is using full Elekta Solution: Mosaiq v2.64, Monaco v5.11.03 with 2 Agility Head Linacs: 1 VersaHD and 1 Infinity.

 

 

1. I am confused with iComLogFiles directory and the iComLogFiles\live directory described

https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml

May I know what the differences are?

Both the live and the patients directories within the iCOM directory are for storing iCOM data. The live directory is a simple .txt dump of the most recent iCOM stream records. These files a regularly overwritten. The patients directory is where iCOM stream data is grouped and indexed by patient.

What are the recommendation for the location of logfiles, live and grouped patient directory: local or NAS?

I would have the iCOM files be stored on the same machine that is running the iCOM listener, and then have these files shared via the network so that they are accessible by the machine that is running pymedphys gui.

2. In the example of listeners at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, both batch scripts consolidate the logfiles from two linacs' NSS to D:\iComLogFilesThen, in the patient archiving (grouping) it does something using D:\iComLogFiles\live and Z:\iCom\live to produce something at D:\iComLogFiles\patients

May I know what is “Z:\iCom\live” ? 

Is Z:\ a local drive or network drive?

Z:\ is a network drive shared via intranet from another site. We run this tool for 4 Linacs at 3 sites.

Can a single computer listen to both NSS simultaneously or we need to deploy one listener per computer for each NSS?  

A single computer can listen to both NSS’. We have three computers doing it since there is one computer at each site and I don’t want the internet connection to cause the dropout issues discussed above.

 

3. In the process of grouping patients at https://nbviewer.jupyter.org/url/simonbiggs.net/icom.ipynb, there is a line: “This is done by setting up a .bat file to run on machine boot”. 

Could you elaborate further? 

Is the machine here referring to the linacs or physics-server/pc? 

Do you mean patient archiving (grouping) is done daily in the morning?

Actually, if you install pymedphys 0.29.1, this archiving tool is no longer needed (and the CLI interface has been removed). It is now undergone automatically by the listener process already. No need to manage any of that part yourself any longer.

 

3.       Regarding the command line tool at https://docs.pymedphys.com/ref/cli/logfile.html: pymedphys logfile orchestration [-h] [-m MOSAIQ_SQL] [-l LINAC_DETAILS] data_directory (orchestration.bat)

May I know whether this data_directory referring to ~.\pymedphys\data or D:\iComLogFiles\patients or the TRF directory?

That data directory is actually where ever you would like it to be... I personally put it on the D drive of a server’s local machine where that D drive points to a rather large spinning disk storage. The TRF files end up taking quite a lot of space.

Which data directory should we configure?

In your config.toml point to the same directory as written within the logfile orchestration tool.

Do we need to map network drive to each separate NSS with the rtduser?

No, don’t map the NSS network drive, but do save the rtduser password within the Windows credential manager:

 

Instead of writing “tunnel-sash-9002”, instead write in the IP address of the NSS on both the network address section, and the username section.

4.       How do we enable reading all clinics or designated clinics appear under the \\FocalData\Installation directory described at https://github.com/pymedphys/pymedphys/blob/master/examples/site-specific/cancer-care-associates/config.toml? We have several folders for different groups of doctors.

Unfortunately this is not currently directly supported at the moment, I might be able to make this possible. I do wonder if it might “just work” if you treat each clinic as its own “site” within the config file, as in create a new site for each clinic, and have all the configuration be the same except change the clinic path. It would be nice to directly support this use case though.

5.       Do we need the D:\iComLogFiles\patients for the gui to read trf ? What are the configurations required just to make the trf portion work?

Nope, you don’t need that patient directory if you’re going to be just using TRF. However, now that patient indexing is included within the listener maybe this is not as much of an issue?

Would the trf files from two NSS override each other?

They shouldn’t. The TRF files are organised by IP address. Let me know if you have any issues here though.

Thank you for your patience :)

My pleasure :) Things are quite hectic at the moment, thanks for baring with response delays :). Feel free to let me know if you have any more issues.

warm regards,

LipTeck

[Simon Biggs]

Out of interest, when the Linac is delivering these beams what patient ID appears on the NRT monitor? (As in, what patient ID does the Linac appear to be assigning to the patient).

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMhozOjo3T9OYov-SkaTR4%2BDL%2BeyoHyF9-XrEG_N2NGmw%40mail.gmail.com.

[Simon Biggs]

Hi LipTeck,

Would you be able to run that failing case again (as attached), and copy and attach the live files that correspond to the delivery? Can you make sure to grab all the live files that correspond to the delivery as well as 10 or so files from before the patient is even opened in Mosaiq, all the way through to about 10 or so files after the patient is completely closed, without missing any in the middle. Easiest way to do it is to probably sort by modified date within the live directory and copy the most recent files across into a separate directory every 10 seconds or so. If you could also send the corresponding debug log, that would be really helpful.

I'll then use that stream to set up a "mock Linac" that sends out those streams on the iCOM port and, then I'll listen to that mock stream with the CLI, reproduce what you're seeing on my machine, and then that can help me iterate working out how to solve this issue, as well as add that "mock Linac" to the PyMedPhys testing suite once I'm done.

Cheers,

Simon

[LipTeck CHEW]

Patient ID = FP ID

Same on both screens.

[Simon Biggs]

Kk, thanks LipTeck. I think the key to finding a solution will be me making up that mock Linac on my end with the live data when you get a chance to provide it.

Cheers :),

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPPxGcoCi5ZDimzMZf0hnkcgbukx1fJPX1Dmc9OWzUi_A%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon 

Is there a simple method to TRF works without orchestration ? Something like upload or browse trf files or zip file or folder that contains a few TRF ?

In this way, there’s a workaround and also troubleshooting for GUI to work with a case manually without the need to wait for an account for db_datareader or problems of time difference?

Regards 

LipTeck 

[Simon Biggs]

Hi LipTeck,

Not at the moment unfortunately. Given the GUI was built with our centre in mind under the assumption that TRF orchestration was all working nice and smoothly.

It certainly could be implemented though. The TRF input method for the GUI is all defined at the following location of the following file:

https://github.com/pymedphys/pymedphys/blob/2d5148e2eabce3a6a4fd54e43e7dc8d4e050f5ed/pymedphys/_gui/streamlit/mudensity-compare.py#L805-L906

And the following function can be passed the contents of any trf file and it will return the header and contents as two pandas DataFrames:

https://github.com/pymedphys/pymedphys/blob/2d5148e2eabce3a6a4fd54e43e7dc8d4e050f5ed/pymedphys/_trf/trf2pandas.py#L26-L36

That function is exposed as part of the PyMedPhys API:

https://github.com/pymedphys/pymedphys/blob/2d5148e2eabce3a6a4fd54e43e7dc8d4e050f5ed/pymedphys/__init__.py#L8

And it can be called by running `header, contents = pymedphys.read_trf(trf_filepath)`

So theoretically, with a few changes to the GUI file linked above, you could make it so that any trf file could be selected using the file_upload widget. This could be done in a way which is similar to the dicom file option -- https://github.com/pymedphys/pymedphys/blob/2d5148e2eabce3a6a4fd54e43e7dc8d4e050f5ed/pymedphys/_gui/streamlit/mudensity-compare.py#L536-L554

If you wanted to implement that as an option within the GUI I would be more than happy to lend a hand :). To begin you would need to set up a PyMedPhys developer environment -- https://docs.pymedphys.com/howto/advanced/setup-win.html

Cheers,

Simon

[Simon Biggs]

Hi LipTeck,

 

I imagine to be able to resolve these remaining issues in a timely fashion I would likely need to spend a consolidated block of time within work hours. Would you be open to working out a consultancy agreement with my employer so that I can be given the time + focus to fix these issues for you?

 

Let me know your thoughts,

Cheers,

Simon

 

 

 

From: pyme...@googlegroups.com [mailto:pyme...@googlegroups.com] On Behalf Of Simon Biggs
Sent: Saturday, 13 June 2020 5:36 PM
To: PyMedPhys
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi LipTeck,

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/6363e5ca-f67d-4d4b-9a34-33d75d344efao%40googlegroups.com.

[LipTeck CHEW]

Hi Simon

I am open to this. Please send me your proposal and the deliverables for me to approach the management. 

Thank you. 

Regards 

LipTeck 

[Simon Biggs]

Hi LipTeck,

Are you in a position to do a video call this afternoon to go over the details of where you're up to and what needs to be done? I recognise most things have been covered in this email list but it would be good to touch base before writing up a proposal. We can discuss times etc off list.

Cheers,

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMNicV5jcy_EXpSztHv_mzqGz-Co_CkpVA%3Dx3DzpVcKgQ%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon,

I need to check with you our configurations:

inside config_linac_details.csv

I think we should configure

,Centre,IP

VersaHD,Farrer_Park,172.31.255.2

Infinity,Farrer_Park,172.31.255.52

instead of

,Centre,IP

3316,Farrer_Park,172.31.255.2

4181,Farrer_Park,172.31.255.52

since the machine id recorded in Mosaiq are VersaHD and Infinity.

While 

inside the file config.toml, since it is accessing the NSS I'll use "VersaHD" and "Infinity" (instead of 3316 and 4181) since the IP were already defined in the config_linac_details.csv

[[site.linac]]

name = "VersaHD"

icom_live_directory = 'D:\pymedphys\iComLogFiles\live\172.31.255.2'

  

[[site.linac]]

name = "Infinity"

icom_live_directory = 'D:\pymedphys\iComLogFiles\live\172.31.255.52'

Hope this is correct.

regards,

LipTeck

On Wed, 3 Jun 2020 at 06:01, Simon Biggs <sbi...@riverinacancercare.com.au> wrote:

Hi LipTeck,

 

I can reproduce your error with the file you provided by running the following:

 

 

It appears the issue is due to the spaces within the csv file.

 

I have reattached the config file you provided just with those spaces removed. Let me know if that fixes your issues.

 

-->

 

I’ll need to update the docs (https://docs.pymedphys.com/ref/cli/logfile.html#config-mosaiq-sql-csv). It seems my best intentions to make the displayed csv files be human readable resulted in an issue. I suspect I might go down the path of using the `.pymedphys/config.toml` file for this configuration also. I think that might be a more robust and less segmented configuration choice.

 

Make sure to keep an eye on the release notes (https://docs.pymedphys.com/release-notes.html) whenever you upgrade just to be abreast of what parts of your set up might be adjusted. Because the version number is still 0.*.* PyMedPhys is still going to be going through quite a few adjustments before its configuration API (and other APIs) stabilise.

 

Cheers,

Simon

 

 

 

From: LipTeck CHEW [mailto:liptec...@gmail.com]
Sent: Tuesday, 2 June 2020 10:26 PM
To: Simon Biggs; PyMedPhys
Subject: Fwd: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi Simon

 

Is there any solution to the timezone error? Singapore is at the GMT +8. Are there alternatives to replace "Asia/Singapore" to enable the CLI to run first?

Thanks :)

 

regards,

LipTeck

---------- Forwarded message ---------
From: LipTeck CHEW <liptec...@gmail.com>
Date: Tue, 2 Jun 2020 at 14:36
Subject: Re: PyMedPhys Activity Report -- The new PyMedPhys GUI

To: Simon Biggs <m...@simonbiggs.net>

 

Hi Simon

 

Please see attached orchestration error text file.

 

regards,

LipTeck

 

[Simon Biggs]

Hi LipTeck,

That does look promising. The most recent version of pymedphys did however make a change to some of the config.toml layout. Let me get back to you on that. (Just about to sit down with my wife and watch Agent Carter) :).

Cheers,

Simon

--

You received this message because you are subscribed to the Google Groups "PyMedPhys" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pymedphys+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkMWrFDsn%3DFOr%2BtQOyn0gqMXy0xE6jPRUWL1sXch4E-kTg%40mail.gmail.com.

[Simon Biggs]

Hi LipTeck,

That should work. I thought the config.toml changes had landed as a release, but they're still just within the development branch. Before upgrading to 0.32.0 you'll need to check the changelog on how to update your config.toml.

Cheers,

Simon

[LipTeck CHEW]

Hi Simon,

iCom:

The iCom is still unable to catch the PatientID. So far, the only successful iCom log were the delivery in Service Mode (quickbeams, etc.) and the QA Patient TG119. All our actual patients were not able to log.

The main difference is all of our patients were imported into Mosaiq via HIS while the TG119 patient was manually registered in Mosaiq. For some reasons, it is not reading the PatientID. Any hope looking into:

https://github.com/pymedphys/pymedphys/blob/a94ec69297ce482e5ade33e2bc32a1b144bc0967/pymedphys/_icom/patients.py#L150-L186

Did you find anything in the livestream provided previously?

Orchestrate:

I've encountered messages:

Header(machine='3316', date='20/08/04 03:20:45 Z', timezone='+08:00', field_label='V1', field_name='Arc1')

Traceback (most recent call last):

  File "c:\users\physics\miniconda3\lib\site-packages\pymedphys\_trf\manage\index.py", line 164, in file_ready_to_be_indexed

    centre = machine_map[header.machine]["centre"]

KeyError: '3316'

I find the time is incorrect. Any comments? For a successful orchestration, what do we get to see?

If we can copy the Mosaiq iCom log files out to a NAS, would it be helpful?

Thank you.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

Just focusing on one of those points for now. Could you try calling the orchestrate function directly from within Python:

https://github.com/pymedphys/pymedphys/blob/9bfe1bab6c2f798e8ab674d65ae65c3ebd26d37c/pymedphys/_trf/manage/orchestration.py#L36-L64

That way you can play around with different inputs to get to the bottom of why the TRF orchestration isn't working.

Cheers,

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkOjVTOf4NXnM7GCfyGXTKGF78r%3DyLcda%2BXNtgaqRHk6DQ%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

Thanks for the advice. 

I've decided to comment out the fetch and extraction of the zip files in the orchestration.py since I have the live trf files in the NAS but I've found the error persisted. So I took 2 trf files, one from each linac and converted them to CSV. From here, I realised from the CSV header files, the machines were defined as 3316 and 4181 instead of VersaHD and Infinity. Is this causing any problems since we have configured the linacs as VersaHD and Infinity in the config_linac_details.csv and config.toml ?

I have also found that the trf file name and CSV header files have time information 8 hours behind thought the trf file time stamp is correct. We're going to check whether the linac is configured in the Singapore time zone though Intellimax is indicating SG. 

Please see attached screenshot, trf and CSV header files for your comments. Thank you.

regards,

LipTeck

[Simon Biggs]

Hi LipTeck,

 

To get to the bottom of the identifying issue, could you try calling the following function directly within Python:

 

https://github.com/pymedphys/pymedphys/blob/9bfe1bab6c2f798e8ab674d65ae65c3ebd26d37c/pymedphys/_mosaiq/delivery.py#L73-L170

 

That’s the function used to identify the TRF file internally, it takes the information from the TRF header and undergoes a Mosaiq query based on that header (and timezone offsets) in order to identify the TRF file.

[LipTeck CHEW]

Hi Simon

I have tried running the delivery.py inside spyder. Here's the result:

-------

Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)]

Type "copyright", "credits" or "license" for more information.

IPython 7.8.0 -- An enhanced Interactive Python.

runfile('C:/Users/physics/Miniconda3/Lib/site-packages/pymedphys/_mosaiq/delivery.py', wdir='C:/Users/physics/Miniconda3/Lib/site-packages/pymedphys/_mosaiq')

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\Lib\site-packages\pymedphys\_mosaiq\delivery.py", line 28, in <module>

    from .connect import execute_sql

ModuleNotFoundError: No module named '__main__.connect'; '__main__' is not a package

Loading KWallet

Loading SecretService

Loading Windows

Loading chainer

Loading macOS

Error in callback <bound method AutoreloadMagics.post_execute_hook of <autoreload.AutoreloadMagics object at 0x0000000003F0CA20>> (for post_execute):

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\extensions\autoreload.py", line 578, in post_execute_hook

    _, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\extensions\autoreload.py", line 184, in filename_and_mtime

    if not hasattr(module, '__file__') or module.__file__ is None:

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 238, in __getattribute__

    return getattr(getmod(), name)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 216, in getmod

    x = importobj(modpath, None)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 103, in importobj

    module = __import__(modpath, None, None, ["__doc__"])

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>

    from .base import CAP_STYLE, JOIN_STYLE

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\base.py", line 18, in <module>

    from shapely.coords import CoordinateSequence

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\coords.py", line 8, in <module>

    from shapely.geos import lgeos

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geos.py", line 145, in <module>

    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))

  File "C:\Users\physics\Miniconda3\lib\ctypes\__init__.py", line 356, in __init__

    self._handle = _dlopen(self._name, mode)

OSError: [WinError 126] The specified module could not be found

---------------

LipTeck

[Simon Biggs]

Hi LipTeck,

 

To run it you want to do the following:

 

import pymedphys

import pymedphys._mosaiq.delivery as delivery

 

with pymedphys.connect(mosaiq_hostname) as cursor:

    # get machine, delivery_time, field_label, field_name from the logfile header

    delivery.get_mosaiq_delivery_details(cursor, machine, delivery_time, field_label, field_name, buffer=240)

 

 

Cheers,

Simon

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPK4FPq_zUyVtBD%2BrApSP7YrnKvmhQUCD%3DY6NHTJtLyPg%40mail.gmail.com.

[Simon Biggs]

With pretty printing:

 

import pymedphys

import pymedphys._mosaiq.delivery as delivery

 

with pymedphys.connect(mosaiq_hostname) as cursor:

    # get machine, delivery_time, field_label, field_name from the logfile header

    delivery.get_mosaiq_delivery_details(cursor, machine, delivery_time, field_label, field_name, buffer=240)

 

 

From: Simon Biggs

Sent: Wednesday, 5 August 2020 3:39 PM
To: 'LipTeck CHEW'
Cc: Simon Biggs; PyMedPhys

Subject: RE: PyMedPhys Activity Report -- The new PyMedPhys GUI

 

Hi LipTeck,

 

To run it you want to do the following:

 

import pymedphys

import pymedphys._mosaiq.delivery as delivery

 

with pymedphys.connect(mosaiq_hostname) as cursor:

    # get machine, delivery_time, field_label, field_name from the logfile header

    delivery.get_mosaiq_delivery_details(cursor, machine, delivery_time, field_label, field_name, buffer=240)

 

 

Cheers,

Simon

 

 

To view this discussion on the web, visit https://groups.google.com/d/msgid/pymedphys/CA%2B5_vkPK4FPq_zUyVtBD%2BrApSP7YrnKvmhQUCD%3DY6NHTJtLyPg%40mail.gmail.com.

[LipTeck CHEW]

Hi Simon

The coded is looking for MacOS but our physics pc is on Windows. Should it be WinOS ?

Here's the result:

Error in callback <bound method AutoreloadMagics.post_execute_hook of <autoreload.AutoreloadMagics object at 0x0000000003F0CA20>> (for post_execute):

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\extensions\autoreload.py", line 578, in post_execute_hook

    _, pymtime = self._reloader.filename_and_mtime(sys.modules[modname])

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\extensions\autoreload.py", line 184, in filename_and_mtime

    if not hasattr(module, '__file__') or module.__file__ is None:

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 238, in __getattribute__

    return getattr(getmod(), name)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 216, in getmod

    x = importobj(modpath, None)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 103, in importobj

    module = __import__(modpath, None, None, ["__doc__"])

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>

    from .base import CAP_STYLE, JOIN_STYLE

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\base.py", line 18, in <module>

    from shapely.coords import CoordinateSequence

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\coords.py", line 8, in <module>

    from shapely.geos import lgeos

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geos.py", line 145, in <module>

    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))

  File "C:\Users\physics\Miniconda3\lib\ctypes\__init__.py", line 356, in __init__

    self._handle = _dlopen(self._name, mode)

OSError: [WinError 126] The specified module could not be found

regards

LipTeck

[Simon Biggs]

Hi LipTeck,

 

I’m not sure about spyder and what magic it is doing. Could you either try within Jupyter Lab or an ipython console prompt?

[LipTeck CHEW]

Hi Simon

I've just tried it in iPython:

(base) D:\pymedphys\config>ipython

Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)]

Type 'copyright', 'credits' or 'license' for more information

IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pymedphys

In [2]: import pymedphys._mosaiq.delivery as delivery

In [3]: with pymedphys.connect(mosaiq_hostname) as cursor:

   ...:     delivery.get_mosaiq_delivery_details(cursor,machine,delivery_time,field_label,field_name,buffer=240)

   ...:

ERROR:root:Internal Python error in the inspect module.

Below is the traceback from this internal error.

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3326, in run_code

    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-3-45879e5a5efe>", line 1, in <module>

    with pymedphys.connect(mosaiq_hostname) as cursor:

AttributeError: module 'pymedphys' has no attribute 'connect'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2040, in showtraceback

    stb = value._render_traceback_()

AttributeError: 'AttributeError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\ultratb.py", line 1101, in get_records

    return _fixed_getinnerframes(etb, number_of_lines_of_context, tb_offset)

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\ultratb.py", line 319, in wrapped

    return f(*args, **kwargs)

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\ultratb.py", line 353, in _fixed_getinnerframes

    records = fix_frame_records_filenames(inspect.getinnerframes(etb, context))

  File "C:\Users\physics\Miniconda3\lib\inspect.py", line 1502, in getinnerframes

    frameinfo = (tb.tb_frame,) + getframeinfo(tb, context)

  File "C:\Users\physics\Miniconda3\lib\inspect.py", line 1460, in getframeinfo

    filename = getsourcefile(frame) or getfile(frame)

  File "C:\Users\physics\Miniconda3\lib\inspect.py", line 696, in getsourcefile

    if getattr(getmodule(object, filename), '__loader__', None) is not None:

  File "C:\Users\physics\Miniconda3\lib\inspect.py", line 733, in getmodule

    if ismodule(module) and hasattr(module, '__file__'):

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 238, in __getattribute__

    return getattr(getmod(), name)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 216, in getmod

    x = importobj(modpath, None)

  File "C:\Users\physics\Miniconda3\lib\site-packages\pymedphys\_vendor\apipkg\__init__.py", line 103, in importobj

    module = __import__(modpath, None, None, ["__doc__"])

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>

    from .base import CAP_STYLE, JOIN_STYLE

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geometry\base.py", line 18, in <module>

    from shapely.coords import CoordinateSequence

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\coords.py", line 8, in <module>

    from shapely.geos import lgeos

  File "C:\Users\physics\Miniconda3\lib\site-packages\shapely\geos.py", line 145, in <module>

    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))

  File "C:\Users\physics\Miniconda3\lib\ctypes\__init__.py", line 356, in __init__

    self._handle = _dlopen(self._name, mode)

OSError: [WinError 126] The specified module could not be found

---------------------------------------------------------------------------

In [4]:

[Simon Biggs]

Yup, that’s fair.

 

I should have written:

 

pymedphys.mosaiq.connect :)

[LipTeck CHEW]

Hi Simon,

The results is now:

(base) D:\pymedphys\config>ipython

Python 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)]

Type 'copyright', 'credits' or 'license' for more information

IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pymedphys

In [2]: import pymedphys._mosaiq.delivery as delivery

In [3]: with pymedphys.mosaiq.connect(mosaiq_hostname) as cursor:

   ...:     delivery.get_mosaiq_delivery_details(cursor, machine, delivery_time, field_label, field_name, buffer=240)

   ...:

ERROR:root:Internal Python error in the inspect module.

Below is the traceback from this internal error.

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3326, in run_code

    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-3-728a3f3aaf30>", line 1, in <module>

    with pymedphys.mosaiq.connect(mosaiq_hostname) as cursor:

NameError: name 'mosaiq_hostname' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\physics\Miniconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2040, in showtraceback

    stb = value._render_traceback_()

AttributeError: 'NameError' object has no attribute '_render_traceback_'