Problems exporting mamo exams

[daniel.fern...@gmail.com]

Hi, everybody:

I'm a new user of Openrem, vers. 0.7.1. When I try to export mamo exams, the app enter a loop, showing this information in the webpage:

Export Tasks in Progress
Task ID Exported Modality Export type
8da80378-5972-44f4-b952-ea726293c17c 15 minutes ago MG CSV export

No. records Progress
2 Required study filter complete.

The export doesn't finished. ¿Can anybody help me, please? Thank you.

Daniel Fernández Molina
Seville, Spain

[Ed McDonagh]

Hi Daniel

Welcome to OpenREM, sorry you have hit an issue!

Can you have a look at the Celery log to see what is at the end of it? It will probably tell you what the problem is. You will have specified where the log is when you started Celery: http://docs.openrem.org/en/0.7.1/startservices.html#celery-task-queue

However, I suspect it might be related to a bug that I have recently fixed (fix will be in the next beta release) - do you have any non-ASCII characters in your export - in the protocol names for example? If so, it might be this bug: https://bitbucket.org/openrem/openrem/issues/425/ct-csv-export-with-non-ascii-fails

Please have a look and report back. Thank you for helping make OpenREM better!

Kind regards

Ed

[Ed McDonagh]

Hi Daniel

Just to let you know that the current beta (0.7.3b2) has the required fixes for non-ASCII export to csv.

I would be very interested if that was indeed your issue, and if it was, whether 0.7.3b2 fixed it. See http://docs.openrem.org/en/latest/release-0.7.3.html#upgrading-from-version-0-7-1 for instructions for upgrading, or http://docs.openrem.org/en/latest/install-firsttime.html for a fresh install.

Kind regards

Ed

[Ed McDonagh]

Thanks for this Daniel

Would it be possible to send me a QA image or non-patient image that has this character in the study description so that I might try and fix the problem?

Kind regards

Ed

On 5 August 2016 at 09:58, Daniel Fernández <daniel.fern...@gmail.com> wrote:

Hi, Ed:

Excuse me for the delay in answering. You were right, the problem was related with non-ASCII characters. Now it works with a little bug: the non-ASCII charater isn't represented correctly. For example, the study description Mamografía is imported as MamografÃ-­a. This is a minor issue, but I report just to let you know.

Thank you for your help, and for sharing Openrem with the community. It's a very useful application.

Best regards,

Daniel Fernández

Spain

[Ed McDonagh]

Dear Daniel

Thank you for the files you have sent me. You are right in that the characters are correctly written to the csv file, but then are corrupted when imported to Excel.

The issue is that Excel requires a specific BOM (Byte Order Mark) to be at the beginning of the file on order to make the right decision about the data being UTF-8. OpenREM currently does not prefix the files with this.

The immediate solution is to open the CSV files using the text import wizard (Data -> From Text), and specifying the encoding in the first page of the wizard: in the File origin box, near the end of the list you can choose '65001 : Unicode (UTF-8)'.

A longer term solution could be to change OpenREM to encode the BOM characters so Excel knows what to do.

I was confused that the file you sent me used semi-colons instead of commas - do you know why that might have been?

With regards to the columns in the CT exports, the 'E#' relates to Exposure 1, Exposure 2, Exposure 3 etc as they were presented in the dose report (not necessarily the order they took place, but usually is). The 'S#' refers to the source 1 and 2, as found on some Siemens scanners (ie the Flash and the Force), where each tube can have a different kV, mA, exposure time per rotation etc. For most scanners, the S2 columns will be empty.

I hope that is clear - perhaps some modifications to the documentation would be useful to explain!

Kind regards

Ed

On 6 September 2016 at 11:19, Daniel Fernández wrote:

Hi, Ed:

I've found out that the issue doesn't appear if you open the exported csv file with a plain text editor like Notepad +. However, if the csv file is imported into an Excel 2007 workbook, the problem arise. I send you a csv file resulting of exporting the study in the zipped file and an Excel 2007 workbook with the issue in the field Study Description (column N), related with the character í.

I'm working know with exported CT files and I have some doubts about the meaning of some ot the fields. For example, "E1 S1 mA" versus "E1 S2 mA". Is there any documentation explaining this point?

Thank you. Kind regards,

2016-08-30 15:08 GMT+02:00 Ed McDonagh:

Thanks for this Daniel

Would it be possible to send me a QA image or non-patient image that has this character in the study description so that I might try and fix the problem?

Kind regards

Ed

On 5 August 2016 at 09:58, Daniel Fernández wrote:

Hi, Ed:

Excuse me for the delay in answering. You were right, the problem was related with non-ASCII characters. Now it works with a little bug: the non-ASCII charater isn't represented correctly. For example, the study description Mamografía is imported as MamografÃ-­a. This is a minor issue, but I report just to let you know.

Thank you for your help, and for sharing Openrem with the community. It's a very useful application.

Best regards,

Daniel Fernández

Spain

2016-07-29 15:04 GMT+02:00 Ed McDonagh:

[David Platten]

This might be very easy to fix:

http://xlsxwriter.readthedocs.io/example_unicode_python2.html

I'll take a look at some point.

David