Unable to upload DICOM-RTSTRUCT file to XNAT using XNATpy

[Cristiano]

Hello all,

I'm not only new to XNAT but also XNATpy and I've been trying to upload a DICOM-RTSTRUCT file to XNAT using XNATpy. 

The DICOM scans are taken from the Brain Tumor Progression Collection that is part of the XNAT Academy 101 Course.  The RTSTUCT file was created using the OHIF-Viewer plugin and downloaded from XNAT using XNATpy.

I''ll attempt to keep this as readable as possible but will try not to leave out any code snippets that'll make it impossible for someone to understand what I've done. Here goes:

import xnat

session = xnat.connect(xnatAddress, user=username, password=password, debug=True)

# Create new project in XNAT:

newProjectName = 'BrainTumorProg_copy'

newProject = session.classes.ProjectData(parent=session, name=newProjectName)

# Create a new subject:

newSubjectName = 'PGM-002_copy'

newSubject = session.classes.SubjectData(label='PGM-002_copy', parent=newProject)

# Create a new experiment:

newExperimentName = 'PGM-002_copy_MR_2'

newExperiment = session.classes.MrSessionData(label=newExperimentName, parent=newSubject)

# Create an assessor following Hakim Achterberg's guidance here:

assessorLabel = 'ROI_tumour'

newExperiment.create_assessor(assessorLabel, type_='xnat:qcAssessmentData')

# This has created an Assessment with the Experiment label "Auto QC" in the web interface.

# An alternative method suggested by Hakim Achterberg (can't find the link...):

assessorLabel2 = 'ROI_tumour_2'

assessor = session.classes.QcAssessmentData(parent=newExperiment, label=assessorLabel2)

# which has also created an Assessment with the Experiment label "Auto QC" in the web interface.

resource = session.classes.ResourceCatalog(parent=assessor, label='RESOURCE_LABEL')

# The above line was part of the example code by Hakim. It's unclear to me what it has done, however.

# Returning to my web interface, it appears that the command placed "+ Session QC Status" above Scans.

"""

So far here's what I get:

In: assessor

Out: <QcAssessmentData ROI_tumour_2 (XNAT_E00026)>

In: resource

Out: <ResourceCatalog RESOURCE_LABEL (RESOURCE_LABEL)>

"""

# Next I tried to import via the prearchive:

prearchive_session = session.services.import_(roiFpathDL, \
project=newProjectName, \
destination='/prearchive')

"""

Debug output:

Received response with status code: 500

access.log file:

2020-02-12 16:13:30,178 - admin 127.0.0.1 POST http://10.1.1.17/data/services/import?dest=%2Fprearchive&project=BrainTumorProg_copy

dicom.log file: 

2020-02-12 16:13:31,811 [pool-2-thread-1] ERROR org.nrg.dcm.xnat.DICOMSessionBuilder - Session builder not implemented for SOP class [1.2.840.10008.5.1.4.1.1.481.3] or modality [RTSTRUCT]

XNAT web interface prearchive error: 

XNAT has encountered an error with your request:¶

Exception: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.¶

If this error continues to occur, please contact your system administrator with information about how to recreate the problem.

"""

Unfortunately the "session builder not implemented" message doesn't mean much to me.  And as for "premature end of file" - given that I'm trying to upload an unmodified RTSTRUCT file previously downloaded from XNAT, I don't expect an issue to be with the file itself.  I should add that I've been able to upload the DICOM-RTSTRUCT file to XNAT using the ICR's ROIUploadAssistant tool.

Does anyone have any ideas?

Thanks!

-Cristiano

[Moore, Charlie]

Hello Cristiano,

 

The behavior you’re seeing is what I would expect. There may be ways to work around it, but I’m having difficulty understanding what the end goal is here, so can you please clarify more specifically what you’re trying to do? It sounds like you already have an RTSTRUCT file exported from the OHIF viewer to your session, so I’m confused on the purpose of trying to upload it again to XNAT.

 

Thanks,

Charlie

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/ae5b930d-18e4-44c4-aba4-7f42c4f3f0c7%40googlegroups.com.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Cristiano]

Hi Charlie,

Unfortunately the website isn't live yet and I'm sure I won't do the project justice, but broadly speaking I'm playing a small part within a consortium of UK universities (and I believe some of the people behind XNAT at Washington University) to further develop an XNAT-based pipeline to facilitate the sharing, storing, access and analysis of medical images, with the aim to help accellerate advances in cancer research. 

My focus for the moment is getting to grips with the basics of downloading and uploading to XNAT (currently using XNATpy) and the DICOM standard.  If I can figure out how to upload a DICOM-RTSTRUCT file that I trust has been formatted correctly, the next step would be uploading one that I've modified (using an existing file as a template), and ultimately if needed, creating one from scratch, with contour data obtained through means other than manual creation of ROIs.

Best regards,

Cristiano

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/ae5b930d-18e4-44c4-aba4-7f42c4f3f0c7%40googlegroups.com.

[Moore, Charlie]

Cristiano,

 

That sounds like a very exciting and ground-breaking project! I think I was the one too vague here as I really meant the goal of uploading an RTSTRUCT file to XNAT when it already exists. Unfortunately, the complete answer to the question of how to upload an RTSTRUCT instance in the general case is rather complex and involved.

 

How you upload an RTSTRUCT file is largely determined by a few competing factors:

1.       Does the study to which the RTSTRUCT file corresponds already exist in XNAT?

2.       What do you want to do with the file?

 

If you upload the file to the prearchive (as you tried), or archive it directly, it will normally need to be accompanied by its corresponding image data. This is a known limitation of XNAT, documented here: https://issues.xnat.org/browse/XNAT-4527 . Essentially, there is a list of “primary” modalities in XNAT where it knows how to construct a session object from the DICOM. Since you’re dealing with RTSTRUCT files, the main modality of your studies is likely MR or CT. Both of those are fine for creating a session, but just RTSTRUCT is not. The prearchive sees a “session” of just RTSTRUCT files (yes, even if the MR or CT already exist in the archive), and errors out. This workflow is fine for the case where the studies have not yet been uploaded to XNAT (i.e. the answer to factor 1 is NO), since you can send the whole session at once. However, it sounds like in your case the sessions already exist in XNAT, and you would want to add RTSTRUCT files to them. It might be possible to send the whole session again, including the RTSTRUCT, with the prearchive option for the project set to the most aggressive option (auto-archive and overwrite files), but this is rather inefficient and a bit hacky. (I’m also not sure it would work)

 

Now, we need to detour a little bit. RTSTRUCT files are completely valid DICOM series, so XNAT normally includes them as “scans” in your session. So, if you upload a study with 9 MR series and 1 RTSTRUCT series, XNAT will create an MR session with 10 scans (9 MR, and 1 RTSTRUCT). Here, I mean “upload” through one of the standard mechanisms (C-STORE, REST, Upload Application, Compressed Uploader). However, this is *not* how the OHIF-viewer plugin for XNAT works. That plugin has an added datatype for “ROI Collections”. When you use the OHIF viewer (in XNAT) to create a new RTSTRUCT file, the plugin will store the file in a new “ROI Collection” assessor for the session. Likewise, the viewer reads contours/segmentations from ROI Collections and not XNAT scans. So, the question of how to upload it becomes: what’s it for?

 

If you want it to be a standard XNAT scan (for searching, viewing in the scan table, etc.), the only thing that comes to mind is scripting the upload with REST, or the hack I mentioned earlier. However, this is not sending the file to the prearchive and letting XNAT handle a merge. Instead, this would be explicitly creating a new scan under the pre-existing session and then adding the file as a resource. I don’t know if XNATpy has support for that or not, but it’s not terribly difficult if just accessing the REST API directly.

 

If you want it to be an ROI collection for viewing in the OHIF viewer, you already found your answer. It’s likely possible to do this only through the REST API, but ICR wrote that ROIUploadAssistant tool for precisely this case (I believe). I haven’t used it myself, but I imagine that tool will be much easier than using the REST API directly. However, it sounds like you’re needing to do this in a scripted way, so hopefully the team from ICR might be able to tell you how that tool works to do a similar thing in XNATpy / REST.

 

This was a bit of a brain dump, so I hope it makes sense.

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/5c6c312e-dbd9-4dbc-886a-e81e71edfa05%40googlegroups.com.

[Cristiano]

Thanks very much for your highly detailed response Charlie!

It's a bit clearer now that what I was trying to do is not as straightforward as I thought it might be. 

-C

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/5c6c312e-dbd9-4dbc-886a-e81e71edfa05%40googlegroups.com.

[Daniel Marcus]

You can also use the container service to generate ROI Collections for DICOM RT files that have been uploaded as scans to a session.  The container to do this is here.  And you can set up an automation to trigger automatically when the session is created.

-Dan

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/dd59c896-a9a9-4d4b-b8e7-ee5463f59204%40googlegroups.com.

[Cristiano]

 Thanks Dan.  It's useful to know of another way of doing it.

-Cristiano

[Prerak Mody]

Hi,

 - I have a CTSession in which I have uploaded dicom files for a CT and 1 file for the associated RTSTRUCT.

 - I then ran the python script inside the container which generated an XML file for RoiCollection and I uploaded that via the web interface

Problem: When I now go inside the ROICollection, it seems to not have any resources associated with it.

Note:  I have checked that the series UID in the uploaded .xml file matches with the RTSTRUCT file which was already present on the XNAT instance.

What am I missing here?

[Moore, Charlie]

Hello there,

If you just uploaded the XML for the ROI Collection and didn't also upload any resources, then your ROI Collection won't have any resources. The scans of a session (where XNAT extracts all DICOM data when imported, including RTSTRUCTs) is totally separate from assessors (such as an ROI Collection), so even if the series UIDs match between a scan and an ROI Collection, XNAT will not tie their resources together.

Thanks,

Charlie

---

From: xnat_di...@googlegroups.com <xnat_di...@googlegroups.com> on behalf of Prerak Mody <prera...@gmail.com>
Sent: Tuesday, October 12, 2021 12:54 PM

To: xnat_discussion <xnat_di...@googlegroups.com>
Subject: Re: [XNAT Discussion] Unable to upload DICOM-RTSTRUCT file to XNAT using XNATpy

 

* External Email - Caution *

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/d4c73664-6e33-446d-8215-bbda4ff19e18n%40googlegroups.com.

[Prerak Mody]

Hi,

I realized this later, but thank you for your response.

For future reference, this is how I did it using xnatpy

```

self.session.classes.ResourceCatalog(parent=<ExperimentObj>, label="rtstruct_name")

<ExperimentObj>.resources["rtstruct_name"].upload_dir("path_to_rtstruct.dcm", method='per_file', overwrite=True)

```

Here <ExperimentObj> can be a CT/MR session