Uploading compressed DICOMs using pyxnat

[Dave W]

Hello all, 

I'm trying to use pyxnat to upload a zipped scan to an empty experiment and I'm having trouble finding documentation on the put()/insert() command.  What I'm doing is this:

filename = r"/Volumes/scratch/uploadToXnat/{p}/{s}/{e}.zip".format(p=project, s=subject, e=experiment)
try:
    xnat = openXNAT() # returns a pyxnat.Interface instance
    # verify that the project/subject/experiment exist
    # <...>
    experiment = xnat.select.project(project).subject(subject).experiment(experiment) # pyxnat.EObject
    experiment.resource('DICOM').file(os.path.basename(filename)).insert(filename, format=".zip", content="application/zip", extract="true")

finally:

    xnat.disconnect()

   

Unfortunately, this isn't extracting the zip file on the server.  Is there a way in pyxnat to get the curl command from the Interface after a query/post?  That would be super helpful here...

Cheers, 

Dave W.

[felix.rui...@icometrix.com]

Hi, 

i encountered this problem recently too. 

Could you try, after having uploaded the scans, the following method?

experiment.trigger()

this should normally extract dicom headers and put everything in the right place.

grtz

[Dave W]

Still no good.

header: Connection: close

['status', 'content-length', 'set-cookie', 'accept-ranges', 'server', 'connection', 'date', 'content-type']

500

https://xnat.hdni.org/xnat/data/JSESSION

connect: (xnat.hdni.org, 443)

send: 'DELETE /xnat/data/JSESSION HTTP/1.1\r\nHost: xnat.hdni.org\r\nconnection: keep-alive\r\ncookie: JSESSIONID=B7505EDB5CA94F1A244144E36AF55513\r\naccept-encoding: gzip, deflate\r\nauthorization: Basic ZG13ZWxjaDpldGhlcjEyMjc=\r\nuser-agent: Python-httplib2/0.8 (gzip)\r\n\r\n'

reply: 'HTTP/1.1 200 OK\r\n'

header: Date: Tue, 17 Sep 2013 19:12:31 GMT

header: Server: Noelios-Restlet-Engine/1.1.6

header: Accept-Ranges: bytes

header: Content-Length: 0

header: Set-Cookie: SESSION_EXPIRATION_TIME="1379445152005,900000"; Version=1; Path=/

header: Connection: close

header: Content-Type: text/plain; charset=UTF-8

Traceback (most recent call last):

  File "trackon/upload_images_to_xnat_to_hdni_server.py", line 788, in <module>

    send_dcm_tmp_files_to_xnat('HDNI_004', '121366443', '121366443_20130807_30', '/Shared/johnsonhj/TrackOn_raw_data/TRACKON/121366443/165_dyn_resting_state/2013-08-07_13_07_04.0/S197443')

  File "trackon/upload_images_to_xnat_to_hdni_server.py", line 363, in send_dcm_tmp_files_to_xnat

    experiment.trigger()

  File "/IPLlinux/raid0/homes/dmwelch/.virtualenvs/xnat/lib/python2.7/site-packages/pyxnat/core/resources.py", line 1437, in trigger

    self._intf._exec(self._uri + options, 'PUT')

  File "/IPLlinux/raid0/homes/dmwelch/.virtualenvs/xnat/lib/python2.7/site-packages/pyxnat/core/interfaces.py", line 434, in _exec

    catch_error(content)

  File "/IPLlinux/raid0/homes/dmwelch/.virtualenvs/xnat/lib/python2.7/site-packages/pyxnat/core/errors.py", line 68, in catch_error

    raise DatabaseError(error)

pyxnat.core.errors.DatabaseError: Unable to derive session directory

The zip file is not being extracted on the server - I can see the experiment in the web browser if I search for the experiment label and in the 'Manage Files' I can see the compressed zip file under the 'DICOM' tree.  But since the file isn't extracted, the scan isn't showing.  Just for giggles, I've tried to create the scan, e.g.

experiment = xnat.select.project(project).subject(subject).experiment(experiment) # pyxnat.EObject
experiment.resource('DICOM').file(os.path.basename(filename)).insert(filename, format=".zip", content="application/zip", extract="true")

experiment.trigger()
experiment.scan('401').create()

but that's no good.  Anyone have a suggestion?

Dave

[Felix Ruiz de Arcaute]

What do the server logs say when you call  experiment.trigger()

Vriendelijke groeten,

Kind regards,

Felix Ruiz de Arcaute

icoMetrix - Quantifying Your Images

Felix Ruiz de Arcaute

M +32 494 98 23 99

Felix.rui...@icometrix.com

Tervuursesteenweg 244, 3001 Leuven, Belgium

T +32 16 84 96 97 - www.icometrix.com

BE 0833 377 379

This e-mail and all attachments are confidential.

2013/9/17 Dave W <david....@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "xnat_discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/xnat_discussion/AIVNBuMdniM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/groups/opt_out.

[Tim Olsen]

Dave,

 

The method you are using here will not create individual scans that correspond to the data in the zip.  Instead, it will create one big DICOM resource at the experiment level.

 

If you want this data to be used to create the full XNAT hierarchy, you should use one of XNAT’s importer services.  I’m not sure what the proper PyXNAT api calls would be, but it would be easy to do via curl to a URL like /REST/services/import?dest=/projects/PROJ/subjects/SUBJ/experiments/EXPT.

 

Hope that helps,

Tim

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.

[Robin Koch]

Is there anything new on this matter?

I tried to accomplish the same thing as Dave, with a different code (taken from here):

xnat = pyxnat.Interface(server=SERVER, user=USER, password=PASS, proxy=PROXY)
remote_scan = xnat.select.project(PROJECT).subject(SUBJECT).experiment(SCAN)
remote_scan.resource(SCAN).file(FILE).put(PATH + FILE)

When I try

remote_scan.trigger()

I get this in the restlet.log:

ERROR org.nrg.xnat.restlet.resources.SubjAssessmentAbst -
java.lang.Exception: Unable to derive session directory
 at org.nrg.xnat.restlet.actions.PullSessionDataFromHeaders.call(PullSessionDataFromHeaders.java:75)
 at org.nrg.xnat.restlet.resources.SubjAssessmentResource.handlePut(SubjAssessmentResource.java:578)

1. Is it possible to use the importer via pyxnat?
2. Am I wrong or is the trigger() function (still) not documented!?

Robin

[Herrick, Rick]

I’m almost certain you can use the importer through pyxnat, but I have no idea how to do so. This doc from BIGR describes one way to do so.

 

I also have no clue on the trigger() function.

 

It’s worth noting that you could use a regular HTTP library like requests to send data to the import REST API directly.

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

--

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.

Visit this group at https://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Robin Koch]

Am Montag, 8. August 2016 19:12:56 UTC+2 schrieb Rick Herrick:

I’m almost certain you can use the importer through pyxnat, but I have no idea how to do so. This doc from BIGR describes one way to do so.

 
That site only describes how to upload files, not importing experiments.

Using the importer through REST requiers an URI like .../services/import?...
I went through the sourcecode of pyxnat and couldn't find any occurrence of "import" or "services". Which for me is a good indicators, that's not implemented.

However I saw some code "abusing" the Interface._exec() to send arbitrary requests. That's certainly a way to do it.

It’s worth noting that you could use a regular HTTP library like requests to send data to the import REST API directly.

Yes, that's apparently what pyxnat is doing under hood, too.
Using the _exec may not be the pythonic way, but it sure is a helpful shortcut..

Robin

[Hakim Achterberg]

Hi Robin, Rick,

I did actually implemented /data/services/import in xnatpy. The relevant sourcecode is located at https://bitbucket.org/bigr_erasmusmc/xnatpy/src/82616fe4cd7dae91b2876473c82160215d333a1b/xnat/services.py?fileviewer=file-view-default 

Hope this helps a bit. It shouldn't be difficult to implement something similar in pyxnat.

It is important to note that I hadn't had the subject and experiment parameters to the method and I just added them in the develop version (but that is not on pypi yet). I tested it briefly with .zip files downloaded from xnat (entire experiments) and that seemed to work.

Cheers,

Hakim

[Herrick, Rick]

Another option–which I haven’t tried at all and have no idea how well it works–would be to use pynetdicom to do a C-STORE to the XNAT DICOM SCP receiver, similar to the example in “Send DICOM data from python”:

 

https://pythonhosted.org/pynetdicom/usecases.html

 

The package is version 0.8.1, but then again pyxnat was 0.9.4 for multiple years, so maybe that’s OK J

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 740-5961

 

From: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com> on behalf of Hakim Achterberg <hakim.ac...@gmail.com>
Reply-To: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com>
Date: Tuesday, August 9, 2016 at 8:22 AM
To: "xnat_di...@googlegroups.com" <xnat_di...@googlegroups.com>
Subject: Re: [XNAT Discussion] Re: Uploading compressed DICOMs using pyxnat

 

Hi Robin, Rick,

--

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.