bug in pyXNAT and classic DICOM

[bennett landman]

We have upload DTI sessions using classic dicom (1736 slices/volume). We seem to only be able to retrieve 536 files using pyXNAT. Has anyone else had this problem and/or know of a solution? 

Thanks!

Bennett

[bennett landman]

Here is the low down on a really critical issue for us:

1.       If we look at a problematic session, the preview exists and the number of files is correct.

2.       If we look on the xnat server in the /archive directory, all dcm files are there and we can view the contents. (In other words, the data are “there.”)

3.       If we click on “Download”, we can download the session zip file. It has the correct number of files and we can view the contents of the files.

HOWEVER,

4.       If we click on “Manage Files” and try to download the session in any browser (except IE), we get a “Corrupted Content Error” exception page.

5.       If we attempt to use PyXNAT, we get the error message like: httplib2.HttpLib2Error:            http://masi.vuse.vanderbilt.edu/xnat/data/projects/<REMOVED>I/subjects/VUIISXNAT_S00750/experiments/VUIISXNAT_E00736/scans/5/resources/17014/files/ANON11165.MR.ANONYMIZED.5.1668.20120307.100109.garc94.dcm 404 Not Found

6.       If I put the above URL into my browser (after logging in), I get “HTTP Status 404 - Unable to find catalog entry for given uri.”

7.       If I change the last part of the URL to match another file that I know is in the same directory (in /archive/…), then the download succeeds!

Help! This really isn’t making any sense to us. It is almost like some part of XNAT knows about all the files while another part has selective amnesia.

Thanks,

Bennett

[Timothy Olsen]

We will have to look into the other issues, but regarding #4 I think that was an issue that was resolved in 1.5.4.

Regarding issue 6/7/8, this sounds a bit like an issue UIOWA reported last year, regarding downloading the Nth file failing on any session.  Maybe they have an ear open to this and can chime in if it sounds familiar.

Tim

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msg/xnat_discussion/-/xgw1k1t1IpQJ.
To post to this group, send email to xnat_di...@googlegroups.com.
To unsubscribe from this group, send email to xnat_discussi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xnat_discussion?hl=en.

[chen yang]

Hi Tim,

We have found the bug and solved it.
The bug is that all the dcm files that containing the following
patterns will have this 404 file not found error.
For example,
xxxx.jar34f.dcm
xxxx.zipqro.dcm
xxxx.garkob.dcm
xxxx.rarftm.dcm
xxxx.ear34f.dcm

And the file in Bennett's #5 .....100109.garc94. dcm

All those files contains .jar,.zip,.gar,.rar,.ear which are defined in
the FileList.java in the /xnat_builder_1_5/plugin-
resources/webapp/xnat/java/org/nrg/xnat/restlet/resources/files/
folder.
static String[] zipExtensions={".zip",".jar",".rar",".ear",".gar"};

The REST API will try to download xxx.jar, xxx.zip,and xxx.rar files
when you want to access dcm file xxx.garc94.dcm.
However, xxx.jar does not exist in the server so it will response with
404 file not found.

We changed our pyxnat code to download all the dicom/files and then it
works fine.
We have changed the XNAT REST API matching patterns, but not tested.
Hope the XNAT team can fix t his bug officially.

Thanks a lot!
Chen

On Jun 1, 2:46 pm, Timothy Olsen <t...@deck5consulting.com> wrote:
> We will have to look into the other issues, but regarding #4 I think that
> was an issue that was resolved in 1.5.4.
>
> Regarding issue 6/7/8, this sounds a bit like an issue UIOWA reported last
> year, regarding downloading the Nth file failing on any session.  Maybe
> they have an ear open to this and can chime in if it sounds familiar.
>
> Tim
>
>
>
>
>
>
>

> On Fri, Jun 1, 2012 at 2:34 PM, bennett landman <baland...@gmail.com> wrote:
> > Here is the low down on a *really *critical issue for us: ****

>
> > 1.       If we look at a problematic session, the preview exists and the

> > number of files is correct. ****

>
> > 2.       If we look on the xnat server in the /archive directory, all dcm
> > files are there and we can view the contents. (In other words, the data are

> > “there.”)****

>
> > 3.       If we click on “Download”, we can download the session zip file.
> > It has the correct number of files and we can view the contents of the

> > files. ****
>
> > HOWEVER, ****

>
> > 4.       If we click on “Manage Files” and try to download the session in
> > any browser (except IE), we get a “Corrupted Content Error” exception page.

> > ****

>
> > 5.       If we attempt to use PyXNAT, we get the error message like:
> > httplib2.HttpLib2Error:
> >http://masi.vuse.vanderbilt.edu/xnat/data/projects/<REMOVED>I/subjects/VUIISXNAT_S00750/experiments/VUIISXNAT_E00736/scans/5/r esources/17014/files/ANON11165.MR.ANONYMIZED.5.1668.20120307.100109.garc94. dcm 404

> > Not Found****

>
> > 6.       If I put the above URL into my browser (after logging in), I get

> > “HTTP Status 404 - Unable to find catalog entry for given uri.”****

>
> > 7.       If I change the last part of the URL to match another file that
> > I know is in the same directory (in /archive/…), then the download

> > succeeds! ****

>
> > Help! This really isn’t making any sense to us. It is almost like some
> > part of XNAT knows about all the files while another part has selective

> > amnesia. ****
>
> > Thanks,****
>
> > Bennett****

[David Gutman]

Chen et al... great work!!

--
David A Gutman, M.D. Ph.D.
Assistant Professor of Biomedical Informatics
Senior Research Scientist, Center for Comprehensive Informatics
Emory University School of Medicine

[bennett landman]

We'll setup our scripts to download all data. I'll let ya'll know how it works for us. 

 

Hopefully, this is an easy fix for the 1.6 release? 

[bennett landman]

I can confirm that download by zip works (and is MUCH faster). THANK YOU for helping us resolve this so quickly. We look forward to upgrading to 1.6. 

[Herrick, Rick]

Easy enough that we just fixed it in two minutes :D

--

You received this message because you are subscribed to the Google Groups "xnat_discussion" group.

To view this discussion on the web visit https://groups.google.com/d/msg/xnat_discussion/-/QGruOpEoijcJ.

To post to this group, send email to xnat_di...@googlegroups.com.
To unsubscribe from this group, send email to xnat_discussi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xnat_discussion?hl=en.

---

The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Adam Harding]

Just putting in a request that your official bugfix / relevant
changeset gets into the XNAT 1.5.x series (and 1.4 if it's still being
supported with release fixes) to keep fixes upstream. Pending any
testing you apply, that is-- though the fix seems pretty
straightforward.

On Jun 4, 9:28 am, "Herrick, Rick" <herri...@mir.wustl.edu> wrote:
> Easy enough that we just fixed it in two minutes :D
>
> From: xnat_di...@googlegroups.com [mailto:xnat_di...@googlegroups.com] On Behalf Of bennett landman
> Sent: Saturday, June 02, 2012 4:43 PM
> To: xnat_di...@googlegroups.com
> Subject: Re: [XNAT Discussion] Re: bug in pyXNAT and classic DICOM
>
> We'll setup our scripts to download all data. I'll let ya'll know how it works for us.
>
> Hopefully, this is an easy fix for the 1.6 release?
> --
> You received this message because you are subscribed to the Google Groups "xnat_discussion" group.

> To view this discussion on the web visithttps://groups.google.com/d/msg/xnat_discussion/-/QGruOpEoijcJ.
> To post to this group, send email to xnat_di...@googlegroups.com<mailto:xnat_di...@googlegroups.com>.
> To unsubscribe from this group, send email to xnat_discussi...@googlegroups.com<mailto:xnat_discussi...@googlegroups.com>.
> For more options, visit this group athttp://groups.google.com/group/xnat_discussion?hl=en.

[Rick Herrick]

The change for this for 1.5dev is available here: https://bitbucket.org/nrg/xnat_builder_1_5dev/changeset/0cbaae59469f

On Monday, June 4, 2012 10:32:01 AM UTC-5, Adam Harding wrote:

Just putting in a request that your official bugfix / relevant
changeset gets into the XNAT 1.5.x series (and 1.4 if it's still being
supported with release fixes) to keep fixes upstream. Pending any
testing you apply, that is-- though the fix seems pretty
straightforward.

On Jun 4, 9:28 am, "Herrick, Rick" <herri...@mir.wustl.edu> wrote:
> Easy enough that we just fixed it in two minutes :D
>
> From: xnat_discussion@googlegroups.com [mailto:xnat_discussion@googlegroups.com] On Behalf Of bennett landman
> Sent: Saturday, June 02, 2012 4:43 PM
> To: xnat_discussion@googlegroups.com
> Subject: Re: [XNAT Discussion] Re: bug in pyXNAT and classic DICOM
>
> We'll setup our scripts to download all data. I'll let ya'll know how it works for us.
>
> Hopefully, this is an easy fix for the 1.6 release?
> --
> You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
> To view this discussion on the web visithttps://groups.google.com/d/msg/xnat_discussion/-/QGruOpEoijcJ.

> To post to this group, send email to xnat_discussion@googlegroups.com<mailto:xnat_discussion@googlegroups.com>.
> To unsubscribe from this group, send email to xnat_discussion+unsubscribe@googlegroups.com<mailto:xnat_discussion+unsubscribe@googlegroups.com>.