ProtocolCheckPipeline

195 views
Skip to first unread message

Gaelle

unread,
Aug 23, 2013, 6:46:30 AM8/23/13
to xnat_di...@googlegroups.com, gaelle...@u-bordeaux2.fr
Dear all,

My problem deals with the execution of the protocol check / validate pipeline.
I precise that the DicomToNifti pipeline works fine so a few things are well set up I guess.


The configuration I use is:
- NeuroDebian 6.0.6 (64 bit)
- Xnat Release 1.6.1
- java version 1.7.0_25
- Apache Tomcat/6.0.35.0
- PostgreSQL 8.4.17


My Protocol Check pipeline stops at 20% and then I receive a confirmation email of the failure which copies what's in the log file:

-----------
Pipeline:
/usr/local/xnat/pipeline/catalog/validation_tools/Validate.xml

Cause:
org.nrg.pipeline.exception.PipelineEngineException: Unable to complete pipeline Execution of
/usr/local/xnat/pipeline/xnat-tools/XNATRestClient -u ff0f1a06-568e-4816-9cd3-8fbd2534d9d5 -p ######## -host http://127.0.0.1:8080/xnat/ -remote "/REST/projects/MIBRAIN_GL/resources/validation_xnat_mrSessionData/files/" -m GET > /Library/XNAT/build/MIBRAIN_GL/20130823_122415/WE021_MR1/VALIDATION/ at Step[id= 0b:]
was unsuccesfull Execution of /usr/local/xnat/pipeline/xnat-tools/XNATRestClient -u ff0f1a06-568e-4816-9cd3-8fbd2534d9d5 -p ######## -host http://127.0.0.1:8080/xnat/ -remote "/REST/projects/MIBRAIN_GL/resources/validation_xnat_mrSessionData/files/" -m GET > /Library/XNAT/build/MIBRAIN_GL/20130823_122415/WE021_MR1/VALIDATION/ at Step[id= 0b:]
was unsuccesfull at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:226) at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:193) at org.nrg.pipeline.client.XNATPipelineLauncher.launch(XNATPipelineLauncher.java:65)
at org.nrg.pipeline.client.XNATPipelineLauncher.run(XNATPipelineLauncher.java:231) at org.nrg.pipeline.client.XNATPipelineLauncher.main(XNATPipelineLauncher.java:221)
--------------


and the long log file in the xnat_home/pipeline/log folder looks like that:
----------------------
[...]
FATAL main org.nrg.pipeline.client.XNATPipelineLauncher - Couldnt search for queued workflows
java.io.FileNotFoundException: /usr/share/tomcat6/32.xml (Permission denied)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(Unknown Source)
    at java.io.FileOutputStream.<init>(Unknown Source)
    at org.nrg.xnattools.xml.XMLSearch.execute(XMLSearch.java:248)
    at org.nrg.xnattools.xml.XMLSearch.searchAll(XMLSearch.java:109)
    at org.nrg.pipeline.client.XNATPipelineLauncher.isPipelineQueuedOrAwaitingOrOnHold(XNATPipelineLauncher.java:281)
    at org.nrg.pipeline.client.XNATPipelineLauncher.launch(XNATPipelineLauncher.java:63)
    at org.nrg.pipeline.client.XNATPipelineLauncher.run(XNATPipelineLauncher.java:231)
    at org.nrg.pipeline.client.XNATPipelineLauncher.main(XNATPipelineLauncher.java:221)
DEBUG main org.nrg.pipeline.xpath.XPathResolverSaxon - Expre /usr/local/xnat/pipeline/catalog/validation_tools/resources/svrl/nrg_iso_svrl_for_xslt2.xsl [/usr/local/xnat/pipeline/catalog/validation_tools/resources/svrl/nrg_iso_svrl_for_xslt2.xsl]
DEBUG main org.nrg.pipeline.xpath.XPathResolverSaxon - Expre ^concat('validation_',translate(/Pipeline/parameters/parameter[name='sessionType']/values/unique/text(),':','_'))^ [validation_xnat_mrSessionData]
DEBUG main org.nrg.pipeline.xpath.XPathResolverSaxon - Expre ^concat(translate(/Pipeline/parameters/parameter[name='builddir']/values/unique/text(),'/',fileUtils:GetFileSeparator()),fileUtils:GetFileSeparator(),/Pipeline/parameters/parameter[name='sessionLabel']/values/unique/text())^ [/Library/XNAT/build/MIBRAIN_GL/20130823_122415/WE021_MR1]
DEBUG main httpclient.wire.header - >> "GET /xnat/REST/projects/MIBRAIN_GL/files?format=json HTTP/1.1[\r][\n]"
DEBUG main httpclient.wire.header - >> "User-Agent: Noelios-Restlet-Engine/1.1.rc1[\r][\n]"
DEBUG main httpclient.wire.header - >> "Cookie: JSESSIONID=9465AD3FD95DC676BB342C64AA74A9FE[\r][\n]"
DEBUG main httpclient.wire.header - >> "Accept: */*[\r][\n]"
DEBUG main httpclient.wire.header - >> "Host: 127.0.0.1:8080[\r][\n]"
DEBUG main httpclient.wire.header - >> "[\r][\n]"
DEBUG main httpclient.wire.header - << "HTTP/1.1 200 OK[\r][\n]"
DEBUG main httpclient.wire.header - << "HTTP/1.1 200 OK[\r][\n]"
DEBUG main httpclient.wire.header - << "Set-Cookie: SESSION_EXPIRATION_TIME="0,900000"; Version=1; Path=/[\r][\n]"
DEBUG main httpclient.wire.header - << "Date: Fri, 23 Aug 2013 10:24:18 GMT[\r][\n]"
DEBUG main httpclient.wire.header - << "Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept[\r][\n]"
DEBUG main httpclient.wire.header - << "Accept-Ranges: bytes[\r][\n]"
DEBUG main httpclient.wire.header - << "Server: Noelios-Restlet-Engine/1.1.6[\r][\n]"
DEBUG main httpclient.wire.header - << "Content-Type: application/json[\r][\n]"
DEBUG main httpclient.wire.header - << "Transfer-Encoding: chunked[\r][\n]"
DEBUG main httpclient.wire.header - << "[\r][\n]"
DEBUG main httpclient.wire.content - << "1"
DEBUG main httpclient.wire.content - << "3"
DEBUG main httpclient.wire.content - << "e"
DEBUG main httpclient.wire.content - << "[\r]"
DEBUG main httpclient.wire.content - << "[\n]"
DEBUG main httpclient.wire.content - << "{"
DEBUG main httpclient.wire.content - << ""ResultSet":{"Columns":[{"key":"URI","serverRoot":"/xnat"}],"Result":[{"Name":"mrsession_rules.sch","file_content":"","cat_ID":"16","collection":"validation_xnat_mrSessionData","file_format":"","file_tags":"","URI":"/data/projects/MIBRAIN_GL/resources/16/files/mrsession_rules.sch","Size":"6136"}], "title": "Files"}}"
DEBUG main httpclient.wire.content - << "[\r]"
DEBUG main httpclient.wire.content - << "[\n]"
DEBUG main httpclient.wire.content - << "0"
DEBUG main httpclient.wire.content - << "[\r]"
DEBUG main httpclient.wire.content - << "[\n]"
DEBUG main httpclient.wire.content - << "[\r]"
DEBUG main httpclient.wire.content - << "[\n]"
DEBUG main httpclient.wire.header - << "[\r][\n]"
[...]
----------------------


I'm afraid I can't move on in the debug of the pipeline so if someone has suggestions, they are welcome.
Thank you.
Gaelle.

Mohana Ramaratnam

unread,
Aug 23, 2013, 7:07:02 AM8/23/13
to xnat_di...@googlegroups.com
Hi Gaelle,

There seems to be two things going wrong:

a) Home directory of the tomcat user doesnt seem to be set. The pipeline engine will write temporary xmls to the folder specified by the variable pipeline.tmp and if that is not set, it will attempt to write the file in the home folder of the tomcat user. By default the pipeline.tmp directory is not set. That is the reason you are getting the error - Permission denied on /usr/share/tomcat6/32.xml 

You can set the pipeline.tmp location by modifying the file PIPELINE_HOME/config/XnatPipelineLauncher.config file and adding:

XNAT_PIPELINE_LAUNCHER_OPTS="-Dpipeline.tmp=PATH_TO_SOME_FOLDER_TO_WHICH_TOMCAT_USER_HAS_WRITE_RIGHTS"

b) It appears that the the Validate pipeline parameter rulefile is not being set properly. That is the reason you are getting the error:

/usr/local/xnat/pipeline/xnat-tools/XNATRestClient -u ff0f1a06-568e-4816-9cd3-8fbd2534d9d5 -p ######## -hosthttp://127.0.0.1:8080/xnat/ -remote "/REST/projects/MIBRAIN_GL/resources/validation_xnat_mrSessionData/files/" -m GET > /Library/XNAT/build/MIBRAIN_GL/20130823_122415/WE021_MR1/VALIDATION/

The value of the variable is dependent on the variable catalog_content and it is expected that you have added the schematron file to the project as a resource. 

Can you check what you get when you attempt to access the URL using your browser:

http://127.0.0.1:8080/xnat/data/archive/projects/MIBRAIN_GL/resources/validation_xnat_mrSessionData/files 

Mohana

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/groups/opt_out.

plxna...@gmail.com

unread,
Sep 10, 2013, 9:45:42 AM9/10/13
to xnat_di...@googlegroups.com
Hello all,

since the last answer to this thread was posted quite a while ago, I figured out it wouldn't be that bad if I ninja this thread as I have almost the same problem as case b.). This is my setup:
  • XNAT 1.6.1
  • Ubuntu 12.04 LTS
  • Tomcat 7.0.26
  • PostgreSQL 9.2.1
  • Oracle Java JDK 1.7.0_07

I am pretty much a beginner working with XNAT and pipelines and I am absolutely clueless why the validation pipeline isn't working here. It keeps on crashing every time it reaches 20%. The logs shows several entries similar to this one:

[...]

Cause: 

org.nrg.pipeline.exception.PipelineEngineException: Unable to complete pipeline  Execution of /home/xnat/xnat/pipeline/xnat-tools/XNATRestClient -u da85d6b2-579e-40fd-8a8e-546b018ef566 -p ######## -host http://localhost:8080/xnat/ -remote "/REST/projects/TP1/resources/validation_xnat_mrSessionData/files/" -m GET > /opt/xnat/data/build/TP1/20130910_144153/11235811_MR1/VALIDATION/ at Step[id= 0b:] was unsuccesfull  Execution of /home/xnat/xnat/pipeline/xnat-tools/XNATRestClient -u da85d6b2-579e-40fd-8a8e-546b018ef566 -p ######## -host http://localhost:8080/xnat/ -remote "/REST/projects/TP1/resources/validation_xnat_mrSessionData/files/" -m GET > /opt/xnat/data/build/TP1/20130910_144153/11235811_MR1/VALIDATION/ at Step[id= 0b:] was unsuccesfull


    at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:226)
    at org.nrg.pipeline.manager.PipelineManager.launchPipeline(PipelineManager.java:193)
    at org.nrg.pipeline.client.XNATPipelineLauncher.launch(XNATPipelineLauncher.java:65)
    at org.nrg.pipeline.client.XNATPipelineLauncher.run(XNATPipelineLauncher.java:231)
    at org.nrg.pipeline.client.XNATPipelineLauncher.main(XNATPipelineLauncher.java:221)

], html=[<html>The processing request you submitted for 11235811_MR1 could not be completed at this time. 

[...]

I would appreciate any help, as I don't want to waste anymore time googling unsuccessfully through the Internet. I tried your tip and when I entered:

http://localhost:8080/xnat/data/archive/projects/TP1/resources/validation_xnat_mrSessionData/files
it says:

NameSizeURIcollectionfile_tagsfile_formatfile_contentcat_ID

I don't know if this helps to identify the problem. The same error occurs at 66% at the DicomToNifti pipeline. This error is reproducible on another machine, too.

Thanks for your help,
Philip

Herrick, Rick

unread,
Sep 10, 2013, 10:20:50 AM9/10/13
to xnat_di...@googlegroups.com
Philip,

Have you tried re-running the pipeline from the command line directly? You can usually find the full command in XNAT's application.log. If the failed pipeline command is from more than a day or two ago, you should also modify the command line to change the values for the -u and -p options. The values displayed there now are alias tokens and expire after some time. You could fail authentication just because those values are too old. You can substitute in any username and password, but you do want to be careful to not expose those values through the process list or anything similar. If you're on a dev machine, that's fairly low risk though.

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

(314) 827-4250


From: xnat_di...@googlegroups.com [xnat_di...@googlegroups.com] on behalf of plxna...@gmail.com [plxna...@gmail.com]
Sent: Tuesday, September 10, 2013 8:45 AM
To: xnat_di...@googlegroups.com
Subject: [XNAT Discussion] Re: ProtocolCheckPipeline

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at http://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/groups/opt_out.



The material in this message is private and may contain Protected Healthcare Information (PHI). If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

plxna...@gmail.com

unread,
Sep 11, 2013, 3:15:13 AM9/11/13
to xnat_di...@googlegroups.com
Hello Rick,

I tried your tip, but my application.log is empty... additionally I don't know exactly how to run a pipeline from the command line, so I would really appreciate any short tutorial/guide/link on how to do this. I would really love to try to replace the tokens with an owner-login to see if it still fails, because your explanation seems pretty reasonable to me.

Thank you for your help,
Philip

Mohana Ramaratnam

unread,
Sep 11, 2013, 3:26:55 AM9/11/13
to xnat_di...@googlegroups.com
Hi Philip,

There are five steps to get the Protocol Validation pipeline working. These are:

1) Adding the Validation pipeline to the XNAT site

2) Creating the project specific schematron file for validation

3) Uploading the file created in Step 2, to the project as a resource. 

4) Setting up the pipeline for a project

5) Running the pipeline for a session. 


Now, when you perform step 3, you associate a label to the uploaded schematron file and use this label in Step 4 (pipeline parameter catalog_content). 

I suspect that you are missing Step 3 and using default values in Step 4 and hence your pipeline is failing. 


--

plxna...@gmail.com

unread,
Sep 11, 2013, 8:17:59 AM9/11/13
to xnat_di...@googlegroups.com
Hello Mohana,

you were 100% right. Everything works fine now. Apparently I skipped step 3 the first time, or well, I didn't know I had to do this. THANKS A LOT.

Philip
Reply all
Reply to author
Forward
0 new messages