Attempts at pipeline

195 views
Skip to first unread message

David Just

unread,
May 18, 2011, 1:42:57 PM5/18/11
to XNAT_DISCUSSION
I’m back to attempting to get pipeline to run properly,  I’m attempting to add the dicom2nii action from the pipeline catalog for my custom MR Exerpiment type.  I’m fairly certain that I have the pipeline setup properly.  I modified the pipeline xml so that it references my custom type instead of the xnat MRSessionData, and then added it to the available pipelines using the web interface.  I added the build action to my type, and the pipeline to the project.

Any suggestions?

However, when I attempt to prompt the pipeline window I get:
A Pipeline identified by null could not be found for adir:MRExperiment for project MayoInternal


And xdat.log has the following in it:
2011-05-18 11:14:21,082 [http-8080-2] ERROR org.nrg.xft.utils.FileUtils - FileUtils::OutputToFile
java.io.FileNotFoundException: /Applications/Tomcat/webapps/adir_xnat/logs/inserts/pipe:pipelineDetails_/Library/Pipeline_engine/catalog/mriconvert/DicomToNifti.xml.sql (No such file or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at org.nrg.xft.utils.FileUtils.OutputToFile(FileUtils.java:61)
    at org.nrg.xft.XFT.LogInsert(XFT.java:362)
    at org.nrg.xft.db.DBAction.DeleteItem(DBAction.java:3413)
    at org.nrg.xnat.turbine.modules.actions.ManagePipeline.doDelete(ManagePipeline.java:116)
    at org.nrg.xnat.turbine.modules.actions.ManagePipeline.doPerform(ManagePipeline.java:62)
    at org.apache.turbine.modules.actions.VelocityAction.doPerform(VelocityAction.java:46)
    at org.apache.turbine.util.velocity.VelocityActionEvent.perform(VelocityActionEvent.java:82)
    at org.apache.turbine.modules.actions.VelocityAction.perform(VelocityAction.java:72)
    at org.apache.turbine.modules.actions.VelocitySecureAction.perform(VelocitySecureAction.java:61)
    at org.apache.turbine.modules.ActionLoader.exec(ActionLoader.java:96)
    at org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java:113)
    at org.apache.turbine.modules.Page.build(Page.java:53)
    at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:98)
    at org.apache.turbine.Turbine.doGet(Turbine.java:751)
    at org.apache.turbine.Turbine.doPost(Turbine.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:680)
2011-05-18 11:16:36,186 [http-8080-2] ERROR org.nrg.xft.utils.FileUtils - FileUtils::OutputToFile
java.io.FileNotFoundException: /Applications/Tomcat/webapps/adir_xnat/logs/inserts/pipe:pipelineDetails_/Library/Pipeline_engine/catalog/mriconvert/DicomToNifti_adir.xml.sql (No such file or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at org.nrg.xft.utils.FileUtils.OutputToFile(FileUtils.java:61)
    at org.nrg.xft.XFT.LogInsert(XFT.java:362)
    at org.nrg.xft.db.DBAction.DeleteItem(DBAction.java:3413)
    at org.nrg.xnat.turbine.modules.actions.ManagePipeline.doDelete(ManagePipeline.java:116)
    at org.nrg.xnat.turbine.modules.actions.ManagePipeline.doPerform(ManagePipeline.java:62)
    at org.apache.turbine.modules.actions.VelocityAction.doPerform(VelocityAction.java:46)
    at org.apache.turbine.util.velocity.VelocityActionEvent.perform(VelocityActionEvent.java:82)
    at org.apache.turbine.modules.actions.VelocityAction.perform(VelocityAction.java:72)
    at org.apache.turbine.modules.actions.VelocitySecureAction.perform(VelocitySecureAction.java:61)
    at org.apache.turbine.modules.ActionLoader.exec(ActionLoader.java:96)
    at org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java:113)
    at org.apache.turbine.modules.Page.build(Page.java:53)
    at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:98)
    at org.apache.turbine.Turbine.doGet(Turbine.java:751)
    at org.apache.turbine.Turbine.doPost(Turbine.java:846)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:680)
2011-05-18 11:17:16,438 [http-8080-1] ERROR org.nrg.xnat.turbine.modules.screens.DefaultPipelineScreen -
org.nrg.xnat.exceptions.PipelineNotFoundException: A Pipeline identified by null could not be found for adir:MRExperiment for project MayoInternal
    at org.nrg.xdat.om.base.BaseArcProject.getPipelineForDescendantByPath(BaseArcProject.java:136)
    at org.nrg.xnat.turbine.modules.screens.DefaultPipelineScreen.setParameters(DefaultPipelineScreen.java:261)
    at org.nrg.xnat.turbine.modules.screens.DefaultPipelineScreen.doBuildTemplate(DefaultPipelineScreen.java:185)
    at org.nrg.xdat.turbine.modules.screens.SecureScreen.doBuildTemplate(SecureScreen.java:81)
    at org.apache.turbine.modules.screens.TemplateScreen.doBuild(TemplateScreen.java:100)
    at org.apache.turbine.modules.Screen.build(Screen.java:57)
    at org.apache.turbine.modules.ScreenLoader.eval(ScreenLoader.java:104)
    at org.apache.turbine.modules.layouts.VelocityOnlyLayout.doBuild(VelocityOnlyLayout.java:92)
    at org.apache.turbine.modules.Layout.build(Layout.java:53)
    at org.apache.turbine.modules.LayoutLoader.exec(LayoutLoader.java:98)
    at org.apache.turbine.modules.pages.DefaultPage.doBuild(DefaultPage.java:146)
    at org.apache.turbine.modules.Page.build(Page.java:53)
    at org.apache.turbine.modules.PageLoader.exec(PageLoader.java:98)
    at org.apache.turbine.Turbine.doGet(Turbine.java:751)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:680)

Thanks,
Dave.

Mohana Ramaratnam

unread,
May 18, 2011, 11:57:38 PM5/18/11
to xnat_di...@googlegroups.com
Could you send me the pipeline XML document.

Regards
mohana

> --
> You received this message because you are subscribed to the Google Groups
> "xnat_discussion" group.
> To post to this group, send email to xnat_di...@googlegroups.com.
> To unsubscribe from this group, send email to
> xnat_discussi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/xnat_discussion?hl=en.
>

David Just

unread,
May 19, 2011, 10:02:44 AM5/19/11
to XNAT_DISCUSSION
I'm using the pipeline xml file that came with Pipeline 1.5 in
Pipeline/catalog/mriconvert/DicomToNifti.xml The only changes I made are
wherever the xml was referencing xnat:mrSessionData was replaced with
adir:MRExperiment (which extends ImageSessionData. )

I've attached my version of the xml.

>> java:470)
>>     at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>     at
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>     at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
>> 109)
>>     at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>>     at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
>>     at
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http1

DicomToNifti_adir.xml

Mohana Ramaratnam

unread,
May 19, 2011, 4:21:49 PM5/19/11
to xnat_di...@googlegroups.com, Just....@mayo.edu
David,

I added the pipeline that you sent, with the following changes:

a) Changed adir:MRExperiment to xnat:mrSessionData in input-parameters
and xnatInfo elements.

b) Added the pipeline to the XNAT site (as Admin using Administer ->
Pipeline -> Add Pipeline To Repository)

c) Added the DicomToNifti_adir pipeline to my test project

d) Edited the xnat:mrSessionData datatype to include Build under
Actions box on the MrSession report page (as Admin using Administer ->
Datatype -> xnat:mrSessionData -> Edit. See attached image)

e) From the MRSession report, clicked on Build (under Actions).
Selected the DicomToNifti_adir pipeline and selected the scans. This
created a params file in BUILD_DIR/PROJECT/DATESTAMP/SESSION_LABEL
folder.

I didnt encounter the error that you faced.

Mohana

EditDataTypeSetupBuild.png

David Gutman

unread,
May 19, 2011, 10:12:24 PM5/19/11
to xnat_di...@googlegroups.com
I just installed a clean build of 1.5 and so far so good...

If I want to try and get the dcm2nii pipeline working... can I follow
the same (more or less) steps that Mohana presented at the workshop
(i.e. the ones in the video ) or have their been a lot of changes from
1.4 to 1.5 that would make that a bad idea..


dg

On Thu, May 19, 2011 at 4:21 PM, Mohana Ramaratnam

--
David A Gutman, M.D. Ph.D.
Center for Comprehensive Informatics
Emory University School of Medicine

Mohana Ramaratnam

unread,
May 20, 2011, 9:08:15 AM5/20/11
to xnat_di...@googlegroups.com
David,

You could run the dcm2nii pipeline as is. It should work.

Before you start, the dependency for that pipeline are:

a) Install ArcGet. Modify PIPELINE_HOME/catalog/xnat_tools/ArcGet.xml
to set the tag pip:location to point to the location where ArcGet is
installed.

b) Install dcm2nii. Modify
PIPELINE_HOME/catalog/mricron/resources/dcm2nii.xml to set the tag
pip:location to point to the location where dcm2nii is installed.

Mohana

David Gutman

unread,
May 20, 2011, 9:13:01 AM5/20/11
to xnat_di...@googlegroups.com
Ok great--- I'll try it later today and let you know if I get
stuck.... it's been a while since I've messed around with pipeline
stuff..

Thanks!

Are the directions to kick off the pipeline via the REST api also work?


dg

On Fri, May 20, 2011 at 9:08 AM, Mohana Ramaratnam

Mohana Ramaratnam

unread,
May 20, 2011, 9:15:12 AM5/20/11
to xnat_di...@googlegroups.com
Yes they do .

Mohana

dave J

unread,
May 20, 2011, 9:20:58 AM5/20/11
to xnat_discussion
Wait, I was under the impression that dcm2nii dependencies were
included in the pipeline catalog samples.

Are there instructions somewhere that outline the dependencies and how
to configure each of the pipelines that are shipped in the catalog
folder?



On May 20, 8:15 am, Mohana Ramaratnam <mohanakann...@gmail.com> wrote:
> Yes they do .
>
> Mohana
>
>
>
>
>
>
>
> On Fri, May 20, 2011 at 6:43 PM, David Gutman <dagut...@gmail.com> wrote:
> > Ok great--- I'll try it later today and let you know if I get
> > stuck....  it's been a while since I've messed around with pipeline
> > stuff..
>
> > Thanks!
>
> > Are the directions to kick off the pipeline via the REST api also  work?
>
> > dg
>
> > On Fri, May 20, 2011 at 9:08 AM, Mohana Ramaratnam
> > <mohanakann...@gmail.com> wrote:
> >> David,
>
> >> You could run the dcm2nii pipeline as is. It should work.
>
> >> Before you start, the dependency for that pipeline are:
>
> >> a) Install ArcGet. Modify PIPELINE_HOME/catalog/xnat_tools/ArcGet.xml
> >> to set the tag pip:location to point to the location where ArcGet is
> >> installed.
>
> >> b) Install dcm2nii. Modify
> >> PIPELINE_HOME/catalog/mricron/resources/dcm2nii.xml to set the tag
> >> pip:location to point to the location where dcm2nii is installed.
>
> >> Mohana
>
> >>>> On Thu, May 19, 2011 at 7:32 PM, David Just <Just.Da...@mayo.edu> wrote:
> >>>>> I'm using the pipeline xml file that came with Pipeline 1.5 in
> >>>>> Pipeline/catalog/mriconvert/DicomToNifti.xml  The only changes I made are
> >>>>> wherever the xml was referencing xnat:mrSessionData  was replaced with
> >>>>> adir:MRExperiment (which extends ImageSessionData. )
>
> >>>>> I've attached my version of the xml.
>
> >>>>> On 5/18/11 10:57 PM, "Mohana Ramaratnam" <mohanakann...@gmail.com> wrote:
>
> >>>>>> Could you send me the pipeline XML document.
>
> >>>>>> Regards
> >>>>>> mohana
>
> ...
>
> read more »

Mohana Ramaratnam

unread,
May 20, 2011, 11:31:56 AM5/20/11
to xnat_di...@googlegroups.com
Dave,

Pipeline ships with the process flow not the executables. You will
have to install executables. This is done as XNAT cannt get into
license issues with third party tools.

Regards
Mohana

dave J

unread,
May 24, 2011, 2:14:57 PM5/24/11
to xnat_discussion
How do you kill a pipeline that isn't running but XNAT says it is.
My pipeline tests keep failing, and I get the failure notification,
but XNAT still says they are Queued so I can't re-start it. How do I
get rid of them without ending/rebuild/restarting xnat.

Thanks,
Dave.

On May 20, 10:31 am, Mohana Ramaratnam <mohanakann...@gmail.com>
wrote:
> ...
>
> read more »

David Gutman

unread,
May 24, 2011, 2:17:01 PM5/24/11
to xnat_di...@googlegroups.com
I have a similar number of phantom pipelines running... I've gone into
the ADMIN screen and looked at the running pipelines.. clicked on it..
then went in and manually killed it within the given patient screen..
but obviously this becomes extremely tedious..


dg

> --
> You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
> To post to this group, send email to xnat_di...@googlegroups.com.
> To unsubscribe from this group, send email to xnat_discussi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/xnat_discussion?hl=en.
>
>

--

Mohana Ramaratnam

unread,
May 24, 2011, 2:33:34 PM5/24/11
to xnat_di...@googlegroups.com
Dave,

You could delete the row from wrk_workflowdata table in Postgres

OR create an Actions box on the workflow report page with a Delete option.

Here are the steps:

As Site admin go to Administer -> Data Types

Setup additional datatypes

Select wrk:workflowdata

Click Next

Click Next (ie dont enter anything)

This would result in having an Actions box on the default report page
of the Workflowdata. Now we will add the delete option.

Now go back to Administer -> Datatypes

Select wrk:workflowdata

Select Edit

Enter details into an additional row as:

Name = delete (notice case)

Display Name = Delete

Image = delete.gif

Sequence = 4

(leave other columns empty on this row).

Click on Submit.

Go to your session report, under History click on the queued pipeline.
This will take you to workflow report page with an Action box on top
right corner with a Delete option.

regards,
Mohana

David Just

unread,
May 24, 2011, 2:50:09 PM5/24/11
to XNAT_DISCUSSION
Can you elaborate on what you mean by session report? Is that the
Mrsession page?

Mohana Ramaratnam

unread,
May 24, 2011, 3:03:58 PM5/24/11
to xnat_di...@googlegroups.com
Yes by session report I mean the MRSession report.

David Just

unread,
May 24, 2011, 3:44:08 PM5/24/11
to XNAT_DISCUSSION
XNAT ARCGET failing with an invalid Session ID that is known to be valid?

/Users/bin/xnat_tools/ArcGet -u admin -p #### -host
http://localhost:8080/adir_xnat/ -s "adirxnat_E00001" -o
/Library/XNAT/BUILD/MayoInternal/xxx/xxx/RAW -r xxx -z true
Validating MR Session ID: adirxnat_E00001
Sending Request...
Response Received (49 ms)


No Matches Found.
ERROR CODE 90: INVALID SESSION ID 'adirxnat_E00001'..

Arc-Get Web Service
....

(XXX replaces confidential info)
ArcGet spits out the line "Validating MR Session ID". Could it be looking
for an MRSession instead of ImageSessionData, or MR Experiment (which is my
custom type that extends ImageSessionData?

Thanks,
Dave.

Reply all
Reply to author
Forward
0 new messages