Cannot launch command via REST API

[Jens Petersen]

Hi all,

I'm running XNAT 1.7.2 in with Vagrant (the xnat-release config) and installed the container-service from here: https://github.com/NrgXnat/container-service, the pre-built jar version 1.0-alpha.1.
I tried to follow the tutorial at https://wiki.xnat.org/display/XW2/Part+2%3A+XNAT+Container+Service, which apparently isn't quite up-to-date, but I managed to get a few docker containers into the XNAT.

My first question would be:
The distinction between on-server and in-database was apparently removed and POSTs to xapi/docker/images are no longer possible. Now all my images only have an image-id (sha256:....) and a tag only. Is there a way to add information to the image?

Also:
Saving a command from the image label doesn't do anything, tried with your dcm2niix container, is this known?

Anyway, I can access the images via the image-id (there is no id, which is what the tutorial uses), so I moved on to the commands. Defining them works just fine, but I can't seem to launch them, tried both via xapi/commands/launch and xapi/commands/{id}/launch (Is there a functional difference?). I always receive 500 codes and there's nothing in the logs (checked mostly spring.log). Errors for other requests find their way into the spring.log, but in this case there's nothing. I tried via the Swagger UI, DataClient, curl, python requests, I always get an entire webpage in return. Authentication should be fine however, I was logged in or included authentication in the request. Is "allRequestParams" still being used? In the Tutorials it was context, so maybe the Swagger UI is just not up-to-date. How should I format the request to get it to work?

Thanks in advance for your help! Awesome work by the way, XNAT's a fantastic project overall.

All the best,
Jens

[John Flavin]

Jens,

Thanks for trying the container service! Before I answer, I have to give all the caveats: it is very new, not ready for release yet, it might break your system in unexpected ways, etc. But since you’re using it in a VM, it seems like you aren't using it on a production system with all your real data, so it should be okay.

Yes, the workshop documentation you were working from is pretty old. I have more recent documentation elsewhere on the wiki: https://wiki.xnat.org/display/CS/Command documents the structure of the command; https://wiki.xnat.org/display/CS/Hackathon is a guide I wrote when we had an internal hackathon around the container service, so it has some useful "getting started" info and some completely irrelevant info about where I had set up a server for the hackathon.

The 1.0-alpha.1 jar that you have was created for the hackathon. I've made a lot of improvements since then, and we've got some UI for things like sitewide management. I plan to put out another alpha release in the next couple days, so if you can wait for that I would recommend it. Note, though, that this new version requires some XNAT code that isn't released yet, so you would need to build your vagrant VM with xnat-latest.

My first question would be:
The distinction between on-server and in-database was apparently removed and POSTs to xapi/docker/images are no longer possible. Now all my images only have an image-id (sha256:....) and a tag only. Is there a way to add information to the image?

You are correct that I removed the idea of "saving" images in the database. It added additional complexity without providing any benefits. Now the docker server is the only source of truth about what images you have.

Also:
Saving a command from the image label doesn't do anything, tried with your dcm2niix container, is this known?

I think the xnat/dcm2niix image I have up on docker hub probably won't work with the container service jar you have. I've updated the command definition since releasing that jar, and the xnat/dcm2niix image has a command defined in the new way. 

That does raise the idea that I should be tagging these images with the version(s) of the container service that can read them. I'll think about that.

How should I format the request to get it to work?

There was a bug in that version when launching with swagger. The way I wrote the container launching API endpoint, I intended for users to send in whatever parameters they needed for their launch as a bunch of "name=value" query params, which I would read into a variable named "allRequestParams". But when swagger read the code and generated its UI, it interpreted that as a single parameter named "allRequestParams". So that's sub-optimal.

Sill, you should be able to launch containers with the version you have. I don't know exactly how you’re formatting the commands you used to launch, but (I think) these should work:

curl

curl -u ${username}:${password} -X POST ${xnat}/xapi/commands/${commandId}/launch\?paramName1=paramValue1\&paramName2=paramValue2\&...

python

import requests
s = requests.Session()
s.auth = (username, password)
r = s.post(xnat + '/xapi/commands/%d/launch' % commandId, params = {'paramName1': 'paramValue1', 'paramName2': 'paramValue2', ...})

As to the logs, yes, logging is a problem for XNAT plugins. Everything gets dumped into the application.log at the default level "info". When I'm developing, I have to modify XNAT's log4j.properties to bump up the logging level to "debug" for my container service code and my dependencies, and to create a new file. That's something I'm going to see if I can fix, but as of now it's still a problem.

In summary:

* Thanks for trying this.

* Wait just a couple days for a new release.

Let me know if you find any more issues. Or even if things just don't work like you expect. And feel free to open issues on the github repo, or make pull requests, or whatever. All the work happens on the "dev" branch.

Flavin

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To post to this group, send email to xnat_di...@googlegroups.com.
Visit this group at https://groups.google.com/group/xnat_discussion.
For more options, visit https://groups.google.com/d/optout.

[Jens Petersen]

Wow, thanks for taking the time to write such a detailed reply. I'll wait for the new release and in the meantime try and get the dev version to work. I have one issue that's been bothering me these last days, but it's more a general XNAT one, not with respect to the container service. Do you use the bugtracker on bitbucket for that? Or should that rather go in the reboot repo. There's a lot to choose from :D Anyway, I just don't understand why the DICOM import would label a session with the PatientID (0010,0020) instead of the StudyID (0020,0010). Is that a bug?
Thanks again, looking forward to the new CS release!

[Herrick, Rick]

It’s not a bug, just the way it’s been done with XNAT historically. I honestly have no idea why that field was used in particular. It would be pretty easy to change in code, but at this point does require changing Java code.

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[Jens Petersen]

Thanks for the reply, Rick! Could you point me to where I can change that in the code? All I could find were somewhat abstract implementations, so I figured the actual assignment was configured somewhere in an xml, but I can't find any useful xml files, i.e. ones that configure DICOM related stuff. Should dicom-import-context.xml still exist and if so, where can I find that? I have both the 1.7.2 release and the dev with the latest source running using Vagrant.
Thank you so much!

[Daniel Marcus]

Jens,

The reason Patient ID is used for the session label is b/c that's a field that users can easily enter at the scanner console, whereas Study ID is not. Some details on how XNAT translates DICOM metadata are here and here. 

I wonder if instead of modifying the java, which I suspect would require you to custom build XNAT as the DICOM import part of the codebase probably can't be put into a plugin (Rick, correct me if I'm wrong on that), perhaps you could set up a site-wide DICOM edit configuration that copies the value from Study ID to Patient ID (or Patient Comments).  Then when the study is routed to the project, XNAT's project-level DICOM import would pick up your intended session label  I have no idea if that would work, as I've forgotten the exact details of the DICOM import, but it might be worth trying.

-Dan

To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussion+unsubscribe@googlegroups.com.
To post to this group, send email to xnat_discussion@googlegroups.com.

[Herrick, Rick]

It would require Java code, but you can create a DICOM object identifier on its own and include that in a plugin. 1.7.3 will add the ability to set the DICOM object identifier on a per-DICOM receiver basis (actually, that ability is already there but without any UI: that UI is what will be added in 1.7.3). So you could create a new receiver and set your alternative DICOM object identifier for that receiver. I had started a sample DICOM object identifier for the template plugin project but haven’t committed that yet.

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

 

Voice: (314) 273-1645

Web: http://www.xnat.org

Twitter: @NrgXnat

[Jens Petersen]

Hey Dan, thanks for your reply. Are there more ways to implement such a site-wide edit than with the anonymization script? Because by the time that's called, the session label has already been determined, based on the fact that the default script assigns the "session" variable to the PatientID field. In general I'd like to be able to control in more detail the "sorting" of scans. One of your examples is an implementation of the DicomObjectIdentifier that sorts data into a given project. Can I implement something similar for the session label? And if so, where can I find the xml that makes my projects actually use that identifier? Alternatively, can I implement a custom session builder? I'm quite happy that I managed to get XNAT to work from the xnat-web source and if possible I'd like to leave the other packages (dicom-xnat-...) untouched.
Again, thanks for all the help, really appreciate it!

[Jens Petersen]

I'm fine with writing some code, but I think an entire plugin is more than necessary. There are just two things right now that don't work the way I want them to:

1. I'd like to skip the prearchive. There's an option for that in the project settings, but for that to work the import has to recognize the correct project. A solution would be FixedDicomProjectIdentifier.
2. I want to label sessions with the StudyID.

I was hoping that I can just extend the CompositeDicomObjectIdentifier/implement a custom DicomObjectIdentifier. Should that work in principle without a custom plugin? I tried to get the fixed project import to work as laid out here: https://wiki.xnat.org/display/XNAT16/DICOM+identifier+configuration+and+customization. Because the FixedDicomProjectIdentifier already exists, my thinking was that I just have to have the correct dicom-import-context.xml, however the file does not exist. I'm using the 1.7.2 dev vagrant XNAT with up-to-date source for xnat-web and pipeline-engine. I tried just placing it in xnat-web/src/main/webapp/WEB-INF/conf, but without effect. Has something changed recently in this regard?

[Jens Petersen]

Hey, sorry to bother you again. Could you give me any pointers on the two points I mentioned in my previous post? And maybe more specifically how to get XNAT to use a custom ObjectIdentifier, seeing that the dicom-import-context.xml file no longer exists (apparently).
Again, many thanks!

[Herrick, Rick]

Hi Jens,

 

FixedDicomProjectIdentifier would work to route the DICOM to a single project, but right now that’s only included in one CompositeDicomObjectIdentifier implementation, which is FixedProjectSubjectDicomObjectIdentifier. That identifier is mostly for development purposes, since it only sends to a single fixed subject as well as a single fixed project. It would be very easy to just copy the ClassicDicomObjectIdentifier class, replace Xnat15DicomProjectIdentifier with FixedDicomProjectIdentifier, and create a session extractor that uses the StudyID tag. I actually did this and have attached it to this post.

 

To get this up and running, you can create an @XnatPlugin-annotated class that creates an instance of the identifier, something like this:

 

@XnatPlugin(value = "myPlugin", name = "My Plugin")
public class MyPlugin {
    @Bean
    public CompositeDicomObjectIdentifier myProjectIdentifier(final XnatUserProvider userProvider) {
        return new FixedProjectStudyIdDicomObjectIdentifier("My Project Identifier", "My_Project", userProvider);
    }
}

 

Once that’s up and running, you can assign it to a DICOM receiver by setting the identifier property on the receiver to the appropriate value, in this case “myProjectIdentifer”:

 

{"aeTitle": "HELLO", "port": 8105, "identifier": "myProjectIdentifier"}

 

We plan to make this easier to configure dynamically (i.e. without code) at some point, but we’re just not there right now.

[Jens Petersen]

Rick, thank you so much for your reply! I think I'm getting close :) I put your identifier in a minimal plugin and the plugin is recognized as long as it doesn't try to instantiate the new identifier. If it does, I just get a blank page from XNAT. The spring.log says the following:

org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'testIdentifier' defined in org.dkfz.xnat.nrad.plugin.NradPlugin: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.nrg.xnat.utils.XnatUserProvider]: No qualifying bean of type [org.nrg.xnat.utils.XnatUserProvider] is defined: expected single matching bean but found 2: primaryAdminUserProvider,receivedFileUserProvider; nested exception is org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type [org.nrg.xnat.utils.XnatUserProvider] is defined: expected single matching bean but found 2: primaryAdminUserProvider,receivedFileUserProvider

I'm guessing the receivedFileUserProvider is the one it should be using. How can I resolve this?
Thanks again!

[Herrick, Rick]

Ah sorry, my bad. I should have thought of this. The way these dependencies are tied together uses the Spring Framework’s annotation configuration function. In that framework, when you have an @Bean-annotated method like the myProjectIdentifier() method below, Spring tries to resolve any parameters to the method by looking in the application context, which is essentially a map of object instances with the bean name as the key. The ways that it tries to fill in those parameters is by looking for:

 

·         Beans that have the same name as the parameter (and type: if it finds a bean with the same name but wrong type, you’ll get an exception for that)

·         Beans that have the same type as the parameter

 

In the sample case below, it looks for a bean named userProvider. There is no bean named userProvider, so it moves on to find a unique bean of type XnatUserProvider. As you can see, there is no unique bean of type XnatUserProvider, so it doesn’t know which one to wire in as the parameter submitted to the myProjectIdentifier() method.

 

But it’s an easy fix. Just change the name of your parameter:

 

@XnatPlugin(value = "myPlugin", name = "My Plugin")
public class MyPlugin {
    @Bean

    public CompositeDicomObjectIdentifier myProjectIdentifier(final XnatUserProvider receivedFileUserProvider) {

        return new FixedProjectStudyIdDicomObjectIdentifier("My Project Identifier", "My_Project", userProvider);
    }
}

 

And you’re correct: you want to use the receivedFileUserProvider instance. You could also create your own user provider if, for some reason, you wanted received file operations for a particular project identifier or DICOM receiver to be logged under that user’s name, but that’s pretty straightforward.

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

 

Voice: (314) 273-1645

Web: http://www.xnat.org

Twitter: @NrgXnat

 

[Jens Petersen]

Rick, again, thank you so much for your help. I'm now getting the following error:

Factory method 'testIdentifier' threw exception; nested exception is java.lang.NoSuchMethodError: org.nrg.dcm.id.CompositeDicomObjectIdentifier.<init>(Ljava/lang/String;Lorg/nrg/dcm/id/DicomProjectIdentifier;Ljava/lang/Iterable;Ljava/lang/Iterable;Ljava/lang/Iterable;)V

but as far as I can tell, this

public CompositeDicomObjectIdentifier(final String name,
                                          final DicomProjectIdentifier identifier,
                                          final Iterable<Extractor> subjectExtractors,
                                          final Iterable<Extractor> sessionExtractors,
                                          final Iterable<Extractor> aaExtractors)

should be the matching signature. I honestly don't have any idea why this doesn't work. Have you seen something like that before?

Jens

[Herrick, Rick]

Without seeing the code it’s hard to say. I’m actually not even sure where you’re getting that error, so there could be a few reasons why. Can you provide the method that’s trying to instantiate the identifier, as well as the identifier code?

[Jens Petersen]

Hi Rick,

I attached the two files my plugin uses.

Best,
Jens