Examples of Container Service output handling

[tom.g...@gmail.com]

Hi,

I'm just wondering where the best place for Container Service command.json examples. I remember seeing one that contained logic for automatically uploading an assessor but can't seem to find it any more.

In general, what is considered best practice when handling CS session-level outputs? Just upload them to the session resources or add a new assessor to hold them?

Cheers,

Tom

[Will Horton]

Tom, this is an excellent question. We have some examples of Command JSON formats, attached to various container definitions in our code repos. I'm not sure if this is the example that you saw earlier,  but I believe this worked at one point: https://github.com/NrgXnat/docker-images/blob/master/sample-qc-assessor/command.json 

^^ Caveat: I have tested the above definition in XNAT 1.8.5 with CS 3.2, and it currently fails. It gets as far as preparing outputs to write to "out.xml", but the container fails to mount the corresponding directory. I'm escalating to our container service gurus to see if there's an easy fix. 

As far as handling session-level outputs, there are valid use cases for using either assessors or resources. I think the design choice of resource vs assessor basically boils down to a few factors: 

1. Do you want indexed, searchable results? If you do, then an assessor is the best way to go
2. Do you want to allow for multiple container runs that don't overwrite each other? If so, then an assessor is the best way to go 
3. Are you generating output files and you want a single source of truth from any number of container runs? If so, then a session-level resource is best

[Will Horton]

As an update, I had tested the above container on an old development VM of XNAT, which I think was misbehaving. I've tried this again on a fresh build of XNAT 1.8.5 and CS 3.2 and it works perfectly. 

[tom.g...@gmail.com]

Thanks Will, yes that was the one, I had forgotten about the docker-images repo :)

Interesting. We would like the ability to overwrite output files in the case that a parameter needs to be tweaked, is this not possible with assessors or just difficult to do with the XNAT CS command definitions?

[Will Horton]

Tom, 

Sorry if I oversimplified my examples when I was writing up that little decision tree. It is easy to update or overwrite an existing assessor via REST as long as you have the accession ID of that assessor. The Image Assessor API documentation has examples of this. 

As far as getting this to work in the Container Service, I believe in the containerized Freesurfer command workflow, there is an original "Run Freesurfer" command, and then a separate "Relaunch" command that accepts an existing assessor ID as an input, and allows you to tweak parameters. That would be one way of approaching the problem. 

Regards,

Will 

[Charlie Moore]

Hi Tom,

It might help if you can provide a little more detail on what you want to do, because I'm a bit confused. My understanding of your request is you want to have an input of a session, producing a new assessor via CS to store outputs (including files). If so, there isn't really a need to worry about overwriting output files, because relaunching your container with tweaked parameters would produce a new assessor with distinct files from the first run. If instead you're wanting some sort of construct where "each session should only have 1 assessor, we create it if it doesn't exist, but update/overwrite it if it does exist", that's a very different question. Could you clarify which [or maybe I'm off-base and your answer is "neither" :) ] of these you're asking?

Thanks,

Charlie Moore

[John Flavin]

In general, what is considered best practice when handling CS session-level outputs? Just upload them to the session resources or add a new assessor to hold them?

I'm not sure there is a single best practice in general. It depends on what you mean by "outputs" and what you want to do with them. 

If the outputs are structured data for which you have a data type or data that you want to be searchable and/or viewable in a report, they should be written to an assessor. A collection of files that don't need to be searchable would most likely be written to a session resource. But those are just broad generalizations with a decent amount of overlap, so they won't be helpful in every case. Plus those are just the use cases that I came up with off the top of my head; there are plenty of other use cases I didn't mention that would be best served by one or the other way of storing outputs, or maybe even other ways besides those two.

To echo Charlie, could you give us some more detail on the goal you're trying to accomplish?

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/502470cf-3efa-4bbb-a29c-9fe5197afad7n%40googlegroups.com.

[tom.g...@gmail.com]

Hi guys,

Thanks for clarifying.

"If instead you're wanting some sort of construct where "each session should only have 1 assessor, we create it if it doesn't exist, but update/overwrite it if it does exist", that's a very different question." would be closest to what I am wanting to achieve.

For the full detail, maybe have a quick skim of the Arcana docs (https://arcana.readthedocs.io/en/latest/). We would like to be able to update derivatives stored back in XNAT if upstream processing steps change.

Cheers,

Tom