Running container on a Subject level to pick multiple tagged PET and MR scans

[Sofia Kola]

Good evening,

We have a MATLAB script that runs a script and takes specific MR and PET scans that are tagged with a specific quality label. Everything is dockerized. The requirement is to run the container on a subject level (meaning the XNAT's context in JSON Command Definition is set to "xnat:subjectData"), and all the other inputs are derived such as the Sessions, Scans and Resources from the subject data for both MR and PET separately. The goal is to ultimately pick scans that are tagged as "analysis" in case of an MR scan under multiple MR sessions and "petavg" in case of a PET scan all under one Subject. 

The issue that I am currently facing is that based on my current JSON Command Definition it is able to only pick one of each scan from each session that are tagged.

For example: In a "Test" project, I have a subject with multiple MR and PET sessions. And two of the four PET sessions have a scan tagged with "petavg" usability each and a single MR scan is tagged with "analysis" in one of the MR sessions. 

I want to pick only those two PET scans under each PET session and the single MR scan to pass it down the container to run the MATLAB script (also mounting of the files in specified in the json cmd definition file). And this could be dynamic, meaning in some other subject IDs it could have 3 PET scans tagged and it should be able to pick all those 3 scans across all sessions.

Is there a way to dynamically achieve this? 

Thanks,

Sofia

[John Flavin]

Hi Sofia,

This kind of "dynamic" input selection is not currently possible to achieve using only the command definition. When resolving the inputs to launch a container, the container service expects that there will be a single object for each input. (For example, if it's a scan input, at launch time it expects to find only a single scan.) 

I think what we have done for cases like this in the past is to mount the session input—which gives your container all the scan files, even ones you don't need—and push the logic of determining which scans you need into your script that runs within the container.

It's not a super satisfying solution, I know. The ability to have multiple objects for an input is absolutely something that I wanted to do and spent a lot of time thinking about, but it is a tricky problem to solve. The possibilities for what you could potentially want to do are almost endless and all of them are difficult to handle correctly. 

(NOTE: What follows is me detailing a lot of the difficulties that this kind of feature would present. Feel free to stop reading now! I thank you for asking the question. It would be super cool if things worked like this and I'm sorry that they don't already. Plus, this is an area of interest for me and I love having the excuse to write about it.)

For instance, let's say you have a scan input that ends up at launch time resolving to N matching scans. What does the container service do with those?

• If the command says the scan input provides files for a mount, CS would need to actually create multiple mount points at different subpaths. That's not something it does right now, but it's potentially doable.
• What if the command has a template in the CLI string that uses some property of that scan input? Does CS repeat the template with each scan object, separating them by spaces? Or does it fill the template in once, joining the N values with commas or some other separator? 
  
• What if the command also derives another input from the scan, like a resource? Then the N scan objects -> N resource objects (assuming all scans have a matching resource). CS would similarly need to have unique subpaths for each resource within the single mount. But if all the resources have the same label, like DICOM, then all those subpaths would collide unless we also included the scan ID in the path. That means CS would need to remember the whole history of how an input was derived when deciding how to deal with it. And does the command need to specify how to construct those subpaths, or is that a standard thing CS always does the same way?

All of the above assumes you want to launch a single container even when an input matches N objects. But what if you don't? What if you want your N matching objects to turn into N container launches, each with only one of the matching inputs? So if I got N matching scans I would launch N containers with one scan each. Further, what if the command has multiple levels of derived inputs that have different numbers of matching inputs; like the root/external input is a subject from which are derived sessions and scans, and I expect that to turn into (number sessions)x(number scans) container launches with 1 session and 1 scan each? Leaving aside the difficulty in actually implementing that, which is considerable, how would I specify in the command the kind of behavior I expect, either N objects -> 1 launch or N objects -> N launches?

Again, thank you for the question! And thank you for pushing against the limits of the container service and the command. I would be happy to hear your thoughts—or the thoughts of anyone else reading this—on how you would expect the container service to handle this. What would you think you would need to write in the command to control this behavior? 

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/f6a9e6a1-f0aa-4974-b7b2-d609264bd137n%40googlegroups.com.

[Sofia Kola]

Thank you for the quick reply, John! 

I even thought of mounting all the files onto the container under a Subject and then picking the scans needed for processing. But, I was wondering how to get the metadata attached to the files for me to pick the tagged scans. Since, my scans that were to be processed had a usability label attached to it, how can I use it in the container space and use the logic to eventually process those in my MATLAB script.

Although, I am not satisfied with this approach since it takes up a lot of space and it might affect the processing but due to the time crunch I have to choose this route. But, will definitely think over the other possible ways you have listed to "dynamically" input multiple scans. 

Have a great day!

Sofia

You received this message because you are subscribed to a topic in the Google Groups "xnat_discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/xnat_discussion/LpKW1F2ysDE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/CAET4zqfSUeLNksUifXr_eUuZ3XLKr5eBds_uBpu5Ox%3DOzW16qw%40mail.gmail.com.