Container Service Plugin - dropdown selection feature

Ajay Kurani

unread,

Mar 30, 2023, 5:45:27 PM3/30/23

to xnat_discussion

To whom it may concern:

We are creating our first containers on XNAT 1.8 using the container plugin. We have some projects where the T1 image is collected on Day 1 and a backup T1 on day 2, as well as the resting state scan is collected on day 2. We want to be able to select day 1 T1 and day 2 resting state in some cases (especially if day 2 T1 has motion corruption). When we create the Contain Launch Values box, we want to be able to have a dropdown appear with the multiple T1 scans available. Is there a way to create such a dropdown that uses the scan type (e.g. MPRAGE) to dynamically show the matching items (e.g. Day1: MPRAGE, Day 2:MPRAGE)?

Thanks,

Ajay

John Flavin

unread,

Apr 3, 2023, 1:49:05 PM4/3/23

to xnat_di...@googlegroups.com

Based on your description this sounds possible to me. Or if it is not possible directly as you describe, then it is almost certain possible with slight modifications. The Launch Container interface is generated for you based on the Command and Command Wrapper that you define for your container, and selecting scans to satisfy inputs is absolutely something that you can do.

It looks to me like you would want to launch at the Subject level—i.e. your Command Wrapper would have an external input with type Subject. From there I think you would want to have two derived inputs of type Session which are both derived from the subject input, then two more derived inputs for the T1 and resting state scans, respectively, which are each derived from one of the session inputs.

Given those inputs, I think the resulting launch workflow and UI would be like this:

User navigates to a subject, brings up container launch UI
The subject input value is automatically populated because it is known from the context.
The two session inputs are populated with choices for all the sessions under the subject. For one, the user selects the session containing the T1 scan; for the other, the session with the resting state scan. (Those could in general be the same session or different sessions.)
Based on those session selections, the two scan inputs populate with the scans on the respective sessions. The user selects the T1 scan and the resting state scan from the options.

That isn't the only possibility, though. You could instead have an external input of type Session (which would mean the user would launch from a session page), from there derive a Subject input, from the Subject derive an input for the second Session, then have the two Scan inputs as before. The main difference is what page the user launches from, but one or the other possible layout might make it easier to store your outputs where you want them to go.

Based on your description it sounds to me like you would prefer to have the launch UI only contain two inputs for the scans, with the possible values taken from all the matching scans in all the subject's sessions. I don't think that is (currently) possible. You will likely need to include inputs for the sessions, from which the scan input values will be derived. It's not theoretically impossible to implement something like this in the Container Service, but it's more complex than it might first appear.

Ideally you could define appropriate matchers on the session and scan inputs so that the appropriate sessions and scans would be selected automatically based on the criteria you provide and the user would not need to actually select anything. That may or may not be possible, depending on a lot of things about your data and so forth that I don't know.

Good luck with your tasks ahead! Let us know if you need any help.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/b2eeed5a-ff89-4dd2-93c8-3f538f11cacen%40googlegroups.com.

Ajay Kurani

unread,

Apr 14, 2023, 11:56:08 AM4/14/23

to xnat_discussion

Hi John,

Thank you for the great suggestions. We will let you know if there are any other issues.

Thanks,

Ajay

Ajay Kurani

unread,

Jun 27, 2023, 5:06:17 PM6/27/23

to xnat_discussion

Hi John,

I wanted to see how to implement the following in the command.json file:

I want the user to select multiple scan types for a given variable called T1w_Scantype. Right now the for selecting multiple values I typically use the following under inputs but it requires me to a priori enter values from select-values :

{
"name": "T1w_scan_type",
"description": "User specified T1w scan types to process",
"type": "select-multiple",
"required": true,
"select-values": ["value1", "value2"]
}

Is there a way to use derived inputs section to list all scan-types in that particular session and have this fill the select-values dropdown instead of filling out a priori as this would change project to project and possibly session to session? If not, is there another mechanism to achieve this?

Thanks,

Ajay.

John Flavin

unread,

Jun 29, 2023, 10:32:41 AM6/29/23

to xnat_di...@googlegroups.com

Unfortunately, no, I can't think of a way to do what you want to do. There isn't anything within the command (or the "command resolution" mechanism that processes it internally) which can take a group of things and collect them into a single thing. In more concrete terms, within the command you can derive all the scans from a session, and you can derive all their scan types, but you can't then collect all those scan types into a list and use that for the "select-values" of another input. We can filter and match, but can't gather.

This is something that is only just barely beyond the current capabilities of the container service. I'll capture this use case you've described, though, so if we ever do find time to work on expanding this capability I'll make sure it can do what you asked for here.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/a6fd5876-8be0-47c1-9f0d-087f3e4d57b1n%40googlegroups.com.

Ajay Kurani

unread,

Jun 29, 2023, 10:47:35 AM6/29/23

to xnat_discussion

Hi John,

Thank you very much. In the meantime can you let me know what is the best way to structure the command to accept multiple "scan-types" if the user types this in.

In using type "string" I know how to create the box but what is the best way to allow multiple entries? Is there an option that we have to set to allow this, as I assume the default is one value. Is there an automatic delimiter applied one this is typed in or do we specify in directions to use a semicolon (or other delimiter)?

Additionally, for this use case of dropdowns that are fed inputs the reason for this case is the following:

1) For pipelines such as MRIQC you may want to run all T1 scans (as defined with your scan-types you select from the dropdown), along with T2, bold and dwi. By allowing multiple entries of scan-type per session you can first run scan-type cleanup on a project and then be able to identify the multiple items. A more generalized case of this would be the original email where you want to use a T1 scan selected on one session as the reference T1 for all analyses but the resting state is collected on a different day (we have multi-day studies). So by being able to filter at a subject level for scans you can do this sort of selection.

2) Another use case is BIDS conversion where you have to select what is T1, T2, FLAIR ....etc etc and you have multiple scans in a session. Being able to filter with usable because you want the second T1 scan is beneficial, or for selecting scans that will be combined such as multi-shell diffusion imaging where the shells are in different runs.

3) For cases where you have multiple field maps (PEpolar per BIDS naming convention).

I have seen many use cases for these so any help would be much appreciated as you look into it. Looking forward to the next update regarding dropdowns! Thanks in advance for the clarification on the multi-string inputs as the intermediate solution.

Best,

Ajay

Ajay Kurani

unread,

Jun 29, 2023, 4:11:55 PM6/29/23

to xnat_discussion

Hi John,

One other question/feedback is for the "name" field. I was hoping to use the "name" field similar to the description where I write actual text like the following:

:

"name": "MRIQC Pipeline Version",

"description": "Select the version of the MRIQC pipeline to run:",

"replacement-key": "PIPELINE_VERSION"

:

Is it possible to have the "name" field accept inputs with spaces as long as the "replacement-key" field is present and without spaces? This would help the GUI look cleaner and more instructive, while the functionality is preserved with the replacement-key value.

John Flavin

unread,

Jun 30, 2023, 10:07:46 AM6/30/23

to xnat_di...@googlegroups.com

In the past, when we have needed to accept multiple options in a single input field we've used a comma-separated string. But that's just one convention, you can do whatever delimiter you like so long as the users know how to enter the input and the container that receives the input knows how to interpret the delimited values it gets.

If I remember correctly the "name" field can't have spaces. But there is a "label" field on the input that you could use for this purpose.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/bfe5d60a-5034-4d09-a76c-4205da0cc7acn%40googlegroups.com.

Ajay Kurani

unread,

Jul 7, 2023, 3:12:00 AM7/7/23

to xnat_discussion

Thanks! For the command.json is there a way to populate inputs subject_id and session_id both from an xnat object that is subjectData context and imageSessionData context? I want the same pipeline to work regardless of if it is launched on the project or session.......below is some code I'm trying to get working....

{
"name": "mriqc",
"label": "mriqc",
"description": "Run mriqc on subject-level",
"version": "1.0",
"schema-version": "1.0",
"image": "image:5000/mriqc:1.0",
"type": "docker",
"command-line": "python mriqc_v1.0.py mriqc #PIPELINE_VERSION# #ANALYSIS_NAME# #PHI_STATUS# #PROJECT# #SUBJECT# #QUEST_ACCOUNT# #MODALITY# #SESSION# #SCAN_TYPE#",
"mounts": [
{
"name": "tmp-out",
"writable": true,
"path": "/output"
}
],
"environment-variables": {},
"ports": {},
"inputs": [
{
"name": "SUBJECT",
"type": "string",
"required": true,
"select-values": []
},
{
"name": "T1w_SCANTYPES",
"description": "Enter T1-weighted scantypes to analyze. Seperate multiple entries with a comma (MPRAGE,T1).",
"type": "string",
"matcher": "@.quality == 'usable'",
"required": false,
"select-values": []
},
{
"name": "T2w_SCANTYPES",
"description": "Enter T2-weighted scantypes to analyze. Seperate multiple entries with a comma (T2,T2w).",
"type": "string",
"matcher": "@.quality == 'usable'",
"required": false,
"select-values": []
},
{
"name": "PROJECT",
"type": "string",
"required": true,
"select-values": []
},
{
"name": "Analysis_Name",
"description": "Output name the processed data will be stored under.",
"type": "string",
"default-value": "mriqc",
"required": true,
"replacement-key": "ANALYSIS_NAME",
"select-values": []
},
{
"name": "SESSION",
"type": "string",
"required": true,
"select-values": []
}
],
"outputs": [
{
"name": "tmp_docker",
"description": "space for txt file",
"required": true,
"mount": "tmp-out"
}
],
"xnat": [
{
"name": "mriqc",
"label": "mriqc",
"description": "Run mriqc on subject-level",
"contexts": [
"xnat:subjectData"
],
"external-inputs": [
{
"name": "subject",
"type": "Subject",
"required": true,
"provides-value-for-command-input": "SUBJECT",
"load-children": true
}
],
"derived-inputs": [
{
"name": "project",
"type": "string",
"required": true,
"provides-value-for-command-input": "PROJECT",
"user-settable": false,
"load-children": true,
"derived-from-wrapper-input": "subject",
"derived-from-xnat-object-property": "project-id",
"multiple": false
},
{
"name": "session",
"type": "Session",
"required": true,
"provides-value-for-command-input": "SESSION",
"load-children": true,
"derived-from-wrapper-input": "subject",
"multiple": true
}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
},
{
"name": "mriqc-sess",
"label": "mriqc",
"description": "Run mriqc on session-level",
"contexts": [
"xnat:imageSessionData"
],
"external-inputs": [
{
"name": "session",
"type": "Session",
"required": true,
"provides-value-for-command-input": "SESSION",
"load-children": true
},
{
"name": "subject",
"type": "Subject",
"required": true,
"provides-value-for-command-input": "SUBJECT",
"load-children": true
}
],
"derived-inputs": [
{
"name": "session_id",
"type": "string",
"required": true,
"provides-value-for-command-input": "SESSION",
"load-children": true,
"derived-from-wrapper-input": "session",
"derived-from-xnat-object-property": "id",
"multiple": false
},
{
"name": "project",
"type": "string",
"required": true,
"provides-value-for-command-input": "PROJECT",
"user-settable": false,
"load-children": true,
"derived-from-wrapper-input": "subject",
"derived-from-xnat-object-property": "project-id",
"multiple": false
}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "session",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
}
],
"container-labels": {},
"generic-resources": {},
"ulimits": {},
"secrets": []
}

Any help debugging would be much appreciated!

Best,

Ajay

John Flavin

unread,

Jul 11, 2023, 11:09:46 AM7/11/23

to xnat_di...@googlegroups.com

Yes, I think you can do what you’re trying to do there with a few tweaks.

First, if you want to pass the session and subject ids to the container inputs, you will need to derive them explicitly. Something like this should work for the session:

{
    "name": "session-id",
    "description": "The session's id",
    "type": "string",


    "derived-from-wrapper-input": "session",
    "derived-from-xnat-object-property": "id",

    "provides-value-for-command-input": "session_id"
}

And the subject id would be similar.

Second, I see that you have two command wrappers, one for launching at the subject level and another for launching at the session level. That's perfect, I think that is the right approach. For the subject level you have the subject as the external input, which looks correct to me, but for the session level you have both the session and subject as external inputs. I wouldn't do that. I would have only the session as the external input, and derive the subject from the session.

Those were the things that jumped out at me. Let us know how it goes!

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/3acd571c-c0b2-4416-ae37-86dd9ce03b47n%40googlegroups.com.

Ajay Kurani

unread,

Jul 25, 2023, 3:09:37 PM7/25/23

to xnat_discussion

Hi John,

Thanks for the feedback. The last problem I am stuck on is related to listing all scan types across sessions for a given subject. I was able to list all scans for the context "xnat:imageSessionData"' and have it as a user selectable menu. This same structure does not work when moving one level up to subject delve. Any suggestions?

Thanks

Ajay

{
"name": "scans",
"type": "Scan",

"matcher": "@.quality == 'usable'",

"required": true,
"provides-value-for-command-input": "SCANS",

"load-children": true,
"derived-from-wrapper-input": "subject",
"multiple": true
}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]

John Flavin

unread,

Jul 26, 2023, 11:01:02 AM7/26/23

to xnat_di...@googlegroups.com

My guess: The scans input currently has "derived-from-wrapper-input": "subject", but that needs to be derived from session, not subject. Subjects don't have scans.

Also, a couple side notes. You should generally set "load-children": false on projects. Otherwise all the objects in the hierarchy for the entire project will be loaded into memory. Sometimes you need that, but more often project inputs are used to mount the entire project into the container and aren't used to derive anything in the command. In that case loading the children (i.e. loading all the subjects and their resources and so on) is just a waste of time.

In this specific instance it doesn't look to me like you’re using that project derived input for anything. You could simply remove it.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/fd89cafd-be94-43f4-a265-11d6a8395b5bn%40googlegroups.com.

Ajay Kurani

unread,

Jul 26, 2023, 2:26:57 PM7/26/23

to xnat_discussion

Hi John,

Please see the simplified code below. The "context" is xnat:subjectData but when I change T1scan to "derived-from-wrapper-input": "session" the GUI fails to open with the following error:

Error undefined: Unknown Reason

Could not launch UI with value: "NURIPS_DEV_S02067" for root element: "subject".

When I change the derived-from-wrapper-input to "subject" the GUI loads but the value for T1scan is empty as it is derived from the session level. Is there a way at the subject "context" to get the session level information as the way you suggested above does not work? I do not want to add imageSessionData as a secondary context as the full interface does not behave as expected on the full version of the code. The overall goal is to have all scantypes from sessions 1 and 2 listed in the GUI that meet the condition "usable" for the user to select using the "multiple": true line, similar to how it works on the derived-input session variable.

Thanks,

Ajay

{
"name": "mriqc",
"label": "mriqc",
"description": "Run mriqc on subject-level",

"version": "1.0",
"schema-version": "1.0",
"image": "image:5000/mriqc:1.0",
"type": "docker",

"command-line": "python mriqc_v1.0.py mriqc #PIPELINE_VERSION# #ANALYSIS_NAME# subject #SUBJECT# # #SESSION# #T1w_SCANTYPES#",

"mounts": [
{
"name": "tmp-out",
"writable": true,
"path": "/output"
}
],
"environment-variables": {},
"ports": {},
"inputs": [
{

"name": "MRIQC_Pipeline_Version",
"label": "MRIQC",
"description": "Select which version of the pipeline to run.",
"type": "select-one",
"default-value": "v0.16.1",
"required": true,
"replacement-key": "PIPELINE_VERSION",
"select-values": [
"v0.16.1"
]
},
{
"name": "SESSION",
"label": "Session:",

"type": "string",
"required": true,
"select-values": []
},
{
"name": "Analysis_Name",

"label": "Analysis Name:",

"description": "Output name the processed data will be stored under.",
"type": "string",
"default-value": "mriqc",
"required": true,
"replacement-key": "ANALYSIS_NAME",
"select-values": []
},
{

"name": "T1w_SCANTYPES",
"label": "T1w Scantypes:",
"description": "Enter T1-weighted scantypes to analyze. Seperate multiple entries with a comma (T1,T1w)",

"type": "string",
"required": true,
"select-values": []
}
],
"outputs": [
{
"name": "tmp_docker",
"description": "space for txt file",
"required": true,
"mount": "tmp-out"
}
],
"xnat": [
{
"name": "mriqc",
"label": "mriqc",
"description": "Run mriqc on subject-level",
"contexts": [
"xnat:subjectData"
],
"external-inputs": [
{

"name": "SUBJECT",

"type": "Subject",
"required": true,

"load-children": false
}
],
"derived-inputs": [

{
"name": "session",
"type": "Session",
"required": true,
"provides-value-for-command-input": "SESSION",
"load-children": true,
"derived-from-wrapper-input": "subject",
"multiple": true
},
{

"name": "T1scan",

"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": true,

"provides-value-for-command-input": "T1w_SCANTYPES",

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": true
}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
}
],

"container-labels": {},
"generic-resources": {},
"ulimits": {},
"secrets": []
}

Ajay Kurani

unread,

Oct 20, 2023, 12:07:33 PM10/20/23

to xnat_discussion

Hi John,

I hope all is going well. I wanted to follow up with this use case to see if "command resolution" is on the roadmap with any specific timeline or on the to-do list, as it will help us make a few decisions on our end for next steps. Any insights would be much appreciated.

Thanks,

Ajay

On Thursday, June 29, 2023 at 9:32:41 AM UTC-5 John Flavin wrote:

John Flavin

unread,

Oct 23, 2023, 11:16:13 AM10/23/23

to xnat_di...@googlegroups.com

Ajay,

Sorry, it has been a while since we had this conversation so I have lost track of some of the details of what you were asking about.

Command resolution is already a thing that happens in the Container Service. That's what we call the process that starts with the Command as a template for what to run and whatever input values you provide, loads the relevant XNAT data, and determines all the specific values to launch a container job on the compute backend. So Command Resolution as a whole can't really be on the roadmap because it already exists; but I'm guessing you had something more specific in mind.

I've reviewed the past conversation, and it appears that you want a way to aggregate all the usable scan types into an input at the subject level. I am not sure that is possible. I can try it and see, though.

These kinds of aggregations are a weak point in the current capabilities of the Command. It isn't hard to derive inputs down a couple levels and get N objects of a certain kind, but there aren't a lot of tools within the Command syntax to do anything with those N items. It seems to me like what you would want is a way to derive a new input which takes the N scan types and combines them into a single CSV string (with non-repeating values). That's only one possible option of what someone might want to do with those N inputs, though. You can imagine all the aggregation functions from a GROUPBY operation in a database or dataframe; max, min, mean, and concat all jump to mind.

I do have another possibility for you to consider, though. Perhaps you could launch the session-level Command from the subject level with the Batch Launch Plugin. I'm not 100% sure it will do what you want, but if it does then you wouldn't need to wait around indefinitely for us to implement better aggregation capabilities in the Command.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/c614b241-3499-408d-9619-99da32229d82n%40googlegroups.com.

Ajay Kurani

unread,

Oct 26, 2023, 1:35:51 AM10/26/23

to xnat_discussion

Hi John,

Thanks for the follow up. For the use case I had in mind, for certain projects they have multi-day scanning so for a given subject they have 3 session: Day1, Day2 and Day3. They have collected a T1 and T2 image on day 1 and the resting state scan on day 3. For me, if the dropdown is in the format session:scantype for all matcher "usable" such as [Day1: T1, Day1: T2, Day1: task1, Day2:hiresMPRAGE, Day2:task2, Day3:rsfmri, Day3:dti] then the user can select from the "subject-level" which T1-weighted image they want to use and which task/resting state scan for a given pipeline. For instance they may want to use Day2:hiresMPRAGE, Day3:rsfmri as the input for a resting state pipeline and then we batch launch these options for the entire project for all subjects. This is similar to how a particular project was processed in the past. Another example is with mriqc where the user may want to process Day1:T1 and Day2:hiresMPRAGE as the T1 scan inputs since MRIQC can run on multiple modalities at the same time.

I wasn't clear if it is possible to do this at the subject level where it lists all "usable" scans in the dropdown. My current fix is having a string input where they list the scan-type but this does not help if there are repeated scans of T1w unless you mark one as unusable. The first option of listing session:scantype for all modalities that match criteria across sessions would be ideal. As from my previous email, my attempt to do this at the session level was successful in the case that all scans needed for an analysis are present in that session. However, in the case of multi-day scans where each day is a separate session that was uploaded, I was unable to do this at the subject-level. It could be that I'm just setting up my command.json incorrectly to do this and that this functionality already exists.

Can you confirm if this is indeed a new use case that is currently not possible, or if I just made a mistake in my subject-level command and that I should be able to list all scans across sessions that the user highlights?

Thanks,

Ajay

John Flavin

unread,

Oct 26, 2023, 6:06:00 PM10/26/23

to xnat_di...@googlegroups.com

Thank you for that clear description of your ideal use case. I think that we might be able to do something that will work for you.

Could you try something along these lines?

"derived-inputs": [
	{
		"name": "T1session",
		"type": "Session",
		"derived-from-wrapper-input": "subject",
		"required": true
	},
	{


		"name": "T1scan",
		"type": "Scan",

		"derived-from-wrapper-input": "T1session",
		"matcher": "@.quality == 'usable'",
		"required": true
	},
	{
		"name": "MPRsession",
		"type": "Session",
		"derived-from-wrapper-input": "subject",
		"required": true
	},
	{
		"name": "MPRscan",
		"type": "Scan",
		"derived-from-wrapper-input": "MPRsession",


		"matcher": "@.quality == 'usable'",
		"required": true
	},

	...
]

With this kind of pattern, the user could select a session for, say, the T1, and then the T1scan input values would only show the scans from that selected session.

You'd need to do something with these inputs after that, of course. Maybe mount the files, maybe pass some property to the command-line string, maybe something else. I'll leave that as an exercise for the reader. But I'm thinking this kind of derived input pattern will at least let you get a UI that will work for your use case and let users select all the scans they need from the subject level.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/f6ed6974-b7d6-4000-bb8d-71c535cfc7c8n%40googlegroups.com.

Ajay Kurani

unread,

Oct 29, 2023, 11:02:35 PM10/29/23

to xnat_discussion

Hi John,

I tried doing a simple example with your suggestion below. While the configuration saved, when I went to run the container at the subject level i got the following error. I'm not sure if there is something you see that would explain the error.

Thanks,

Ajay

Error undefined: Unknown Reason

Could not launch UI with value: "XNAT_S02070" for root element: "subject".

{
"name": "mriqc",
"label": "mriqc",
"description": "Run mriqc on subject-level",
"version": "1.0",
"schema-version": "1.0",

"image": "mriqc:1.0",
"type": "docker",
"command-line": "python /usr/local/bin/mriqc_v1.0.py #PROJECT# #SUBJECT# #SESSION#",

"mounts": [
{
"name": "tmp-out",
"writable": true,
"path": "/output"
}
],
"environment-variables": {},
"ports": {},
"inputs": [
{

"name": "PROJECT",
"label": "Project:",

"type": "string",
"required": true,

"command-line-flag": "--project",
"command-line-separator": "=",
"select-values": []
},
{
"name": "SUBJECT",
"label": "Subject:",

"type": "string",
"required": true,

"command-line-flag": "--subject",
"command-line-separator": "=",

"select-values": []
},
{
"name": "SESSION",

"label": "Session:",
"type": "string",
"required": true,

"command-line-flag": "--session",
"command-line-separator": "=",

"select-values": []
}
],
"outputs": [
{
"name": "tmp_docker",
"description": "space for txt file",
"required": true,
"mount": "tmp-out"
}
],
"xnat": [
{
"name": "mriqc",
"label": "mriqc",

"description": "Run mriqc v1.0 on subject-level",

"contexts": [
"xnat:subjectData"
],
"external-inputs": [
{
"name": "subject",
"type": "Subject",
"required": true,
"provides-value-for-command-input": "SUBJECT",
"load-children": true
}
],
"derived-inputs": [
{
"name": "project",
"type": "string",
"required": true,
"provides-value-for-command-input": "PROJECT",
"user-settable": false,
"load-children": true,
"derived-from-wrapper-input": "subject",
"derived-from-xnat-object-property": "project-id",
"multiple": false
},
{

"name": "T1session",
"type": "Session",

"required": true,

"load-children": true,
"derived-from-wrapper-input": "subject",

"multiple": false

},
{
"name": "T1scan",
"type": "Scan",

"matcher": "@.quality == 'usable'",
"required": true,

"load-children": true,
"derived-from-wrapper-input": "T1session",

"multiple": false
}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",

"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
}
],
"container-labels": {},
"generic-resources": {},
"ulimits": {},
"secrets": []
}

John Flavin

unread,

Oct 30, 2023, 11:24:05 AM10/30/23

to xnat_di...@googlegroups.com

I don't know. I pasted that command JSON in exactly as you have it here, no edits, and the UI came up fine for me with no errors.

Could you try deleting it and re-adding it?

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/9c8efe7b-1671-4a9e-8ca1-153b67428cdfn%40googlegroups.com.

Ajay Kurani

unread,

Oct 30, 2023, 11:50:56 AM10/30/23

to xnat_discussion

Hi John,

I removed and re-added and while the command saves, when I go to a subject-level to run the container I'm still receiving that error that the GUI won't pop up. To give you info on our dev system, it is running XNAT 1.8.9.1 and Container Service plugin v3.4.0-fat. Is there any specific logs you think would help troubleshoot this issue? If you don't mind, would you paste what worked for you and I'll copy and paste that in case I have something I missed like an extra space etc.

Thanks,

Ajay

John Flavin

unread,

Oct 30, 2023, 2:23:08 PM10/30/23

to xnat_di...@googlegroups.com

There was a bug introduced in CS 3.3.2 and fixed in 3.4.1 which causes the container launch UI to fail to generate when a nested derived input has multiple possible values. That is definitely the case with your inputs.

I think updating to CS 3.4.1 (or the very recently released 3.4.2) will fix the error you’re seeing.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/Him

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/fce054c9-c601-4b14-8b1d-fa7cefe4fbabn%40googlegroups.com.

Ajay Kurani

unread,

Oct 30, 2023, 2:32:09 PM10/30/23

to xnat_discussion

Hi John,

Thank you very much for the help. I will do that and follow up if there are any other issues.

Best,

Ajay

Ajay Kurani

unread,

Nov 17, 2023, 2:33:54 PM11/17/23

to xnat_discussion

Hi John,

I was able to get things working on the subject and session level, but when I do batch launch as below all scans will get selected as it's based solely on the @quality matcher since I'm not selecting from a dropdown. I was thinking the best way to get this fixed is add a variable called "T1type" that is a string the user sets and then use it with the matcher function. I tried using "matcher": "@.scan-type == #T1type#", but it did not work. I wasn't sure if this is the proper syntax or if there is another way I should be passing the user-typed field into the matcher, or if this part of the limitations of "gathering" you mentioned in the first email. Any suggestions would be much appreciated.

Best,

Ajay

T1w Scantype:

[Derived from session Scan scan-type via matcher]
@.quality == 'usable'

Values for first session as a sample:
2 - 3D T1 _weighted
7 - 2D GRE-NM_MT
8 - 2D GRE-NM_MT
9 - 2D GRE-NM_MT
10 - 2D GRE-NM_MT
11 - 2D GRE-NM_MT
14 - 3D T2 FLAIR

John Flavin

unread,

Nov 20, 2023, 12:28:11 PM11/20/23

to xnat_di...@googlegroups.com

I was thinking the best way to get this fixed is add a variable called "T1type" that is a string the user sets and then use it with the matcher function. I tried using "matcher": "@.scan-type == #T1type#", but it did not work. I was thinking the best way to get this fixed is add a variable called "T1type" that is a string the user sets and then use it with the matcher function. I tried using "matcher": "@.scan-type == #T1type#", but it did not work.

I would think this approach could work. Can you tell me what error you got when you tried this? Was anything logged in the containers.log or the containers-services-commandresolution.log file?

Just as a guess, the issue might be that the template value is being replaced in your matcher string, but what results isn't a valid JSONPath matcher. Something you could try is putting single quotes around the #T1type# in the matcher string, like this:

"matcher": "@.scan-type == '#T1type#'"

Again, that's just a guess.

John Flavin

Backend Team Lead

johnf...@flywheel.io

He/They/Any

flywheel.io

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/93a66229-99dd-43e6-9861-c097a93757ccn%40googlegroups.com.

Ajay Kurani

unread,

Apr 4, 2024, 2:27:57 AM4/4/24

to xnat_discussion

Hi John,

I used the derived-inputs area to get the scantype;

{
"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",

"type": "Scan",
"matcher": "@.quality == 'usable'",

"required": false,
"provides-value-for-command-input": "T1w_SCANTYPE",

"load-children": true,
"derived-from-wrapper-input": "session",

"derived-from-xnat-object-property": "scan-type",
"multiple": true
},

The command input line is as follows:

"command-line": "python /usr/local/bin/mriqc.py #PIPELINE_VERSION# #PHI_STATUS# #PROJECT# #SUBJECT# \"#SESSION#\" #ANALYSIS_NAME# #OVERWRITE# \"#T1w_SCANTYPE#\" \"#T2w_SCANTYPE#\" \"#TASK_SCANTYPE#\" \"#REST_SCANTYPE#\" \"#DWI_SCANTYPE#\" #ACCOUNT# #CORES# #RAM# #WALLTIME# #GPU_STATUS#",

The issue is that the input is ["3d MPRAGE"] but it does not fill in T1w_SCANTYPE since it is in brackets.......if it was "3d MPRAGE" it works fine. Is there a different syntax in command-line to accept brackets?

Thanks

Ajay

John Flavin

unread,

Apr 4, 2024, 10:03:28 AM4/4/24

to xnat_di...@googlegroups.com

I’m a bit confused by that input. It looks like it has type Scan, which means each value will be an object with all the properties of a single Scan, but you seem to want to use it as the “scan type”, which is a single property of a Scan.

Could you try splitting that input that you have now into two derived inputs, one for the scan itself (derived from the session) and another for the scan type (derived from a property of the scan)? The latter input is the one that you would want to set as providing the value for the command input. And you could mark that one as “user-settable”: false so it wouldn’t be editable by users in the UI.

I’m not entirely sure if that will fix the issue you’re seeing—in fact I suspect that it won’t, you may be running into a fundamental limitation of the current command resolution process—but I think trying that first will at least help make things more clear for me.

Thanks,

John

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/581d5e5a-be62-49ac-83a0-492a9b6dee72n%40googlegroups.com.

Ajay Kurani

unread,

Apr 10, 2024, 3:54:42 PM4/10/24

to xnat_discussion

I tried saving the split command as I understood from your last message in the derived-input section, but I would get HTTP 400 Error: The server could not process the request. This may be due to an error in the edited command definition.

{
"name": "T1scantype",

"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": true
},
{
"name": "T1scan",
"user-settable": "false",
"required": false,
"load-children": true,
"provides-value-for-command-input": "T1w_SCANTYPE",
"derived-from-wrapper-input": "T1scantype",

"derived-from-xnat-object-property": "scan-type",
"multiple": true
},

The only thing I want is the scan-type of the selected scan by the user for a particular session that is "usable". I have attached the full original command so you can replicate on your end. Additionally, in taking a screen shot of from the processing dashboard I see the raw value fills out fine for T1scan when using the syntax below but the "wrapper-derived" value is null. My format may be overkill as I may not need the derived-input but any guidance would be helpful. Even though T2scan is incorrect (gives scan path instead of scan-type), it fills the raw and derived value and appears in the command for T2w_SCANTYPE correctly, whereas T1w_SCANTYPE remains blanks.

{
"name": "mriqc_singlelevel",
"label": "mriqc_singlelevel",
"description": "Run mriqc on subject- and session-level",
"version": "1.4",
"schema-version": "1.0",
"image": "docker.nu.edu:80/mriqc_single:1.4",
"type": "docker",
"command-line": "python /usr/local/bin/mriqc.py #PIPELINE_VERSION# #PHI_STATUS# #PROJECT# #SUBJECT# \"#SESSION#\" #ANALYSIS_NAME# #OVERWRITE# \"#T1w_SCANTYPE#\" \"#T2w_SCANTYPE#\" #ACCOUNT# #CORES# #RAM# #WALLTIME# #GPU_STATUS#",

"mounts": [
{
"name": "tmp-out",
"writable": true,
"path": "/output"
}
],
"environment-variables": {},
"ports": {},
"inputs": [
{

"name": "PIPELINE_VERSION",

"label": "MRIQC",
"description": "Select which version of the pipeline to run.",
"type": "select-one",
"default-value": "v0.16.1",
"required": true,

"command-line-flag": "--pipeline_version",
"command-line-separator": "=",
"select-values": [
"v0.16.1",
"v23.1.0"
]
},
{
"name": "PHI_STATUS",
"label": "Project PHI Status:",
"description": "User specifies if Patient Health Information is present.\nThis determines processing location as PHI prohibited on Quest.",
"type": "select-one",
"required": true,
"command-line-flag": "--phi_status",
"command-line-separator": "=",
"select-values": [
"Yes",
"No"
]
},

{
"name": "PROJECT",
"label": "Project:",
"type": "string",
"required": true,
"command-line-flag": "--project",
"command-line-separator": "=",
"select-values": []
},
{
"name": "SUBJECT",
"label": "Subject:",
"type": "string",
"required": true,
"command-line-flag": "--subject",
"command-line-separator": "=",
"select-values": []
},
{
"name": "SESSION",
"label": "Session:",
"type": "string",
"required": true,
"command-line-flag": "--session",
"command-line-separator": "=",
"select-values": []
},

{
"name": "ANALYSIS_NAME",

"label": "Analysis Name:",
"description": "Output name the processed data will be stored under.",
"type": "string",
"default-value": "mriqc",
"required": true,

"command-line-flag": "--analysis_name",

"command-line-separator": "=",
"select-values": []
},
{

"name": "OVERWRITE",
"label": "Overwrite Analysis?",
"description": "If analysis name already exists, select Yes to overwrite with existing name, otherwise data will not be processed.",
"type": "select-one",
"default-value": "No",
"required": true,
"command-line-flag": "--overwrite",
"command-line-separator": "=",
"select-values": [
"Yes",
"No"
]
},
{
"name": "T1w_SCANTYPE",
"type": "string",
"required": false,
"command-line-flag": "--t1w",

"command-line-separator": "=",
"select-values": []
},
{

"name": "T2w_SCANTYPE",
"type": "string",
"required": false,
"command-line-flag": "--t2w",

"command-line-separator": "=",
"select-values": []
},
{

"name": "ACCOUNT",
"label": "Quest Account Number:",
"description": "Quest account (pXXXXX or bXXXX) to use for processing both on Quest and NURIPS clusters.\nNURIPS ADMIN must be added to this account to run jobs.",

"type": "string",
"required": true,

"command-line-flag": "--account",

"command-line-separator": "=",
"select-values": []
},
{

"name": "CORES",
"label": "Processing Cores:",
"description": "Number of Cores used for processing.",
"type": "select-one",
"required": true,
"command-line-flag": "--cores",
"command-line-separator": "=",
"select-values": [
"1",
"2",
"4",
"8",
"12"
]
},
{
"name": "RAM",
"label": "RAM (GB):",
"description": "Amount of RAM per core used for processing.\n Recommended: 2-4GB/core. High cores/ram may slow down launch times.",
"type": "select-one",
"required": true,
"command-line-flag": "--ram",
"command-line-separator": "=",
"select-values": [
"2",
"4",
"8",
"16",
"32",
"64"
]
},
{
"name": "WALLTIME",
"label": "Walltime (Hours):",
"description": "Amount of time used for processing.",
"type": "select-one",
"required": true,
"command-line-flag": "--walltime",
"command-line-separator": "=",
"select-values": [
"4",
"8",
"12",
"24",
"48"
]
},
{
"name": "GPU_STATUS",
"label": "Running cuda accelerated code requiring a GPU?",
"description": "Is a GPU required to run this pipeline? NOTE: Limited GPUs available which may slow down launch times.",
"type": "select-one",
"default-value": "No",
"required": true,
"command-line-flag": "--gpu_status",
"command-line-separator": "=",
"select-values": [
"No",
"Yes"

]
}
],
"outputs": [
{
"name": "tmp_docker",
"description": "space for txt file",
"required": true,
"mount": "tmp-out"
}
],
"xnat": [
{

"name": "mriqc-single-sub",
"label": "mriqc-single-sub",

"description": "Run mriqc on subject-level",

"contexts": [
"xnat:subjectData"
],
"external-inputs": [
{
"name": "subject",
"type": "Subject",
"required": true,
"provides-value-for-command-input": "SUBJECT",
"load-children": true
}
],
"derived-inputs": [
{

"name": "session",

"type": "Session",
"required": true,

"provides-value-for-command-input": "SESSION",

"load-children": true,
"derived-from-wrapper-input": "subject",
"multiple": false
},
{

"name": "project",
"type": "string",
"required": true,
"provides-value-for-command-input": "PROJECT",
"user-settable": false,
"load-children": true,
"derived-from-wrapper-input": "subject",
"derived-from-xnat-object-property": "project-id",
"multiple": false
},
{

"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,
"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,
"derived-from-wrapper-input": "session",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
},

{
"name": "T2scan",
"label": "T2w Scantype:",
"description": "Select T2-weighted scans to analyze. Use shift key if multiple entries are required.",

"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,

"provides-value-for-command-input": "T2w_SCANTYPE",

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": true

}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
},

{
"name": "mriqc-single-sess",
"label": "mriqc-single-sess",
"description": "Run mriqc on a session-level",

"contexts": [
"xnat:imageSessionData"
],
"external-inputs": [
{

"name": "session",

"type": "Session",
"required": true,

"provides-value-for-command-input": "SESSION",

"load-children": true
}
],
"derived-inputs": [
{
"name": "project",
"type": "string",
"required": true,
"provides-value-for-command-input": "PROJECT",
"user-settable": false,
"load-children": true,

"derived-from-wrapper-input": "session",

"derived-from-xnat-object-property": "project-id",
"multiple": false
},
{

"name": "subject",
"type": "Subject",
"required": true,
"provides-value-for-command-input": "SUBJECT",

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": false
},
{
"name": "T1scan",

"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,

"load-children": true,
"derived-from-wrapper-input": "session",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
},

{
"name": "T2scan",
"label": "T2w Scantype:",
"description": "Select T2-weighted scans to analyze. Use shift key if multiple entries are required.",

"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,

"provides-value-for-command-input": "T2w_SCANTYPE",

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": true

}
],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",

"as-a-child-of": "session",

Screen1of2.png

Screen2of2.png

Ajay Kurani

unread,

Apr 12, 2024, 4:23:05 PM4/12/24

to xnat_discussion

Hi John,

Just wanted to follow up to see if you had any suggestions for deriving scan-type from a derived scan value. The first part works with getting the scan values, just the second part fails.

Thanks,

Ajay

John Flavin

unread,

Apr 16, 2024, 10:33:17 AM4/16/24

to xnat_di...@googlegroups.com

Hi Ajay,

Sorry for the delayed response. Things have been busy over here.

I am still confused. But not by what you’re saying, I think I understand now what you’re trying to do and what your command is doing. Now I'm confused by the input values you showed.

I think I know why the T1scan value used the scan type but the T2scan value used the scan path instead of the scan type. In T1scan you have "derived-from-xnat-object-property": "scan-type", which you do not have in T2scan. I think if you add that property to T2scan it should have a scan type value.

Another discrepancy I noticed between those two inputs is that T2scan has the property "provides-value-for-command-input": "T2w_SCANTYPE", but T1scan does not have the corresponding property "provides-value-for-command-input": "T1w_SCANTYPE". I think that would be necessary, but I don't know if that explains any of the issues you were seeing.

Lastly, I'm not sure why the T1scan wrapper input didn't have a value resolved when there was a raw value sent in, especially when T2scan did have a resolved value. I wonder if it is due to the fact that T2scan's value was a path rather than a scan type. Maybe CS is able to take a scan path as a raw value and instantiate a full scan object for the derived input during command resolution, but can't do that when the raw value is a scan type. If that hypothesis is correct, then "fixing" the T2scan input by adding "derived-from-xnat-object-property": "scan-type" will "break" it in the same way as T1scan, where the derived input has no value.

And if that is true, then I'm not sure that we will be able to make the inputs work the way you need them to.

John Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/cc827528-c7ab-464a-9b2f-e262bdba115an%40googlegroups.com.

Ajay Kurani

unread,

Apr 19, 2024, 10:52:03 PM4/19/24

to xnat_discussion

Hi John,

No worries I'm sure you guys are slammed and I appreciate your help with this issue. I wanted to clarify a few points:

1) I purposefully put T1scan and T2scan as different properties to see why T2scan/T2w_SCANTYPE has an output (although not what I wanted) while T1scan/T1w_SCANTYPE was empty. I think if the array/brackets aren't getting stripped correctly then regex may not read it in which could be what is going on (total guess).

2) For the screen shots I sent you this was when I launched from a subject level (picking the session). The following is my code (found in the previous email) from the subject level and as you can see I do indeed have "provides-value-for-input": "T1w_SCANTYPE". My session level did not have this as I was trying to test these different configurations out. However the screenshots correspond to the command items below:

{

"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,
"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,
"derived-from-wrapper-input": "session",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
},

{
"name": "T2scan",
"label": "T2w Scantype:",
"description": "Select T2-weighted scans to analyze. Use shift key if multiple entries are required.",

"type": "Scan",
"matcher": "@.quality == 'usable'",
"required": false,

"provides-value-for-command-input": "T2w_SCANTYPE",

"load-children": true,
"derived-from-wrapper-input": "session",

"multiple": true

}

3) I purposefully kept T2scan as Scan and T1scan as scan-type to see why T2scan populates whereas T1scan does not populate the derived value at the end. When adding scan-type to T2scan this also fails so I think it is an issue resolving scan-type. I wanted to see if there is a way to extract the T1scan "raw" value instead of derived value or to have the derived value just match the raw in any way? Additionally, I notice the derived T2scan removes the brackets, but if the T1 scan-type fail in stripping the brackets, regex wouldn't likely recognize the array and it may be why it fails. Is there way to check if this is the case?

Any suggestions forward would be much appreciated.

Thanks,

Ajay

Ajay Kurani

unread,

May 4, 2024, 3:38:45 AM5/4/24

to xnat_discussion

Hi John,

I tested the following code with 2 cases, a single entry selected and multiple entries selected and it worked in the single entry case and failed in the multiple entry case.

Derived Inputs:

{
"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",

"required": false,
"provides-value-for-command-input": "T1w_SCAN",

"load-children": true,
"derived-from-wrapper-input": "session",
"multiple": true
},

{
"name": "T1scantype",

"type": "string",
"required": false,

"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,

"derived-from-wrapper-input": "T1scan",

"derived-from-xnat-object-property": "scan-type",
"multiple": true
}

This generates a dropdown menu and when I select one item the command resolves as expected giving me the scan-type. HOWEVER, when I select 2 items in the drop-down the command fails with the error: T1scan must have precisely one command input child element

This suggests the derived-from-xnat-object-property can resolve a single entry but not multiple comma separated entries which is line with the following thread: https://groups.google.com/g/xnat_discussion/c/LpKW1F2ysDE/m/-i_w67hEEgAJ

In my case I do want to convert all entries into their respective scan-type , which does work with the single entry. If the route still is to create custom function in the image (as suggested in the above link) to do this, I wanted to see if you can point me to the code you use to perform this scan-type resolution so that I can extend it for a list instead of re-inventing the wheel.

Thanks

Ajay

John Flavin

unread,

May 4, 2024, 8:07:40 AM5/4/24

to xnat_di...@googlegroups.com

Hi Ajay,

The process you’re talking about is "command resolution", where the command and wrapper templates are combined with runtime input values and the data in your XNAT to determine all the information needed to launch a container. It is by far the most complex part of the Container Service. Here is a wiki page going over the process: https://wiki.xnat.org/container-service/command-resolution, and here is the code: https://bitbucket.org/xnatdev/container-service/src/master/src/main/java/org/nrg/containers/services/impl/CommandResolutionServiceImpl.java

What you describe is possible in principle. The command resolution process can resolve wrapper inputs into multiple possible values. What it can't do yet is take those multiple wrapper input values and combine them into a single container input value. The reason why is because there are many ways that this could happen, and at the time it was too much complexity for me to manage.

I wrote a whole page (Bulk Launching Containers) about some of the possible things one might want to do with container launches. Most of that document is 1. out of date, and 2. about the more difficult 1 -> N case (1 input resulting in N container launches). But you’re talking about the relatively easy N -> 1 case (N resolved inputs resulting in 1 container launch). I even have a note at the very bottom of that page that seems to me to be more or less what you’re talking about:

Can I use the values of “listable” inputs in the various template strings, like the command line? What happens if input A is listable, and in the command line I need to use the value of A? Is that an error? Or do I just concatenate all N values of input A?

After I wrote that document, I did implement handling for this case in code. A wrapper input is allowed to have multiple resolved values right up until the moment it needs to provide its value to a command input, at which time (as of right now) having multiple resolved values becomes an error.

...Actually, that may not be true. I'm looking through the command resolution code and I think I see that this might have been implemented (but not documented) by Kate during the years I was away from XNAT at a different job. If you have the command input type as "select-multiple" (which I think I remember was in your original command months ago) and you have a property on that input "multiple-delimiter" which can take values "space", "quoted-space", "comma", or "flag", then I think maybe it will be able to combine your multiple resolved derived wrapper inputs into a command line string.

I'm reading the code trying to understand what this will do. It seems like, if you had a wrapper input that got resolved into values ["A", "B"], and it provides the value for a command line input which is a "select-multiple" (and let's say it has a flag "--t1" with no equals sign) then with the different options for "multiple-delimiter" you would get these on the command line.

space: --t1 A B
quoted-space: --t1 'A B'
comma: --t1 A,B
flag: --t1 A --t1 B

Give that a try and see if it works.

John Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/db238d99-ce9b-4829-a911-9e05654381a2n%40googlegroups.com.

Ajay Kurani

unread,

May 7, 2024, 4:35:29 AM5/7/24

to xnat_discussion

Hi John,

Please find the simplified case. below as multiple items for T1scan still causes an issue, in case i'm not handling this correctly. For single selections it now outputs wrapper derived output as ; A INSTEAD OF ["A"]. The error for multiple selected items is T1scan must have precisely one command input child element

{
"name": "mriqc_singlelevel",
"label": "mriqc_singlelevel",
"description": "Run mriqc on subject- and session-level",
"version": "1.4",
"schema-version": "1.0",

"image": "test.edu:80/mriqc_single:1.4",
"type": "docker",
"command-line": "python /usr/local/bin/mriqc.py #PROJECT# #SUBJECT# \"#SESSION#\" \"#T1w_SCANTYPE#\"",

{
"name": "T1w_SCANTYPE",
"type": "string",
"required": false,

"command-line-flag": "-t1w",
"select-values": [],
"multiple-delimiter": "flag"
},
{
"name": "T1w_SCAN",

"type": "string",
"required": false,

"command-line-flag": "-t1s",
"select-values": [],
"multiple-delimiter": "flag"

"required": false,
"provides-value-for-command-input": "T1w_SCAN",
"load-children": true,
"derived-from-wrapper-input": "session",
"multiple": true
},
{
"name": "T1scantype",
"type": "string",
"required": false,
"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,
"derived-from-wrapper-input": "T1scan",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
}

],
"output-handlers": [
{
"name": "tmp-resource",
"accepts-command-output": "tmp_docker",
"as-a-child-of": "subject",
"type": "Resource",
"label": "TMP_DOCKER",
"tags": []
}
]
}
],
"container-labels": {},
"generic-resources": {},
"ulimits": {},
"secrets": []
}

Ajay Kurani

unread,

May 7, 2024, 2:48:51 PM5/7/24

to xnat_discussion

Hi John,

To clarify my last post a bit further the crux of the issue comes in their derived inputs section for this SUBJECT-LEVEL launch as seen in the snippet below. The testing seems not work with the changes suggested. My previous email has the full command.json and also happy to set up a zoom call if that would help. We tested on XNAT 1.8.10 build 117 with container plugin 3.4.3-fat . Is my container plugin have the fixes you described or am I implementing it wrote or could it be the scan-type derived from Scan while ok for the raw value does not work with the fix you described?

1) When I have 1 input for T1scan it resolves the scan and then T1scantype can resolve the scantype for that particular scan. However, when I select 2 scans then T1scan has 2 derived values and so T1scantype I think is not able to handle those 2 values to give me the two scan-types in that list. The error becomes: T1scan must have precisely one command input child element.

Using this method I get T1scan raw : ["A"] but derived output as null. Also for two inputs I get T1scan raw value as ["A", "B"] bit derived value as null

Inputs partial section:

{
"name": "T1w_SCANTYPE",
"type": "string",
"required": false,

"command-line-flag": "-t1w",
"select-values": [],

"multiple-delimiter": "comma"
},

derived-inputs partial section:

{
"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",

"required": false,
"provides-value-for-command-input": "T1w_SCAN",
"load-children": true,
"derived-from-wrapper-input": "session",
"multiple": true
},
{
"name": "T1scantype",
"type": "string",
"required": false,
"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,
"derived-from-wrapper-input": "T1scan",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
}

2) Next, I tried to simplify the SUBJECT-Level derived-input command to the original way where I derive scan-type from T1scan directly. For both the single case and double case, even though I use "multiple-delimiter as comma on the inputs section, the issue is in the derived-inputs where the T1scan raw value is ["A"] or ["A", "B"] but the outputs for single and multiple inputs are null for T1scan wrapper derived inputs.

Inputs partial section:

{
"name": "T1w_SCANTYPE",
"type": "string",
"required": false,

"command-line-flag": "-t1w",
"select-values": [],

"multiple-delimiter": "comma"
},

derived-inputs partial section:

{
"name": "T1scan",
"label": "T1w Scantype:",
"description": "Select T1-weighted scans to analyze. Use shift key if multiple entries are required.",
"type": "Scan",
"required": false,

"provides-value-for-command-input": "T1w_SCANTYPE",
"load-children": true,
"derived-from-wrapper-input": "session",
"derived-from-xnat-object-property": "scan-type",
"multiple": true
}

Thanks,

Ajay

Ajay Kurani

unread,

May 9, 2024, 3:05:16 PM5/9/24

to xnat_discussion

Hi John,

I wanted to follow up as I did a bit more testing. My input (T1w_SCANTYPE) will not switch to select-multiple since the values are provided from the derived inputs section. It keeps switching it back to string even after I save and open again. Also when having derived-inputs as the following example, the T1scan raw value fills out ok with ["A", "B] as discussed but the wrapper-derived/resolved T1scan value returns null. This may be due to the fact that a list is the output but T1w_SCANTYPE is of type string. In what you sent me it sounds like it may work IF T1scan is properly resolved, but for some reason even though the raw value is correct the "resolved" final (wrapper-derived T1scan) output keeps saying "null" in the logs . Please let me know if there are any further suggestions.

Thanks for the help!

Ajay

Ajay Kurani

unread,

May 13, 2024, 10:45:07 AM5/13/24

to xnat_discussion

Hi John,

I was able to resolve the issue by passing the scan pathway to my container, parsing out the session and scan number from the path to form a unique entry and then looking up scantype via the rest-api, in case this path is useful for others in the future.

Best,

Ajay

John Flavin

unread,

May 13, 2024, 12:13:04 PM5/13/24

to xnat_di...@googlegroups.com

Hi Ajay,

I'm really glad you were able to figure something out that worked for you. I think that what you were trying to do with the command sounds completely reasonable and should ideally be possible and supported, but my guess is that CS just wasn't going to be able to do it. But I appreciate your willingness to continue trying to make it work. I hope that one day we can improve CS's command resolution and make things like this possible.

John Flavin

To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/cad84800-5dd1-49a3-98e8-f827e27dc16dn%40googlegroups.com.

Reply all

Reply to author

Forward