container json matcher, how to list folders in resource

67 views
Skip to first unread message

Schnei Gamu

unread,
Aug 28, 2024, 4:15:41 PM8/28/24
to xnat_discussion
Hi all,

After writing analysis results to a new resource folder in a session we would like to be able to use that new folder as input to an new analysis. 
Below is the folder structure, we would like to get for this example all mriqc_run1, mriqc_run2 etc listed in a dropdown:

- Resources
      -mriqc   
     -mriqc_run1  
     -mriqc_run2
      -freesurfer
     -freesurfer_run1  
     -freesurfer_run2

We can get the top-level resource folders listed in a drop down, but not the subfolders inside. Could you have a look at the json below to see if I did something wrong or advise us if this is possible at all. 

---------------------
"external-inputs": [
          {
            "name": "session",
            "description": "Input session",
            "type": "Session",
            "required": true,
            "provides-files-for-command-mount": "in",
            "load-children": true
          }
        ],
        "derived-inputs": [
          {
            "name": "resource",
            "description": "Input resource: all",
            "type": "Resource",
            "matcher": "@.label  =~ /mri.*/",
            "required": true,
            "provides-value-for-command-input": "RESOURCE",
            "load-children": true,
            "derived-from-wrapper-input": "session",
            "derived-from-xnat-object-property": "label",
            "multiple": false
          },
          {
            "name": "resource_subfolder",
            "description": "Input resource: second level",
            "type": "Resource",
            "matcher": "@.label  =~ /mri.*/",
            "required": true,
            "provides-value-for-command-input": "RESOURCE_SUBFOLDER",
            "load-children": true,
            "derived-from-wrapper-input": "resource",
            "derived-from-xnat-object-property": "label",
            "multiple": false
          }
        ],
-----------------

The first derived input "resource" shows a dropdown, but the second derived input "resource_subfolder" stays blank after selecting mriqc. 

Thanks for looking into this,
Nils

Ajay Kurani

unread,
Sep 3, 2024, 3:02:03 PM9/3/24
to xnat_discussion
Hi XNAT Experts,
  I wanted to follow up to see if there were any suggestions as we are looking to use resources mechanism as suggested.  I recall that there is only 1 level for resources (unlike scans), but if we know the pipeline is mriqc is there a way to get a list of the nested folders (mriqc_run1 and mriqc_run2) from the command.json?

Thanks,
Ajay

kel...@wustl.edu

unread,
Sep 13, 2024, 3:00:13 PM9/13/24
to xnat_discussion
Hi Ajay,
 I think I've found something that could work for your workflow.  In XNAT, "Resource" data types are not nested, so your mriqc_run* directories are seen as File(s) within a Session Resource.  "Directory" is a valid Container Service derived-input type, but listing and mounting directories under a resource is not a supported feature (or at least I could not get it to work as needed).

With that in mind, a small modification to your resource_subfolder block will get you part of the way there.

                {
                    "name": "resource_subfolder",
                    "description": "Input resource: second level",

                    "type": "File",
                    "required": true,
                    "load-children": false,
                    "derived-from-wrapper-input": "resource",
                    "derived-from-xnat-object-property": "uri",
                    "multiple": false
                },


What this gets you is essentially a flat listing of all files within the selected session resource. Although listing only first level resource subfolders is not exactly supported, we are able to select files within a resource and match a pattern to those files.  In this example (attached and below) , we can find specific subfolders in our session resource by filtering for a unique file within subfolders of interest.  I've added a file named ".root" to each of the subfolders of interest, then used the matcher "@.name  =~ /.*.root/" to identified only those subfolders for the select-box.  Finally, the resource_subfolder_name block is used to extract the folder/file name and set the RESOURCE_SUBFOLDER input and replacement key accordingly.  The resultant command line is resolved from 'echo #RESOURCE# #RESOURCE_SUBFOLDER#'  -to-  'echo mriqc mriqc_run1/.root '.  when selecting the 'run1' folder. Using this approach, note that your containerized script would need to strip the /.root string from the subfolder name.

Screenshot 2024-09-13 at 9.48.51 AM.png      Screenshot 2024-09-13 at 9.49.37 AM.png


Hopefully this works for you. Let me know if you run into any other issues along the way.

Best,
Matt





{
    "name": "resource-select",
    "label": "resource-select",
    "description": "",
    "version": "1.0",
    "schema-version": "1.0",
    "image": "busybox:latest",
    "type": "docker",
    "command-line": "echo #RESOURCE# #RESOURCE_SUBFOLDER#",
    "override-entrypoint": true,
    "mounts": [
        {
            "name": "in",
            "writable": false,
            "path": "/input"
        }
    ],
    "environment-variables": {},
    "ports": {},
    "inputs": [
        {
            "name": "RESOURCE",
            "type": "string",
            "required": true,
            "replacement-key": "#RESOURCE#",
            "select-values": []
        },
        {
            "name": "RESOURCE_SUBFOLDER",
            "type": "string",
            "required": true,
            "replacement-key": "#RESOURCE_SUBFOLDER#",
            "select-values": []
        }
    ],
    "outputs": [],
    "xnat": [
        {
            "name": "resource-select",
            "label": "resource-select",
            "description": "Run resource-select.",
            "contexts": ["xnat:imageSessionData"],


            "external-inputs": [
                {
                    "name": "session",
                    "description": "Input session",
                    "type": "Session",
                    "required": true,
                    "provides-files-for-command-mount": "in",
                    "load-children": true
                }
            ],
            "derived-inputs": [
                {
                    "name": "resource",
                    "description": "Input resource: all",
                    "type": "Resource",
                    "matcher": "@.label  =~ /mri.*/",
                    "required": true,
                    "provides-value-for-command-input": "RESOURCE",
                    "load-children": true,
                    "derived-from-wrapper-input": "session",
                    "derived-from-xnat-object-property": "label",
                    "multiple": false
                },
                {
                    "name": "resource_subfolder",
                    "description": "Input resource: second level",

                    "type": "File",
                    "matcher": "@.name  =~ /.*.root/",
                    "required": true,
                    "load-children": false,
                    "derived-from-wrapper-input": "resource",
                    "derived-from-xnat-object-property": "uri",
                    "multiple": false
                },
                {
                    "name": "resource_subfolder_name",
                    "description": "Input resource: second level filename",
                    "type": "string",


                    "required": true,
                    "provides-value-for-command-input": "RESOURCE_SUBFOLDER",

                    "load-children": false,
                    "derived-from-wrapper-input": "resource_subfolder",
                    "derived-from-xnat-object-property": "name",
                    "multiple": false
                }
            ],
            "output-handlers": []
        }
    ]
}

resource-select.json

Ajay Kurani

unread,
Oct 4, 2024, 3:44:31 PM10/4/24
to xnat_discussion
Hi Matt,
   Thanks so much for the help!  In testing it out things worked.  I modified your code such that I can in my filter include mriqc*/ so that it doesn't list other resources for when multiple pipelines are listed.  The next extension is to see how to select a resource that is created in a different session instead of the session listed, so I am going to see how to list resources across sessions.  An example is if you are using the baseline T1 and create a lesion mask and want to use that for subsequent sessions etc.

Best,
Ajay  

Reply all
Reply to author
Forward
0 new messages