Running pipelines from REST API

[Andy Shieh]

I'm recently looking for ways to automate custom pipeline triggering and monitoring through REST API.

1. Is there a way to get the workflow ID of the launched pipeline in the response, so that we can use the workflow ID to check the status of the pipeline?

For launching a pipeline, I'm currently using the POST data/projects/{PROJECT_ID}/pipelines/{PIPELINE_NAME}/experiments/{EXPERIMENT_LABEL} API. This API call seems to only return a status code and nothing else.

2. Is there a GET API we can use to list all pipelines of the entire site/of a project/of a subject/of an experiment? By pipelines I meant pipelines we add to the pipeline engine, and not just any generic workflow like "Adding Resource File" "Changing Session Notes" etc.

Thank you very much.

Andy

[Will Horton]

Andy,

Firstly, I have to say that our ability to support pipeline execution is somewhat limited, as we have been working in containerized processing for some time and recommend that workflow moving forward.

We do have documentation of existing pipeline APIs (including a deprecated endpoint that lists all installed pipelines per project), but I assume you may already have seen that and are using it to the best of its ability. (Pipeline API - Workflow API - Project Pipeline API)

That said, there was a significant effort by folks in the Human Connectome Project to build automated pipeline monitoring and processing, prior to development of the Container Service, and they ended up moving all processing out of pipelines and into scripts so that they could avoid the limitations of the pipeline engine. I will forward your question on to the HCP developers to see if they at least have a method of getting the workflow ID in a scripted way.

Regards,

Will

[Michael Hodge]

Hi Andy,

We no longer use the pipeline engine, but XNAT does have a workflows API that I think will get you what you need.  You might need to do some coding to handle the return of multiple entries, but but with a little code around it to filter for the best result, I think this will get you what you need:

https://wiki.xnat.org/display/XAPI/XNAT+Workflow+API#XNATWorkflowAPI-GetAWorkflowEntryByQuerystringParameters

Regards,

Mike