Pipeline multiple parallel runs

[shreyasi misra]

Hi,

I had a doubt regarding how I could run the same pipeline with different runtime arguments simultaneously and parallel instead of one after the other.

Also, if I schedule a pipeline to run every 5 minutes, and the previous pipeline is running for more than 5 minutes. Does the next scheduled pipeline deployment get stalled till the previous workflow is complete or is it started simultaneously?

Thanks

[Albert Shau]

Hi Shreyasi,

In the UI, the schedule has a property for the maximum number of concurrent runs. If there are fewer than the max, the run will begin without delay. If there are max concurrent runs already, the run will be rejected and skipped. Note that this only applies to pipelines started by schedules. Runs started directly through the API or the run button on the UI will not check these constraints. By default, the UI sets max concurrent runs to 1, which disallows multiple concurrent runs.

However, there are many more options (like waiting for some time instead of rejecting immediately) allowed if you create a schedule through the REST API (https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477560983/Lifecycle+Microservices#Add-a-Schedule).

Regards,

Albert

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/5e1a11e4-6254-4f02-8b0f-d1e7664f52fen%40googlegroups.com.

[shreyasi misra]

Hi Albert,

Thank you for your response. Understood, thanks.