Abort only a child pipeline

14 views
Skip to first unread message

Mayank Bhagya

unread,
Mar 1, 2016, 8:35:30 AM3/1/16
to Google App Engine Pipeline API
Hello,

Is there any way in which I can abort only a child pipeline without aborting the entire thing upto its root?

Thanks
-mayank

Brett Slatkin

unread,
Mar 1, 2016, 11:41:22 AM3/1/16
to app-engine-...@googlegroups.com
No, the goal of abort is to fully abort the entire hierarchy. If you want to do retries like that you'll need to use the return value of a pipeline to cause retries at a higher level. That's much more "functional" looking and may require a pattern that looks like recursion.

-Brett


--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Arie Ozarov

unread,
Mar 1, 2016, 11:48:10 AM3/1/16
to app-engine-...@googlegroups.com

Is the question regarding the Python or the Java Pipeline implementation.
If for Python then you have Brett's answer. If for Java, you can still
apply the same concept though you could also use the exception handling mechanism.

Mayank Bhagya

unread,
Mar 1, 2016, 12:51:31 PM3/1/16
to Google App Engine Pipeline API
Thanks Brett. I wasn't trying retries.

I was running a batch-like job where child pipelines represent a single unit of work.
So, my parent pipeline spawned about 1000 child pipelines, one 'after' the other.
So if 795th job fails, I still wanted the remaining jobs to finish rather than aborting the entire thing and preventing the subsequent pipelines from running.

Thanks.
-mayank


On Tuesday, 1 March 2016 22:11:22 UTC+5:30, Brett wrote:
No, the goal of abort is to fully abort the entire hierarchy. If you want to do retries like that you'll need to use the return value of a pipeline to cause retries at a higher level. That's much more "functional" looking and may require a pattern that looks like recursion.

-Brett
On Tue, Mar 1, 2016 at 5:35 AM Mayank Bhagya <mayank...@gmail.com> wrote:
Hello,

Is there any way in which I can abort only a child pipeline without aborting the entire thing upto its root?

Thanks
-mayank

--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeline-api+unsub...@googlegroups.com.

Mayank Bhagya

unread,
Mar 1, 2016, 12:52:17 PM3/1/16
to Google App Engine Pipeline API
Hello Arie. Yes, this was regarding python.


On Tuesday, 1 March 2016 22:18:10 UTC+5:30, Arie Ozarov wrote:

Is the question regarding the Python or the Java Pipeline implementation.
If for Python then you have Brett's answer. If for Java, you can still
apply the same concept though you could also use the exception handling mechanism.


On Tue, Mar 1, 2016 at 8:41 AM, 'Brett Slatkin' via Google App Engine Pipeline API <app-engine-...@googlegroups.com> wrote:
No, the goal of abort is to fully abort the entire hierarchy. If you want to do retries like that you'll need to use the return value of a pipeline to cause retries at a higher level. That's much more "functional" looking and may require a pattern that looks like recursion.

-Brett
On Tue, Mar 1, 2016 at 5:35 AM Mayank Bhagya <mayank...@gmail.com> wrote:
Hello,

Is there any way in which I can abort only a child pipeline without aborting the entire thing upto its root?

Thanks
-mayank

--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeline-api+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeline-api+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages