Handling a job "giving up"

[Andrew Ducker]

Let's say I have a job that reads in input from a URL.  Clearly this might go wrong if the server is down, or overloaded.

Normally, I want that job to try a few times - something that's easy enough to do with a JobSetting.  I can tell it to try again an hour later, and see if the server is back up again.

And I don't need to bother the user who set up this job with the fact that it's failed - not if it's going to succeed later.

However, if the job _totally_ fails then I probably want to let the user know "By the way, that process you wanted run, it failed, and here's why."

Is there any way for the Pipeline API to handle that?

Thanks,

Andy

[Tom Kaitchuck]

You can define your own handleException() method on your worker where you can have custom logic.

https://code.google.com/p/appengine-pipeline/wiki/ErrorHandling
This method can be overloaded based on exception type. If it is the most specific type that matches the thrown exception will be called. 

If there is not a handleException method for a given job it will propagate upwards to it's caller.

--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Ducker]

Won't that fire every time the job has an error, rather than just when the job entirely gives up?

To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeline-api+unsub...@googlegroups.com.

[Tom Kaitchuck]

It depends on where you put the handleException method. If Job A invokes B with 3 reattempts and defines a handleException method in A's class, then it will not be invoked until B's retries have been exhausted. 

To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeli...@googlegroups.com.

[Andrew Ducker]

Aaaah.  Thank you - that makes perfect sense, but I misunderstood that bit of the docs!

On Monday, 24 March 2014 22:01:56 UTC, Tom Kaitchuck wrote:

It depends on where you put the handleException method. If Job A invokes B with 3 reattempts and defines a handleException method in A's class, then it will not be invoked until B's retries have been exhausted. 

On Tue, Mar 18, 2014 at 6:18 AM, Andrew Ducker <and...@ducker.org.uk> wrote:

Won't that fire every time the job has an error, rather than just when the job entirely gives up?

On Monday, 17 March 2014 21:11:09 UTC, Tom Kaitchuck wrote:

You can define your own handleException() method on your worker where you can have custom logic.

https://code.google.com/p/appengine-pipeline/wiki/ErrorHandling
This method can be overloaded based on exception type. If it is the most specific type that matches the thrown exception will be called. 

If there is not a handleException method for a given job it will propagate upwards to it's caller.

On Sat, Mar 15, 2014 at 2:51 PM, Andrew Ducker <and...@ducker.org.uk> wrote:

Let's say I have a job that reads in input from a URL.  Clearly this might go wrong if the server is down, or overloaded.

Normally, I want that job to try a few times - something that's easy enough to do with a JobSetting.  I can tell it to try again an hour later, and see if the server is back up again.

And I don't need to bother the user who set up this job with the fact that it's failed - not if it's going to succeed later.

However, if the job _totally_ fails then I probably want to let the user know "By the way, that process you wanted run, it failed, and here's why."

Is there any way for the Pipeline API to handle that?

Thanks,

Andy

--
You received this message because you are subscribed to the Google Groups "Google App Engine Pipeline API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to app-engine-pipeline-api+unsubscr...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.