Which task queue exceptions raised from Queue.add() make sense to retry?

[AndyD]

[hmm, this question didn't attract any interest on StackOverflow..., thought I'd give it a try here]

I'm looking to add some defensive exception handling/retry logic around my (Java) code that enqueues tasks in App Engine. In reviewing the documentation for potential exceptions raised, I see several candidates that might make sense to catch and handle with a retry, but the documentation is a bit vague, so I thought I'd see what others have found to be worthwhile.

Here are the ones I'm looking at, and their descriptions from the docs:

• InternalFailureException - Internal task queue error.
• TransientFailureException - Intermittent failure.
• QueueFailureException - Unspecified queue failure.
• TransactionalTaskException - Queue operation failure caused by Datastore exception.

The only one for which the docs explicitly suggest that retrying might be helpful is TransientFailureException.

Also, regarding TransactionalTaskException, is this something that would only arise when enlisting a task in an enclosing datastore transaction, or could this arise when enqueuing a standalone task?

[Robert Kluin]

I would also suggest you handle TaskAlreadyExistsException. In some
cases you may want to swallow that error, and in others raise it. In
Python we also have a TombstonedTaskError, I guess that is handled by
TaskAlreadyExistsException in Java.

Actually, in Python we have a great deal more exceptions. I've got a
list of exceptions similar to yours that I retry. These also include
deadline exceeded and timeout errors.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/JV4cR4o2BNYJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

[AndyD]

Thanks, Robert.

FYI, you're right, a little code spelunking in QueueApiHelper.java shows that in Java, the tombstone error becomes a TaskAlreadyExistsException:

case TASK_ALREADY_EXISTS:

        return new TaskAlreadyExistsException("Task name already exists : " + detail);
      case TOMBSTONED_TASK:
        return new TaskAlreadyExistsException("Task name is tombstoned : " + detail);
The same file also shows that QueueFailureException is basically a generic catch-all for anything not handled explicitly:

 default:
        return new QueueFailureException("Unspecified error (" + errorCode + ") : " + detail);

So that seems worth a retry if it pops up.

Regarding TransactionalTaskException, it's used for any Datastore errors that arise.  I'm not sure if those would only arise when the task is part of an enclosing transaction or not, but based on the name, I'm going to guess that's the case.  I got a little lost trying to trace back to the possible sources where that could be raised.

Similarly I wasn't able to track down the source of an InternalFailureException.  If anybody has an additional insight into those, please chime in.

-Andy

[Robert Kluin]

Hey Andy,
In Python, I retry internal errors. I believe I have seen timeouts
masquerading as those.

I'm not sure about the transactional error. I do not think I have
ever seen one.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/60AaEXo1XrQJ.