PermanentTaskFailure with deferred API in appengine standard python (maybe datastore related)

[PK]

(This question is also posted as https://stackoverflow.com/questions/58191958 please answer there if you prefer)

I recently have started seeing issues with the deferred API. No change on my app that has been running for years.

If I read what is happening correctly is that when the deferred API has a big payload it stores it in the datastore. Then when the task runs, it tries to retrieve but does not find it and raises PermanentTaskFailure

Anybody, has run into the same issue, any clues what has changed in appengine recently?

The pattern is:

1. Web request comes in

2. web requests starts a deferred task with a an array of N items

3. task executes and performs work for 1 item then repeats step 2 for the remaining N-1 items (util there are no items)

some pointers and logs:

https://cloud.google.com/appengine/docs/standard/python/refdocs/modules/google/appengine/ext/deferred/deferred

Thanks!!!

[George (Cloud Platform Support)]

You have taken the right approach, publishing on stackoverflow. If you think this a Cloud Platform issue, and not on your app's side, you are encouraged to post in Public Issue Tracker. 

This discussion group is oriented more towards general opinions, trends, and issues of general nature touching App Engine and Cloud SQL. For coding and programming architecture, as well as Deferred API specific problems, you may be better served in dedicated forums such as stackoverflow, where experienced programmers are within reach and ready to help.