HTTP 429 Returned for /mapreduce/workerCallback task

[William Speirs]

I'm attempting to run a map-reduce job from Java. The job kicked off and started without a hitch, but now it's stuck waiting on ~34 tasks to finish. The job is stuck in the "ExamineStatusAndReturnResult" phase, after the ShardedJob phase. I see ~34 tasks in my queue all of the form: POST /mapreduce/workerCallback/map-hex-numbers They all say that the previous run returned a 429 Too Many Requests (http://tools.ietf.org/html/rfc6585#section-4). I'm guessing I've hit some kind of limit/quota, but I cannot tell what/where this quota is.

How can I find out what is causing the 429 response code?

Thanks!

[Tom Kaitchuck]

The 429 is the mapreduce framework pushing back to the task queue because there is not enough memory on the instance to handle the number of requests it was given. This isn't necessarily a problem, and if there are no other issues, can be safely ignored. However if it is occurring a lot, you could improve performance by lowering "max-concurrent-requests" in your modules configuration:

https://developers.google.com/appengine/docs/java/modules/#Java_Configuration

You should be able to see the progress of the shards in the lower right panel of the UI. Provided all of the shards are making progress, you probably have nothing to worry about. 

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.