App Engine Java Map Reduce Completion Callback

[Jim]

Hello everyone,

I've just started working with Java Map Reduce on App Engine, and I want to take advantage of the Completion Callback function so I can write a servlet that processes the final output of my Map Reduce job.  I'd like to drop the output data into a task and then persist it into my datastore.  Ikai Lan's tutorial has been very helpful to me as I work my way through this, however it was written 4 1/2 years ago and the Map Reduce library has progressed a lot since then. 

In particular, Ikai's page http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-app-engine/ shows how to define a URI for a callback after completion of your Map Reduce job.  He shows how to define a job in mapreduce.xml which includes a setting for mapreduce.appengine.donecallback.url.   However, with the version I'm using, configuration of jobs is done in code using the MapReduceSpecification builder.  I can't find any reference there or in any of the other configuration objects for input, mapper, reducer, output, etc. to set a URI callback like Ikai describes when using the old mapreduce.xml config file.

I have a feeling I'm overlooking something simple.  Can somebody point me in the right direction?

If the callback method is no longer available, I'm thinking a custom output method would be the right place to capture my final output and drop to the task queue.  Any other ideas?

Thanks for your help,

Jim

[Tom Kaitchuck]

You can use pipelines: https://github.com/GoogleCloudPlatform/appengine-pipelines
to chain things to run after a MR job. One of the included examples does this:

https://github.com/GoogleCloudPlatform/appengine-mapreduce/blob/master/java/example/src/com/google/appengine/demos/mapreduce/entitycount/ChainedMapReduceJob.java

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/cf47c5eb-f8ad-4376-90c7-86e44549f3d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jim]

Looks like that is exactly what I need.

Thank you.