MapReduce: any way to get the value of counters easily?

[mac]

Hi,

I tried to use MapReduce (http://code.google.com/p/appengine-
mapreduce/) to do some jobs in my project.

I am wondering how get the value of the counters?

It seems that it is stored in the counters_map field in MapreduceState
and it's in JSON format.

Is it the correct way to grab the value by read and parse it from
MapreduceState directly?

[Ikai L (Google)]

Here's how you'd get the counter value (excuse the comments, this is copied and pasted from a blog post):

MapReduceState mrState = MapReduceState.getMapReduceStateFromJobID(
					datastore, jobId);

			// There's a bit of ceremony to get the actual counter. This
			// example is intentionally verbose for clarity. First get all the
			// Counters,
			// then we get the CounterGroup, then we get the Counter, then we
			// get the count.
			Counters counters = mrState.getCounters();
			CounterGroup counterGroup = counters.getGroup("CommentWords");
			Counter counter = counterGroup.findCounter("count");
			long wordCount = counter.getValue(); // Finally!

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[mac]

Hi Ikai,

It's quiet useful. But I can't find the definition of MapReduceState
and method getMapReduceStateFromJobID() in the model.py.

I'm using mapreduce version 100 now. I think it's the latest one.

BTW, what was the blog? It seems I can learn something from it. Can
you share it? Thanks!

Regards,
Mac

> > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com>
> > .

[mac]

Oh, I think you're referring to Java version while I'm using Python.
So I can't use these concise APIs.

[Ikai L (Google)]

Yeah, the blog is here:

http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-app-engine/

I would look in the source:

http://code.google.com/p/appengine-mapreduce/source/browse/#svn/trunk/python

Nick Johnson has Python code samples, but I don't think he has an example of a callback handler:

http://blog.notdot.net/2010/05/Exploring-the-new-mapper-API

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[mac]

Hi,

Thanks for mike.aizatsky's update in the svn.

Now I can get the counters by

from mapreduce import model

state = model.MapreduceState.get_by_job_id(mid)
for counter in state.counters_map.counters:
# do something with counter here

On Sep 2, 2:55 am, "Ikai L (Google)" <ika...@google.com> wrote:
> Yeah, the blog is here:
>

> http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-ap...

>
> I would look in the source:
>

> http://code.google.com/p/appengine-mapreduce/source/browse/#svn/trunk...

> > > > > google-appengi...@googlegroups.com<google-appengine%2Bunsubscrib e...@googlegroups.com><google-appengine%2Bunsubscrib