Way to access counter value in driver?

[Jordan Boyd-Graber]

Hi all,

I'd like to use a counter to assess convergence of a MapReduce
algorithm. It's clear how I can increment counters inside of a mapper
or reducer, but how can I get the counter value in the driver (e.g.
before I add another map/reduce iteration)?

Thanks!

-Jordan

[Jordan Boyd-Graber]

Sorry to be a bother, but does anyone have ideas on this?

Thanks,

Jordan

> --
> You received this message because you are subscribed to the Google Groups "dumbo-user" group.
> To post to this group, send email to dumbo...@googlegroups.com.
> To unsubscribe from this group, send email to dumbo-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/dumbo-user?hl=en.
>
>

--
--------------------
Jordan Boyd-Graber
South Hornbake 2118C
University of Maryland School of Information Studies
College Park, MD 20742

Voice: 920.524.9464
j...@umiacs.umd.edu
http://umiacs.umd.edu/~jbg
--------------------

"In theory, there is no difference between theory and practice. But,
in practice, there is."
    - Jan L.A. van de Snepscheut

[Klaas Bosteels]

You can get counter values via the command

$ hadoop job -counter <job-id> <group-name> <counter-name>

which you could also executer from python, but doing it adDumbo runner will be tricky since a runner not only gets executed on the machine from which you start the job but also on the cluster nodes. Also, getting the job ID might be a bit tricky I guess.

Please do let us know if you find a way of doing it though!

-K

[francois kawala]

Hi all,

It's been a year since the last reply, and I'm facing the same need to access counters values, is there any novelty that I should know before trying to work this out ??

Thanks =)

François.