Tools for debugging/profiling Map reduce job

[Shekhar]

Hi,
Is there any tools available for debugging map reduce jobs.

I know some of the ways of debugging like:

1. As suggested by you like using rhex interatively

2. using tryCatch block

For profiling we need to use system.time to get the time consumed by
different segments of code, and outputting via rhcollect and rhcounter


Regards,
Som

[Saptarshi Guha]

Hello,

Unfortunately, I havent had the time to think about a framework for debugging.
HAdoop has something related to debug scripts that can be executed on
the offending 'split'.
I could (in future) look towards using that. It might be of help.
but yes, currently, debugging can be, to say the least, adventurous.

My priority now is to add a custom multiple file output format - i.e.
based on the key value write to different
output directories.

Regards
J