Ubuntu 11.10 ERROR: Streaming jar not found

[Piers Harding]

Hi -

I'm getting the "ERROR: Streaming jar not found" problem that (from
looking at previous posts) should go away if I include the -hadoop
option. this works for dumbo ls /user/hduser/gutenberg -hadoop /usr,
but not for dumbo start ipcount.py -hadoop /usr -input /tmp/accesslogs/
* -output ipcounts .

I'm using the Debian package from
http://mirrors.ibiblio.org/apache//hadoop/common/stable/hadoop_1.0.1-1_amd64.deb,
on Ubuntu 11.10 (64 bit), which in itself works nicely, although the
distribution spreads the files around /etc/hadoop /usr/(bin|sbin) and /
usr/share/hadoop .

Any pointers on how to debug this, or where to look for the problem
gratefully received.

Thanks,
Piers Harding.

[Piers Harding]

Looks like it was a case of adding:
os.path.join(hadoop, 'share', 'hadoop', 'contrib', name),

to findjar() in dumbo/util.py

Now onto the next problem with:

12/03/20 17:11:14 WARN streaming.StreamJob: -jobconf option is
deprecated, please use -D instead.
-inputformat : class not found :
org.apache.hadoop.streaming.AutoInputFormat
Streaming Job Failed!

Which sounds like an unpatched streaming jar file, from a quick look
at this forum.

Cheers,
Piers Harding.

On Mar 20, 2:03 pm, Piers Harding <pi...@ompka.net> wrote:
> Hi -
>
> I'm getting the "ERROR: Streaming jar not found" problem that (from
> looking at previous posts) should go away if I include the -hadoop
> option.  this works for dumbo ls /user/hduser/gutenberg -hadoop /usr,
> but not for dumbo start ipcount.py -hadoop /usr -input /tmp/accesslogs/
> * -output ipcounts .
>

> I'm using the Debian package fromhttp://mirrors.ibiblio.org/apache//hadoop/common/stable/hadoop_1.0.1-...,

[Klaas Bosteels]

Hadoop 1.0 indeed doesn't have the necessary patches (think they'll only be included from 1.0.2 onwards). I'd recommend using CDH (which also has debian packages) instead.

-K

--
You received this message because you are subscribed to the Google Groups "dumbo-user" group.
To post to this group, send email to dumbo...@googlegroups.com.
To unsubscribe from this group, send email to dumbo-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/dumbo-user?hl=en.

[Piers Harding]

OK - thanks.