wordcount.jar not working

[Nick Smith]

Hi Everyone,

I am a newbie to the Hadoop ecosystem and trying out cascading.
I seem to be having some trouble with executing wordcount sample file.

I did the following:
1. Set up single node Hadoop (version 0.20.2) in Ubuntu 10.04 running
on VMware
2. Downloaded the cascading src file from downloads location in
cascading.org
3. Performed the following steps successfully
> cd <path to cascading>
> ant retrieve

To build Cascading,

> ant -Dhadoop.home=<path to hadoop> compile

To make all jars:

> ant -Dhadoop.home=<path to hadoop> jar

To run all tests:

> ant -Dhadoop.home=<path to hadoop> test
4. Next, created a wordcount map reduce project in eclipse. Imported
the downloaded wordcount.tar.gz
and made a build file.

5. Next, I did the step : ant -Dcascading.home=../cascading jar. Build
Successful.

6. When I run this:hadoop jar ./build/wordcount.jar data/url+page.
200.txt output local
I get the following error:
hadoop@ubuntu:~/workspace/wordcount$ hadoop jar ./build/wordcount.jar
data/url+page.200.txt output local
Exception in thread "main" java.lang.ClassNotFoundException:
wordcount.Main
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

Please advise.

[Chris K Wensel]

What does
> tar -tzf ./build/wordcount.jar
say

Its recommended you get things working without a cluster first, as that adds degrees more things likely to go wrong and confuse new users.

ckw

> --
> You received this message because you are subscribed to the Google Groups "cascading-user" group.
> To post to this group, send email to cascadi...@googlegroups.com.
> To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.
>

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com

-- Concurrent, Inc. offers mentoring, support for Cascading

[Nick Smith]

hi Chris,

Thanks so much for the response.

This is the output from 

hadoop@ubuntu:/usr/local/hadoop$  tar -tzf ./build/wordcount.jar

tar: ./build/wordcount.jar: Cannot open: No such file or directory

tar: Error is not recoverable: exiting now

tar: Child returned status 2

tar: Exiting with failure status due to previous errors

I dont have a cluster set up. I am running hadoop in a single node right now.

[Chris K Wensel]

sorry, i meant jar, not tar

[Chris K Wensel]

that is, 

> jar -tf ./build/wordcount.jar

[Nick Smith]

the same error message

hadoop@ubuntu:/usr/local/hadoop$  jar -tf ./build/wordcount.jar

java.util.zip.ZipException: error in opening zip file

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.<init>(ZipFile.java:127)

at java.util.zip.ZipFile.<init>(ZipFile.java:88)

at sun.tools.jar.Main.list(Main.java:979)

at sun.tools.jar.Main.run(Main.java:224)

at sun.tools.jar.Main.main(Main.java:1149)

[Chris K Wensel]

then you have a corrupted jar file. 

might clean and rebuild, or check permissions or some such.

chris

[Nick Smith]

I have tried those things since morning.

Right now, I tried deleting the entire downloads and re-doing from step 1.

But I still seem to get the same error.

I have reset the permissions to 777 for everything now.

I am not sure if I am doing all the steps right.

I just noticed that when I do an ant build with the build.xml in wordcount, it says Build successful but I do not see a jar file.

The .classes do come up in the build folder.

[Nick Smith]

I just repeated the entire steps with loganalysis and I am still getting the same error message.

hadoop@ubuntu:/usr/local/hadoop$ hadoop jar ./build/loganalysis.jar data/apache.200.txt output

Exception in thread "main" java.io.IOException: Error opening job jar: ./build/loganalysis.jar

at org.apache.hadoop.util.RunJar.main(RunJar.java:90)

Caused by: java.util.zip.ZipException: error in opening zip file

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.<init>(ZipFile.java:127)

at java.util.jar.JarFile.<init>(JarFile.java:135)

at java.util.jar.JarFile.<init>(JarFile.java:72)

at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

:-((

-Nick

[Chris K Wensel]

i guess you recognize this is your environment, so you should look to make sure you have a modern java impl (java 6) and the latest ant. 

fwiw, our testing servers rebuild the examples and run the tests after every release, so the only issues we cannot test for is a properly configured system

chris

[Nick Smith]

thanks so much for your response.

Ant:

Apache Ant(TM) version 1.8.2 compiled on December 20 2010

Java:

java version "1.6.0_26"

Java(TM) SE Runtime Environment (build 1.6.0_26-b03)

Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing

If its not too much trouble, would it be possible for you validate the steps I am doing. I have described them in my first email.

Thanks again!

- Nick

[Chris K Wensel]

if you are following the README, it should be fine.

fundamentally you are ending up with a corrupted jar file. there is no misstep that creates one of those. you have a jar or you don't. something else is going on wrong.

ckw

[Nick Smith]

thanks for all the ideas!

So like you mentioned, my build was corrupted and once I fixed that, 

I am getting this error.

hadoop@ubuntu:/usr/local/hadoop$ bin/hadoop jar wordcount.jar input output

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 2

at wordcount.Main.main(Unknown Source)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Any ideas??

[Chris K Wensel]

per the README, there are three arguments...

https://github.com/cwensel/cascading.samples/blob/master/wordcount/README.TXT

> hadoop jar ./build/wordcount.jar data/url+page.200.txt output local