Scalding tutorial: java.lang.ClassNotFoundException

[Darq Moth]

Please help to run Scalding tutorial.

I have Hadoop 2.2 running on a single node and trying to run Scalding tutorial:
https://github.com/Cascading/scalding-tutorial/
After successfuly buiding 'fat jar' with these commands:

$ git clone git://github.com/Cascading/scalding-tutorial.git
$ cd scalding-tutorial
$ sbt assembly

I try to run tutorial examples as suggested with this command:

$ yarn jar target/scalding-tutorial-0.8.11.jar <TutorialPart> --local <addtional arguments>

Both  --local and --hdfs fail with java.lang.ClassNotFoundException:

$ yarn jar target/scala-2.9.3/scalding-assembly-0.10.0.jar 1 --local
Exception in thread "main" java.lang.ClassNotFoundException: 1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:205)

$ yarn jar target/scala-2.9.3/scalding-assembly-0.10.0.jar 1 --hdfs
Exception in thread "main" java.lang.ClassNotFoundException: 1
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:205)

[Darq Moth]

Unfortunately changing command argument to 'Tutorial1', 'Tutorial0' does not help either:

$ yarn jar target/scala-2.9.3/scalding-assembly-0.10.0.jar Tutorial1 --local
Exception in thread "main" java.lang.ClassNotFoundException: Tutorial1

 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:247)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:205)

$ yarn jar target/scala-2.9.3/scalding-assembly-0.10.0.jar Tutorial0 --local
Exception in thread "main" java.lang.ClassNotFoundException: Tutorial0

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:205)

[Andre Kelpe]

I just tried it again and it works for me:

$ yarn jar target/scalding-tutorial-0.8.11.jar Tutorial0 --local
14/07/02 09:36:21 INFO property.AppProps: using app.id: 5EFCC4FC4BD94B21BE309BB04C14F267
14/07/02 09:36:21 INFO management.CascadingServices: loading properties: cascading-service.properties
14/07/02 09:36:21 INFO util.Version: Concurrent, Inc - Cascading 2.5.2
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0] starting
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0]  source: FileTap["TextLine[['offset', 'line']->[ALL]]"]["data/hello.txt"]
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0]  sink: FileTap["TextLine[['offset', 'line']->[ALL]]"]["target/data/output0.txt"]
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0]  parallel execution is enabled: true
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0]  starting jobs: 1
14/07/02 09:36:21 INFO flow.Flow: [Tutorial0]  allocating threads: 1
14/07/02 09:36:21 INFO flow.FlowStep: [Tutorial0] starting step: local

What platform are you on and which java version are you using?

- André

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/84802343-9aa3-4747-b1c9-770ebccbf4d9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

[Darq Moth]

CentOS 6.5 (Final)

$ java -version
java version "1.7.0_51"
Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

$ hadoop version
Hadoop 2.2.0.2.0.6.0-101
Subversion g...@github.com:hortonworks/hadoop.git -r b07b2906c36defd389c8b5bd22bebc1bead8115b
Compiled by jenkins on 2014-01-09T05:18Z
Compiled with protoc 2.5.0
From source with checksum 704f1e463ebc4fb89353011407e965
This command was run using /usr/lib/hadoop/hadoop-common-2.2.0.2.0.6.0-101.jar

--
You received this message because you are subscribed to a topic in the Google Groups "cascading-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cascading-user/IhaeW7pZvj0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cascading-use...@googlegroups.com.

To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/CAA2tiY%2BiR3Oex6vy7e6pqb2%2BcY0%3DmvrFGAy-wnzhThgSJnr_Jg%40mail.gmail.com.

[Darq Moth]

In my case `sbt assembly`(after `sbt update clean`) results in a single `scalding/target/scala-2.9.3/scalding-assembly-0.10.0.jar` file built. Why it does not build `target/scalding-tutorial-0.8.11.jar`? How to build it?

To unsubscribe from this group and stop receiving emails from it, send an email to cascading-user+unsubscribe@googlegroups.com.

To post to this group, send email to cascading-user@googlegroups.com.

Visit this group at http://groups.google.com/group/cascading-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/84802343-9aa3-4747-b1c9-770ebccbf4d9%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

--
You received this message because you are subscribed to a topic in the Google Groups "cascading-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cascading-user/IhaeW7pZvj0/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cascading-user+unsubscribe@googlegroups.com.
To post to this group, send email to cascading-user@googlegroups.com.

Visit this group at http://groups.google.com/group/cascading-user.

[Darq Moth]

Solved! When one follows tutorial instructions at https://github.com/Cascading/scalding-tutorial/ and does 'git clone git://github.com/Cascading/scalding-tutorial.git` instead of tutorial one gets a complete `Scalding` project. Running then `sbt assembly` creates fat Scalding jar `scalding/target/scala-2.9.3/scalding-assembly-0.10.0.jar` instead of tutorial jar. To buld and run tutorial one needs to download a *zip* file from the same git page. This zip extarcts to `scalding-tutorial-wip-2.5`. After `sbt assembly` in this dir,`yarn jar target/scalding-tutorial-0.8.11.jar Tutorial1` works.