Iam running a Storm topology , getting java.lang.NoClassDefFoundError: scala/reflect/ClassManifest

480 views
Skip to first unread message

Madhava reddy

unread,
Aug 28, 2014, 9:16:15 AM8/28/14
to scala...@googlegroups.com
please let me know what changes do i need to do in pom.xml:

Is this a known issue with these versions? Do I need other dependencies? 
Here are my depedencies included in pom.xml file.

<dependency>
           <groupId>org.scala-lang</groupId>
           <artifactId>scala-library</artifactId>
           <version>2.10.3</version>
</dependency>
<dependency>
           <groupId>org.scala-lang</groupId>
           <artifactId>scala-reflect</artifactId>
           <version>2.11.0</version>
</dependency>




java.lang.NoClassDefFoundError: Could not initialize class kafka.utils.Log4jController$
           at kafka.utils.Logging$class.$init$(Logging.scala:29)
           at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:29)
           at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:34)
           at storm.kafka.DynamicPartitionConnections.register(DynamicPartitionConnections.java:60)
           at storm.kafka.PartitionManager.(PartitionManager.java:63)
           at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98)
           at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69)
           at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135)
           at backtype.storm.daemon.executor$fn__4261$fn__4276$fn__4305.invoke(executor.clj:562)
           at backtype.storm.util$async_loop$fn__442.invoke(util.clj:436)
           at clojure.lang.AFn.run(AFn.java:24)
           at java.lang.Thread.run(Thread.java:744)
           
           
           java.lang.NoClassDefFoundError: scala/reflect/ClassManifest
           at kafka.utils.Log4jController$.(Log4jController.scala:29)
           at kafka.utils.Log4jController$.(Log4jController.scala)
           at kafka.utils.Logging$class.$init$(Logging.scala:29)
           at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:29)
           at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:34)
           at storm.kafka.DynamicPartitionConnections.register(DynamicPartitionConnections.java:60)
           at storm.kafka.PartitionManager.(PartitionManager.java:63)
           at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98)
           at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69)
           at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135)
           at backtype.storm.daemon.executor$fn__4261$fn__4276$fn__4305.invoke(executor.clj:562)
           at backtype.storm.util$async_loop$fn__442.invoke(util.clj:436)
           at clojure.lang.AFn.run(AFn.java:24)
           at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassManifest
           at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
           at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
           at java.security.AccessController.doPrivileged(Native Method)
           at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
           at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
           at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
           ... 14 more

Antoine Gourlay

unread,
Aug 28, 2014, 9:27:52 AM8/28/14
to Madhava reddy, scala-user

On Thu, Aug 28, 2014 at 3:16 PM, Madhava reddy <madha...@gmail.com> wrote:
<dependency>
           <groupId>org.scala-lang</groupId>
           <artifactId>scala-library</artifactId>
           <version>2.10.3</version>
</dependency>
<dependency>
           <groupId>org.scala-lang</groupId>
           <artifactId>scala-reflect</artifactId>
           <version>2.11.0</version>
</dependency>

The versions of scala-* artifacts should match. If you are using Scala 2.10.3, then the version of scala-reflect should be changed to 2.10.3, or the other way around, but these two *are not binary compatible*.

Cheers
--
Antoine

Antoine Gourlay

unread,
Aug 29, 2014, 4:08:30 AM8/29/14
to Madhava reddy, scala-user
Can you list all your dependencies?

Looking at the pom.xml for storm-kafka, it looks like kafka has to be user-provided (so that you can choose a version compatible with your version of scala). What version of kafka do you depend on?


On Fri, Aug 29, 2014 at 10:01 AM, Madhava reddy <madha...@gmail.com> wrote:
Antoine,

Thanks for the reply ..I have used same versions for both, But still getting same  error.please find the error below.

<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.4</version>
</dependency>
 <dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-reflect</artifactId>
<version>2.10.4</version>
</dependency>


Error:

java.lang.NoClassDefFoundError: scala/reflect/ClassManifest
at kafka.utils.Log4jController$.(Log4jController.scala:29)
at kafka.utils.Log4jController$.(Log4jController.scala)
at kafka.utils.Logging$class.$init$(Logging.scala:29)
at kafka.consumer.SimpleConsumer.(SimpleConsumer.scala:29)
at kafka.javaapi.consumer.SimpleConsumer.(SimpleConsumer.scala:34)
at storm.kafka.DynamicPartitionConnections.register(DynamicPartitionConnections.java:60)
at storm.kafka.PartitionManager.(PartitionManager.java:63)
at storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98)
at storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69)
at storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:135)
at backtype.storm.daemon.executor$fn__4261$fn__4276$fn__4305.invoke(executor.clj:562)
at backtype.storm.util$async_loop$fn__442.invoke(util.clj:436)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassManifest
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 14 more

Here i have used below storm versions.

<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>0.9.2-incubating</version>
<!-- keep storm out of the jar-with-dependencies -->
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>0.9.2-incubating</version>
</dependency>

Anything i'm doing incorrect ..? what are compatable  versions of scala using with storm-kafka.
Please let me know how to resolve this issue.

Madhav

--

Madhav

Failure is success if we learn from it!!!!!

   

Reply all
Reply to author
Forward
0 new messages