Spark Cassandra connector - 1.2 - java.lang.ClassNotFoundException: com.twitter.jsr166e.LongAdder

[Ajay Kumar]

Hi Team,

In our one of the old deployment we are using Datastax spark cassandra connector version 1.2.

Following are the versions of our components

1. Cassandra - 2.0.4
2. Spark - 1.2
3. Spark Cassandra connector - 1.2

During our write from spark to cassandra we are getting java.lang.ClassNotFoundException: com.twitter.jsr166e.LongAdder exception. When we checked the Connector jar we could see the JSR166e is available. This jar is coming with connector pom. But in run time it is not able to find out the jar.

Code Snippet :

1.    CassandraJavaUtil.javaFunctions(myJavaRdd).writerBuilder( keyspace                            COLUMN_FAMILY, CassandraJavaUtil.mapToRow(Class, myMap))
2.                     .withPerRowTTL(Constants.TTL).saveToCassandra();

Exception:

org.apache.spark.SparkException: Job aborted due to stage failure: Task 8 in stage 3.0 failed 4 times, most recent failure: Lost task 8.3 in stage 3.0 (TID 118, vmc0251): java.lang.NoClassDefFoundError: com/twitter/jsr166e/LongAdder at com.datastax.spark.connector.metrics.OutputMetricsUpdater$TaskMetricsSupport$class.$init$(OutputMetricsUpdater.scala:107) at com.datastax.spark.connector.metrics.OutputMetricsUpdater$TaskMetricsUpdater.<init>(OutputMetricsUpdater.scala:146) at com.datastax.spark.connector.metrics.OutputMetricsUpdater$.apply(OutputMetricsUpdater.scala:76) at com.datastax.spark.connector.writer.TableWriter.write(TableWriter.scala:119) at com.datastax.spark.connector.RDDFunctions$$anonfun$saveToCassandra$1.apply(RDDFunctions.scala:36) at com.datastax.spark.connector.RDDFunctions$$anonfun$saveToCassandra$1.apply(RDDFunctions.scala:36) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61) at org.apache.spark.scheduler.Task.run(Task.scala:56) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: com.twitter.jsr166e.LongAdder at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 12 more Driver stacktrace:

Please help us on that issue.

Regards,

Ajay