Hi,
I'm new to scala-graph and scala. I used scala to write below simple Spark codes, and tried to build a graph using scala-graph library.
package org.apache.spark.examples
import org.apache.spark.SparkContext._
import org.apache.spark.{SparkConf, SparkContext}
import scalax.collection.Graph
import scalax.collection.GraphEdge.DiEdge
object Test{
def main(args: Array[String]) {
val sparkConf = new SparkConf().setAppName("Test")
val ctx = new SparkContext(sparkConf)
val g = Graph[String, DiEdge]("string")
ctx.stop()
}
}
when running the code on Spark platform (the compiling is ok), I met below errors.
$ spark-submit --master spark://localhost.localdomain:7077 --class org.apache.spark.examples.Test ~/../test/bin/Test.jar
Exception in thread "main" java.lang.NoClassDefFoundError: scalax/collection/generic/GraphCoreCompanion
at org.apache.spark.examples.RouteChecker.main(RouteChecker.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: scalax.collection.generic.GraphCoreCompanion
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 8 more
But when I commented out the graph initialization code "val g = Graph[String, DiEdge]("string") " , the compiling and running are both OK.
Does anyone know how to solve this problem? thank you very much.
Best Regards,
Dingfei