NoClassDefFoundError: com/cloudera/sparkts/models/ARIMA

[Abdullahi Abubakar]

To whom it may concern,

I am still finding it difficult to run the sample experiment. I have downloaded/installed the maven and configured the pom.xml file as directed here.

I was able to run mvn package (this is simlar to sbt package) for building the project, and obtained the following jar files.

1. abafo_sparkts-0.0.1-SNAPSHOT.jar
2. abafo_sparkts-0.0.1-SNAPSHOT-jar-with-dependencies.jar
   

ISSUES

Issue No1: When ever i submit with the first jar file I get the following error. (Note the two lines in bold)

Exception in thread "main" java.lang.NoClassDefFoundError: com/cloudera/sparkts/models/ARIMA$
        at com.cloudera.sparkts.models.SingleSeriesARIMA$.main(SingleSeriesARIMA.scala:74)
        at com.cloudera.sparkts.models.SingleSeriesARIMA.main(SingleSeriesARIMA.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.cloudera.sparkts.models.ARIMA$
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 12 more

Note that my spark submit is:

/bin/spark-submit --class "com.cloudera.tsexamples.SingleSeriesARIMA" --master local[*] target/abafo_sparkts-0.0.1-SNAPSHOT-jar-with-dependencies.jar

Issue No1:  Whenever I submit with the second jar file I get the following error

Exception in thread "main" java.lang.NoSuchMethodError: breeze.linalg.DenseVector$.canSetD()Lbreeze/generic/UFunc$InPlaceImpl2;
        at com.cloudera.sparkts.models.ARIMAModel.gradientlogLikelihoodCSSARMA(ARIMA.scala:506)
        at com.cloudera.sparkts.models.ARIMA$$anon$3.value(ARIMA.scala:189)
        at org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.computeObjectiveGradient(GradientMultivariateOptimizer.java:53)
        at org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:254)
        at org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.doOptimize(NonLinearConjugateGradientOptimizer.java:46)
        at org.apache.commons.math3.optim.BaseOptimizer.optimize(BaseOptimizer.java:153)
        at org.apache.commons.math3.optim.BaseMultivariateOptimizer.optimize(BaseMultivariateOptimizer.java:65)
        at org.apache.commons.math3.optim.nonlinear.scalar.MultivariateOptimizer.optimize(MultivariateOptimizer.java:63)
        at org.apache.commons.math3.optim.nonlinear.scalar.GradientMultivariateOptimizer.optimize(GradientMultivariateOptimizer.java:73)
        at org.apache.commons.math3.optim.nonlinear.scalar.gradient.NonLinearConjugateGradientOptimizer.optimize(NonLinearConjugateGradientOptimizer.java:244)
        at com.cloudera.sparkts.models.ARIMA$.fitWithCSSCGD(ARIMA.scala:196)
        at com.cloudera.sparkts.models.ARIMA$.fitModel(ARIMA.scala:105)
        at com.cloudera.tsexamples.SingleSeriesARIMA$.main(SingleSeriesARIMA.scala:43)
        at com.cloudera.tsexamples.SingleSeriesARIMA.main(SingleSeriesARIMA.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

My spark submit is:

./bin/spark-submit --class "com.cloudera.tsexamples.SingleSeriesARIMA" --master local[*] target/abafo_sparkts-0.0.1-SNAPSHOT-jar-with-dependencies.jar

Could you please help me resolve this issue? Thank you.

Regards,

Abdullahi

---