Regarding Spark ARIMA for Java

[Devanshi Desai]

Hello sir,

I am currently working as Researcher and Developer and currently working with spark(new to spark), I want to implement time series models for that i use spark-ts examples and i am comfortable with java only so i tried to convert SingleSeriesARIMA.scala to .java but facing cast exception: ' org.apache.spark.api.java.JavaRDD cannot cast to org.apache.spark.mllib.linalg.vector'
When i  am using
ARIMAModel arimamodel = ARIMA.fitmodel(1,0,1,(vector) ts, false, null, double[])

Where ts is defined JavaRDD<vector> and map with JavaRDD<string,vector> function.

Please give me positive response. Awaiting for response.

Thanks and Regards,
Devanshi Desai

[khatta...@gmail.com]

Hi there,

I used the following to build & forecast using ARIMA. In my case, the time series consists of a csv file comprising a single column. Hope this helps.

..............................

 Dataset<String> lines = session.read().textFile(args[0]);
 
 Dataset<Double> doubleDataset = lines.map( line ->  Double.parseDouble(line), Encoders.DOUBLE());
 
 List<Double> doubleList = doubleDataset.collectAsList();
 
 Double[] doubleArray = new Double[doubleList.size()];
 doubleArray = doubleList.toArray(doubleArray);
 
 double[] values = new double[doubleArray.length];
 for (int i = 0; i < doubleArray.length; i++){
 values[i] = doubleArray[i];
 }
 
 Vector tsVector = Vectors.dense(values);
 System.out.println("TS vector:" + tsVector.toString());
 
 ARIMAModel arimaModel = ARIMA.autoFit(tsVector, 1, 0, 1);
 
 System.out.println("***ARIMA Model Coefficients ***");
 for (double coefficient : arimaModel.coefficients()){
 System.out.println("ARIMA model coefficient:" + coefficient);

 }
 
 Vector forecast = arimaModel.forecast(tsVector, 10);
 System.out.println("ARIMA model forecast for next 10 observations:" + forecast);

..............................................

WK

[mat...@armorway.com]

Hello,

I am using very similar Java code to replicate the SingleSeriesARIMA example. However, whenever I try to create the ARIMAModel (either using autoFit() or fitModel()) I get the following exception:

Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.ObjectRef.create(Ljava/lang/Object;)Lscala/runtime/ObjectRef;
    at com.cloudera.sparkts.UnivariateTimeSeries$.differencesOfOrderD(UnivariateTimeSeries.scala:471)
    at com.cloudera.sparkts.models.ARIMA$$anonfun$2.apply$mcZI$sp(ARIMA.scala:286)
    at com.cloudera.sparkts.models.ARIMA$$anonfun$2.apply(ARIMA.scala:285)
    at com.cloudera.sparkts.models.ARIMA$$anonfun$2.apply(ARIMA.scala:285)
    at scala.collection.Iterator$class.find(Iterator.scala:780)
    at scala.collection.AbstractIterator.find(Iterator.scala:1157)
    at scala.collection.IterableLike$class.find(IterableLike.scala:79)
    at scala.collection.AbstractIterable.find(Iterable.scala:54)
    at com.cloudera.sparkts.models.ARIMA$.autoFit(ARIMA.scala:285)
    at com.cloudera.sparkts.models.ARIMA.autoFit(ARIMA.scala)

Have you ever experienced this issue or know what might be causing it? If it is relevant I am using Java 1.8, ts-spark 0.4.1, spark 2.10, and scala 2.10.

Thanks in advance,
Matthew

[Devanshi Desai]

Hello,

I think here the problem that fitModel and autoFit is not showing as part of ARIMA Model.

In my case I used the same code which share by khattak. I found the proper result with using both fitModel as well as autoFit.

One more thing I used Spark 2.0.1 with scala 2.11.8 and java 1.8. So, please check version also.

Thanks and Regards,
Devanshi Desai

[relas...@gmail.com]

I have used sparkts in java  ,while running the code , have been getting below error.

"requirement failed: Lengths must match!: a.length == b.length ".

could you please solve this problem ASAP.

Code:

ARIMAModel arimamodel = ARIMA.autoFit(tsvector, 1, 1, 1);

System.out.println(arimamodel);

Vector forcst = arimamodel.forecast(tsvector, 30);<<<<<<<here i am getting error

System.out.println("forecast of next 30 observations: " + forcst);

[disha.d...@gmail.com]

Hello,

Which values pass to tsvector? Because in forecast only pass two variables vector which you want to forecast with n- period ahead of time. So I think this problem with tsvector values.

The code you written is proper for arimamodel forecast. But my suggestion is please check model build properly when you pass tsvector to autoFit()? 

Thanks and Regards,

Devanshi