java 8 and scoobi

[Alex Cozzi]

Anybody tried to run a scoobi job with java 8? I tried it running in local mode on my mac and I get the following exception:

Exception in thread "main" com.thoughtworks.xstream.converters.ConversionException: Could not call scala.collection.immutable.$colon$colon.writeObject() : -1

---- Debugging information ----

message             : Could not call scala.collection.immutable.$colon$colon.writeObject()

cause-exception     : java.lang.ArrayIndexOutOfBoundsException

cause-message       : -1

-------------------------------

at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callWriteObject(SerializationMethodInvoker.java:140)

at com.thoughtworks.xstream.converters.reflection.SerializableConverter.doMarshal(SerializableConverter.java:232)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:72)

at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)

at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)

at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:229)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:208)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:171)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:116)

at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:72)

at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)

at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)

at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)

at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)

at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)

at com.thoughtworks.xstream.XStream.marshal(XStream.java:895)

at com.thoughtworks.xstream.XStream.marshal(XStream.java:884)

at com.thoughtworks.xstream.XStream.toXML(XStream.java:872)

at com.nicta.scoobi.impl.util.Serialiser$class.serialise(Serialiser.scala:46)

at com.nicta.scoobi.impl.util.Serialiser$.serialise(Serialiser.scala:63)

at com.nicta.scoobi.impl.util.DistCache$.serialise(DistCache.scala:71)

at com.nicta.scoobi.impl.util.DistCache$.pushObject(DistCache.scala:53)

at com.nicta.scoobi.impl.exec.MapReduceJob.configureMappers(MapReduceJob.scala:140)

at com.nicta.scoobi.impl.exec.MapReduceJob.configure(MapReduceJob.scala:91)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$Execution$$configureMscr$1.apply(HadoopMode.scala:162)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$Execution$$configureMscr$1.apply(HadoopMode.scala:154)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$5.apply(HadoopMode.scala:141)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution$$anonfun$5.apply(HadoopMode.scala:140)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)

at scala.collection.immutable.List.foreach(List.scala:318)

at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)

at scala.collection.AbstractTraversable.map(Traversable.scala:105)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution.runMscrs(HadoopMode.scala:140)

at com.nicta.scoobi.impl.exec.HadoopMode$Execution.execute(HadoopMode.scala:125)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$$executeLayer$1.apply(HadoopMode.scala:115)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$$executeLayer$1.apply(HadoopMode.scala:114)

at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:62)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$$executeLayers$1$1.apply(HadoopMode.scala:73)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$com$nicta$scoobi$impl$exec$HadoopMode$$executeLayers$1$1.apply(HadoopMode.scala:73)

at scala.collection.immutable.List.foreach(List.scala:318)

at com.nicta.scoobi.impl.exec.HadoopMode.com$nicta$scoobi$impl$exec$HadoopMode$$executeLayers$1(HadoopMode.scala:73)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:97)

at com.nicta.scoobi.impl.exec.HadoopMode$$anonfun$executeNode$1.apply(HadoopMode.scala:90)

at org.kiama.attribution.AttributionCore$CachedAttribute.apply(AttributionCore.scala:62)

at scalaz.syntax.IdOps$class.$bar$greater(IdOps.scala:15)

at scalaz.syntax.ToIdOps$$anon$1.$bar$greater(IdOps.scala:82)

at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:52)

at com.nicta.scoobi.impl.exec.HadoopMode.execute(HadoopMode.scala:48)

at com.nicta.scoobi.impl.Persister.persist(Persister.scala:44)

at com.nicta.scoobi.impl.ScoobiConfigurationImpl.persist(ScoobiConfigurationImpl.scala:377)

at com.nicta.scoobi.application.Persist$class.persist(Persist.scala:33)

[Eric Torreborre]

No sorry Alex, we haven't tried yet to run things on Java 8.

Is it possible that you are running your client code and server code in different jvms and then the objects would not be serialised in the same way?

(Actually I think I remember a question on the scala mailing-list about List serialisation being broken with the latest 2.11.0-RC version, could it be something like that?)

Eric.

[Alex Cozzi]

no, I get the error by running the code from my IDE, (local mode), so there is no server involved. It is not urgent, since I can still run on jdk 1.6, but I though you might want to know...

[Alex Cozzi]

btw: I am still using scala 2.10. I just tried to run using the jdk 1.8.

[Normen Müller]

Hi, I'm also facing the same problem on Mac OS X 10.10, Java 8 with the following project spec:

name := """foo"""

version := "1.0"

scalaVersion := "2.10.3"

libraryDependencies ++= Seq(
  "com.nicta" % "scoobi_2.10" % "0.8.5")

resolvers ++= Seq(
  Resolver.sonatypeRepo("releases")
, Resolver.sonatypeRepo("snaspshots"))

What is your solution, Alex?

Cheers, /nm

[Alex Cozzi]

running on jdk 1.7 :-(

[Wen Liu]

I got the same issue, 

OS: mac yosemite, 

java 1.8

scoobi 0.9.1,

After changing back to java 1.6, everything works fine. 

[Alex Kuchaiev]

Anybody was able to make it work on Java 8?