Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
at com.aggregation.job.DataAggregation$.<init>(DataAggregation.scala:30)
at com.aggregation.job.DataAggregation$.<clinit>(DataAggregation.scala)
at com.aggregation.job.DataAggregation.main(DataAggregation.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
I made sure to do an sbt clean and sbt assembly to update the scala version in both projects. Has anyone run into this with jobs on EMR? Does the version of Hadoop have anything to do with this?
Thanks in advance!
Lakshmi
--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/a3ba173a-72ba-41a6-adfd-64f6e4960b62%40googlegroups.com.
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.1/scala-library-2.11.1.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-library;2.11.1!scala-library.jar (1323ms)
[info] downloading https://repo1.maven.org/maven2/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar ...
[info] [SUCCESSFUL ] org.mockito#mockito-all;1.8.5!mockito-all.jar (520ms)
[info] downloading https://repo1.maven.org/maven2/org/scalacheck/scalacheck_2.11/1.12.2/scalacheck_2.11-1.12.2.jar ...
[info] [SUCCESSFUL ] org.scalacheck#scalacheck_2.11;1.12.2!scalacheck_2.11.jar (456ms)
[info] downloading https://repo1.maven.org/maven2/org/scalatest/scalatest_2.11/2.2.4/scalatest_2.11-2.2.4.jar ...
[info] [SUCCESSFUL ] org.scalatest#scalatest_2.11;2.2.4!scalatest_2.11.jar(bundle) (1214ms)
[info] downloading https://repo1.maven.org/maven2/org/slf4j/slf4j-log4j12/1.6.6/slf4j-log4j12-1.6.6.jar ...
[info] [SUCCESSFUL ] org.slf4j#slf4j-log4j12;1.6.6!slf4j-log4j12.jar (354ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.5/scala-library-2.11.5.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-library;2.11.5!scala-library.jar (997ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/modules/scala-parser-combinators_2.11/1.0.2/scala-parser-combinators_2.11-1.0.2.jar ...
[info] [SUCCESSFUL ] org.scala-lang.modules#scala-parser-combinators_2.11;1.0.2!scala-parser-combinators_2.11.jar(bundle) (398ms)
[info] downloading https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.2/scala-reflect-2.11.2.jar ...
[info] [SUCCESSFUL ] org.scala-lang#scala-reflect;2.11.2!scala-reflect.jar (854ms)
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/2f4edf32-7af9-4ee4-978c-155893f23ea9%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "cascading-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cascading-user/MI7EPfb0_Kw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/CANX%3DQ2oUj8TPfiRRKAqYcUtWzBBtixr6SA0RHGbMpc0o0UReBw%40mail.gmail.com.
Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
at com.aggregation.job.DataAggregation$.<init>(DataAggregation.scala:30)
val hadoopVersion = "1.2.1"
val scaldingVersion = "0.15.0"
libraryDependencies ++= Seq(
"com.twitter" %% "scalding-core" % scaldingVersion,
"com.twitter" %% "scalding-json" % scaldingVersion,
"com.twitter" %% "scalding-jdbc" % scaldingVersion,
"com.github.nscala-time" %% "nscala-time" % "2.0.0",
// include Hadoop runtime to run locally in "local" mode
//"org.apache.hadoop" % "hadoop-core" % hadoopVersion
// to run on hadoop in "hdfs" mode, replace above with the following to exclude Hadoop from assembly jar
"org.apache.hadoop" % "hadoop-core" % hadoopVersion % "provided"
)
resolvers ++= Seq(
"Conjars repo" at "http://conjars.org/repo"
)
organization := "com.abc"
name := "aggregator"
scalaVersion := "2.11.2"
ivyScala := ivyScala.value map {
_.copy(overrideScalaVersion = true)
}
javacOptions ++= Seq("-source", "1.6", "-target", "1.7") // not sure if i need this, but my Amazon EMR AMI is 3.8.0 and it uses Java 7.
import AssemblyKeys._
assemblySettings
mergeStrategy in assembly := Merge.mergeStrategy
sbt.version=0.13.1
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.0")
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/91ba0979-d3f6-47b1-a1a5-86a326003e89%40googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "cascading-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cascading-user/MI7EPfb0_Kw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/ce9ce233-ba5a-4db2-b7be-9a72cfb3cf44%40googlegroups.com.
$ sudo find / -name "scala-library-2.10.*.jar" -exec rm -rf {} \;
[ec2-user@ip-172-31-72-130 ~]$ sudo find / -name "scala-library-2.11.*.jar"
/home/hadoop/.versions/hbase-0.94.18/lib/scala-library-2.11.0.jar
/usr/share/doc/scala/api/jars/scala-library-2.11.1-javadoc.jar
[ec2-user@ip-172-31-72-130 ~]$ sudo find / -name "scala-library-2.10.*.jar"
/usr/share/aws/emr/emrfs/lib/scala-library-2.10.5.jar