NoClassDefFoundError: org/apache/hadoop/mapred/JobConf - CDAP 3.2.0 on CDH 5.4 using PageRank Spark
2 935 skatījumi
Pāriet uz pirmo nelasīto ziņojumu
opc...@gmail.com
2015. gada 30. okt. 17:26:0830.10.15
uz CDAP User
Hi all,
I am evaluating CDAP 3.2.0 on CDH 5.4 using the PageRank Spark example. CDAP is installed using the CSD.
On loading the app into CDAP via the UI and running the Spark program, it fails with the following.
Is this example expected to work out of the box with this mix of versions?
Any help appreciated.
Cheers,
/T
2015-10-30 21:02:56,911 - ERROR [Spark - PageRankSpark:c.c.c.i.a.r.s.SparkRuntimeService@203] - Spark program execution failure: SparkContext{id=spark.default.PageRankApp.PageRankSpark, runId=888fe527-7f49-11e5-8c29-fa163e502f52}
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService.run(SparkRuntimeService.java:181) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) [com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService$2$1.run(SparkRuntimeService.java:247) [co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at co.cask.cdap.internal.app.runtime.spark.ScalaSparkFacade.<init>(ScalaSparkFacade.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.setupSparkContext(SparkProgramWrapper.java:76) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.run(SparkProgramWrapper.java:58) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.main(SparkProgramWrapper.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-assembly.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter.submit(AbstractSparkSubmitter.java:136) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter$5.run(AbstractSparkSubmitter.java:99) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_67]
... 1 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_67]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67]
... 20 common frames omitted
2015-10-30 21:02:57,137 - ERROR [Spark - PageRankSpark:c.c.c.i.a.r.ProgramControllerServiceAdapter$1@81] - Program terminated with exception
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService.run(SparkRuntimeService.java:181) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService$2$1.run(SparkRuntimeService.java:247) [co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at co.cask.cdap.internal.app.runtime.spark.ScalaSparkFacade.<init>(ScalaSparkFacade.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.setupSparkContext(SparkProgramWrapper.java:76) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.run(SparkProgramWrapper.java:58) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.main(SparkProgramWrapper.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-assembly.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter.submit(AbstractSparkSubmitter.java:136) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter$5.run(AbstractSparkSubmitter.java:99) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_67]
... 1 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_67]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67]
... 20 common frames omitted
2015-10-30 21:02:57,147 - ERROR [pcontroller-spark.default.PageRankApp.PageRankSpark-888fe527-7f49-11e5-8c29-fa163e502f52:c.c.c.i.a.r.d.AbstractProgramTwillRunnable$1@309] - Program runner error out.
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281) ~[com.google.guava.guava-13.0.1.jar:na]
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService.run(SparkRuntimeService.java:181) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkRuntimeService$2$1.run(SparkRuntimeService.java:247) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/JobConf
at co.cask.cdap.internal.app.runtime.spark.ScalaSparkFacade.<init>(ScalaSparkFacade.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.setupSparkContext(SparkProgramWrapper.java:76) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.run(SparkProgramWrapper.java:58) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.SparkProgramWrapper.main(SparkProgramWrapper.java:40) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) ~[spark-assembly.jar:na]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-assembly.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter.submit(AbstractSparkSubmitter.java:136) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at co.cask.cdap.internal.app.runtime.spark.AbstractSparkSubmitter$5.run(AbstractSparkSubmitter.java:99) ~[co.cask.cdap.cdap-app-fabric-3.2.0.jar:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_67]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
... 1 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf
at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_67]
at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67]
at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67]
at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67]
... 20 common frames omitted
Terence Yim
2015. gada 30. okt. 17:51:1530.10.15
uz opc...@gmail.com,CDAP User
Hi,
The example is designed to work out of the box and it should works on CDH 5.4. It seems like the error is caused by cluster configuration. Can you verify the paths specified by “yarn.application.classpath” in the yarn-site.xml file points to locations containing Hadoop MapReduce jars?
Terence
--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/6f1f6cbf-3c0f-409c-832b-cdeaf61d0033%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
T Smith
2015. gada 30. okt. 18:07:3530.10.15
uz Terence Yim,CDAP User
Looks like the cm defaults are to have $HADOOP_MAPRED_HOME set in mapreduce.application.classpath but not yarn.application.classpath (our cluster creation is fully automated via cm api and we don't touch these settings). I'll adjust this and report back.
Cheers,
/T
T Smith
2015. gada 31. okt. 04:15:4931.10.15
uz Terence Yim,CDAP User
Hi,
Many thanks, that fixed that problem. For anyone else who sees this I adjusted the yarn.application.classpath via cm and it now looks like this -
<property>
<name>yarn.application.classpath</name>
<value>$HADOOP_CLIENT_CONF_DIR,$HADOOP_CONF_DIR,$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,$HADOOP_HDFS_HOM
E/*,$HADOOP_HDFS_HOME/lib/*,$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*,$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/li
b/*,$MR2_CLASSPATH</value>
</property>
Is there a known-good deployment descriptor for CDH posted somewhere? E.g. the output of: api/v6/cm/deployment.
Cheers,
/T
Terence Yim
2015. gada 31. okt. 12:29:0931.10.15
uz T Smith,CDAP User
Hi,
Thanks for the update. After some investigation, the spark shipped with CDH 5.4 is the hadoopless packaging, meaning Hadoop jars need to be provided separately and CDH ship with a modified spark-env.sh and spark.conf for that. We will see how to honor the settings in those two files to make the configuration easier.
Terence
Sent from my iPhone
Sent from my iPhone
Atbildēt visiem
Atbildēt autoram
Pārsūtīt
0 jauni ziņojumi