2018-06-27 09:37:46,239 - ERROR [main:c.c.c.a.r.s.d.SparkContainerLauncher@145] - Exception raised when calling org.apache.spark.deploy.yarn.ApplicationMaster.main(String[]) method
java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
at co.cask.cdap.app.runtime.spark.distributed.SparkContainerLauncher.launch(SparkContainerLauncher.java:114) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
at org.apache.spark.deploy.yarn.ApplicationMaster.main(Unknown Source) [cdap-spark-launcher.jar:na]
Caused by: java.lang.IllegalStateException: SparkContextProvider.getSparkContext should only be called in Spark executor process.
at com.google.common.base.Preconditions.checkState(Preconditions.java:149) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.app.runtime.spark.SparkRuntimeContextProvider.createIfNotExists(SparkRuntimeContextProvider.java:155) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
at co.cask.cdap.app.runtime.spark.SparkRuntimeContextProvider.get(SparkRuntimeContextProvider.java:136) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
... 6 common frames omitted
--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+unsubscribe@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/083c262e-3282-498a-9d25-e49b6d2ffc97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi,Spark 2.3.0 isn't supported at this time. We plan to support spark 2.3 in a future release.Thanks,MattOn Wed, Jun 27, 2018 at 11:20 AM, <ankur.gu...@gmail.com> wrote:We are trying to run realtime pipelines using CDAP. All pipelines are failing when it is being submitted to spark. Following is the exception seen2018-06-27 09:37:46,239 - ERROR [main:c.c.c.a.r.s.d.SparkContainerLauncher@145] - Exception raised when calling org.apache.spark.deploy.yarn.ApplicationMaster.main(String[]) method
java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_131]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_131]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_131]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
at co.cask.cdap.app.runtime.spark.distributed.SparkContainerLauncher.launch(SparkContainerLauncher.java:114) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
at org.apache.spark.deploy.yarn.ApplicationMaster.main(Unknown Source) [cdap-spark-launcher.jar:na]
Caused by: java.lang.IllegalStateException: SparkContextProvider.getSparkContext should only be called in Spark executor process.
at com.google.common.base.Preconditions.checkState(Preconditions.java:149) ~[com.google.guava.guava-13.0.1.jar:na]
at co.cask.cdap.app.runtime.spark.SparkRuntimeContextProvider.createIfNotExists(SparkRuntimeContextProvider.java:155) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
at co.cask.cdap.app.runtime.spark.SparkRuntimeContextProvider.get(SparkRuntimeContextProvider.java:136) ~[co.cask.cdap.cdap-spark-core2_2.11-4.3.4.jar:na]
... 6 common frames omitted
--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/083c262e-3282-498a-9d25-e49b6d2ffc97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "CDAP User" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cdap-user/DuIt8S73PHM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/CAEZUc-psTo4BHpc4XrUK4Pz4akz9XUYqXn9WbBA18KMLefUMHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.