Hue notebook error when adding a new snippet

[jaesub park]

Hi, 

I have a problem when i try to use notebook in hue by adding a new snippet.

whenever i try to add new snippet(R, Pyspark, scala), hue send me a error message like this.

The Spark session could not be created in the cluster: at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:163) at org.apache.spark.deploy.SparkSubmit$$anon$1.run(SparkSubmit.scala:161) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:161) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

livy logs 

INFO SparkProcessBuilder: Running 'spark-submit' '--name' 'Livy' '--jars' '/opt/cloudera/livy/repl/target/jars/scalatra-json_2.10-2.3.0.jar,/opt/cloudera/livy/repl/target/jars/joda-convert-1.6.jar,/opt/cloudera/livy/repl/target/jars/jetty-http-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/commons-codec-1.9.jar,/opt/cloudera/livy/repl/target/jars/grizzled-slf4j_2.10-1.0.2.jar,/opt/cloudera/livy/repl/target/jars/scalatra_2.10-2.3.0.jar,/opt/cloudera/livy/repl/target/jars/mime-util-2.1.3.jar,/opt/cloudera/livy/repl/target/jars/jetty-servlet-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/kryo-2.22.jar,/opt/cloudera/livy/repl/target/jars/rl_2.10-0.4.10.jar,/opt/cloudera/livy/repl/target/jars/jetty-server-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/livy-api-0.2.0-SNAPSHOT.jar,/opt/cloudera/livy/repl/target/jars/async-http-client-1.9.33.jar,/opt/cloudera/livy/repl/target/jars/jetty-util-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/juniversalchardet-1.0.3.jar,/opt/cloudera/livy/repl/target/jars/jetty-io-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/scalatra-common_2.10-2.3.0.jar,/opt/cloudera/livy/repl/target/jars/livy-client-common-0.2.0-SNAPSHOT.jar,/opt/cloudera/livy/repl/target/jars/jetty-security-9.2.10.v20150310.jar,/opt/cloudera/livy/repl/target/jars/livy-repl-0.2.0-SNAPSHOT.jar,/opt/cloudera/livy/repl/target/jars/livy-core-0.2.0-SNAPSHOT.jar,/opt/cloudera/livy/repl/target/jars/joda-time-2.3.jar,/opt/cloudera/livy/repl/target/jars/netty-3.10.5.Final.jar' '--class' 'com.cloudera.livy.repl.Main' '--conf' 'spark.livy.callbackUrl=http://10.0.2.201:8998/sessions/7/callback' '--conf' 'spark.livy.port=0' '--proxy-user' 'jspark' 'spark-internal' 'spark'

We use CDH 5.7.1 and Hue 3.9.0 and Spark 1.6.0

i changed several hue configurations and spark configurations, it doesn't work.

I needs some helps, a simple comments or suggestions are welcome!

Thanks.

[Romain Rigaux]

The error is at the Livy level, so would recommend to reproduce the curl against the latest livy.io (we recommend to use the upstream Livy now).

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[jaesub park]

Thanks for your reply, we solve this problem by rebuilding and using latest version of Livy (https://github.com/cloudera/livy)