Re: Hue - Spark [Help]

[Erick Tryzelaar]

Hello Nawal,

I'm sorry you're still having trouble. Perhaps it's a spark version problem? Do you have Spark 1.4 installed? Another thing that would help is to eliminate livy-server. Could you do:

% cd $HUE_DIR/apps/spark/java

% mvn package -DskipTests

% ./bin/livy-repl spark

then in another shell, do;

% curl -XPOST -H 'Content-Type: application/json' localhost:8999/execute -d '{"code": "1+1"}'

Does that work?

-Erick

On Wed, Aug 26, 2015 at 2:20 AM, Nawal M'HAJAR <nawal....@gmail.com> wrote:

Hello Erick,

I know that I spammed you by the emails ^^

But I still have the same issus :( 

I got hue 3.9 from master on github, but unfortunately I have this in the livy_server console :

 ERROR Session: Exception when executing code
java.lang.IllegalArgumentException: requirement failed
        at scala.Predef$.require(Predef.scala:221)
        at com.cloudera.hue.livy.repl.scala.SparkInterpreter.execute(SparkInterpreter.scala:77)
        at com.cloudera.hue.livy.repl.Session.com$cloudera$hue$livy$repl$Session$$executeCode(Session.scala:96)

        at com.cloudera.hue.livy.repl.Session$$anonfun$3.apply(Session.scala:71)
        at com.cloudera.hue.livy.repl.Session$$anonfun$3.apply(Session.scala:71)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
        at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
127.0.0.1 - - - 26/août/2015:02:09:18 -0700 "POST /execute HTTP/1.1" 200 162
127.0.0.1 - - - 26/août/2015:02:09:18 -0700 "GET /sessions/1/statements/0 HTTP/1.1" 200 182
127.0.0.1 - - - 26/août/2015:02:09:19 -0700 "GET /sessions/1/statements/0 HTTP/1.1" 200 182````

I'm in internship In Capgemini France, and my main objective is to compare the three Big Data technologies : Spark, storm and Flink by doing the same use case.

I want to execute this example : https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterPopularTags.scala

But with errors that I have , I still not able to do this.

Appreciate your help  :)