"Cannot run program 'spark-submit' " when start Livy after RPM install hue

41 views
Skip to first unread message

Wang Bin

unread,
Nov 26, 2015, 11:42:44 AM11/26/15
to Hue-Users
Before RPM, "hue/apps/spark/" directory structure as follows:
-babel.cfg
-hueversion.py -> ../../VERSION
-hueversion.pyc
-java
        -bin
        -conf
                -livy-defaults.conf.tmpl
        -dist.xml
        -livy-assembly
        -livy-core
        -livy-repl
        -livy-server
        -livy-yarn
        -pom.xml
        -README.rst
        -target
-java-lib
-Makefile
-setup.py
-src
        -spark
        -spark.egg-info


After RPM and install, "hue/apps/spark/" directory structure as follows:
-babel.cfg
-ext-eggs
-hueversion.py
-hueversion.pyc
-java-lib
        -livy-assembly.jar
-Makefile
-setup.py
-src
        -spark
        -spark.egg-info

configured "/etc/profile" as follows:
# set spark environment
SPARK_HOME=/home/spark/
export SPARK_HOME
export PATH=${PATH}:${SPARK_HOME}/bin

the hue file
... 
su -s /bash/shell root "cd /var/log/hue/ && nohup /usr/lib/hue/build/env/bin/hue livy_server > livy.log &"
...

when "service hue start", the livy.log as follows: 
Failed to run spark-submit executable: java.io.IOException: Cannot run program "spark-submit": error=2, No such file or directory

I have tried as follows:
1, insert the congiguration into "/etc/hue/conf.empty/hue.ini"
livy.server.spark-submit = /home/spark/bin/spark-submit

The same Error as the log.

2、I want to configure the spark-submit into livy-defaults.conf, but no found the file.
Reply all
Reply to author
Forward
0 new messages