Before RPM, "hue/apps/spark/" directory structure as follows:
-babel.cfg
-hueversion.py -> ../../VERSION
-hueversion.pyc
-java
-bin
-conf
-livy-defaults.conf.tmpl
-dist.xml
-livy-assembly
-livy-core
-livy-repl
-livy-server
-livy-yarn
-pom.xml
-README.rst
-target
-java-lib
-Makefile
-setup.py
-src
-spark
-spark.egg-info
After RPM and install, "hue/apps/spark/" directory structure as follows:
-babel.cfg
-ext-eggs
-hueversion.py
-hueversion.pyc
-java-lib
-livy-assembly.jar
-Makefile
-setup.py
-src
-spark
-spark.egg-info
configured "/etc/profile" as follows:
# set spark environment
SPARK_HOME=/home/spark/
export SPARK_HOME
export PATH=${PATH}:${SPARK_HOME}/bin
the hue file
...
su -s /bash/shell root "cd /var/log/hue/ && nohup /usr/lib/hue/build/env/bin/hue livy_server > livy.log &"
...
when "service hue start", the livy.log as follows:
Failed to run spark-submit executable: java.io.IOException: Cannot run program "spark-submit": error=2, No such file or directory
I have tried as follows:
1, insert the congiguration into "/etc/hue/conf.empty/hue.ini"
livy.server.spark-submit = /home/spark/bin/spark-submit
The same Error as the log.
2、I want to configure the spark-submit into livy-defaults.conf, but no found the file.