Livy Spark Job Server Startup

[Benjamin Kim]

I have a few questions on starting the Livy server in CDH5.4.8.

1. What user should I use to start the Livy server?
2. Is this the right sequence to run for starting up the Livy server?
1. sudo su
2. HUE_CONF_DIR=/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep HUE | tail -1 | awk '{print $9}'`
   
3. export HUE_CONF_DIR
4. su hue
5. ${HUE_HOME_DIR}/build/env/bin/hue livy_server &
   
6. disown
   
3. Is there an argument to allow Livy to pass the user to YARN?
4. Is there an argument to set the YARN pool to run under?

Thanks,

Ben

[Romain Rigaux]

#1 It does not matter ('hue' is recommended for now), just the user should be added in the hadoop proxy user list if you are using impersonation

#2 Yes, something like this will keep i by using the configuration entered in CM

#3 Look for the 'proxyUser' parameter https://github.com/cloudera/hue/tree/master/apps/spark/java#request-body

#4 'queue' parameter?

Romain

On Sat, Nov 21, 2015 at 12:52 AM, Benjamin Kim <bbui...@gmail.com> wrote:

I have a few questions on starting the Livy server in CDH5.4.8.

1. What user should I use to start the Livy server?
2. Is this the right sequence to run for starting up the Livy server?

1. HUE_CONF_DIR=/var/run/cloudera-scm-agent/process/`ls -alrt /var/run/cloudera-scm-agent/process | grep HUE | tail -1 | awk '{print $9}'`
   

1. ./build/env/bin/hue livy_server &
   
2. disown
   

1. Is there an argument to allow Livy to pass the user to YARN?
2. Is there an argument to set the YARN pool to run under?

Thanks,

Ben

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.