Livy not connected to the configured resource manager but the default local one

Shady

unread,

Jun 17, 2016, 4:40:21 AM6/17/16

to Hue-Users

Hi all,

I am using livy to run some PySpark code in yarn mode. But my livy server is connecting the wrong local resource manager.

I configured as the official documentation like this:

export SPARK_HOME=/usr/lib/spark
export HADOOP_CONF_DIR=/etc/hadoop/conf

export CLASSPATH=`hadoop classpath`

And I configured livy.server.session.factory=yarn in the conf/livy-defaults.conf file. Every time I start the livy server, I get log like this:

INFO yarn.Client: Resource Manager address: 0.0.0.0:8032

This is obviously a default local server, but I am using a HA resource manager. I also configured the HA settings in hue/desktop/conf/hue.ini:

[[[default]]]

# Enter the host on which you are running the ResourceManager

resourcemanager_host=rm1.test.com

# The port where the ResourceManager IPC listens on

resourcemanager_port=8032

# Whether to submit jobs to this cluster

submit_to=True

# Resource Manager logical name (required for HA)

logical_name=rm1

# URL of the ProxyServer API

proxy_api_url=rm1.test.com:8088

# URL of the HistoryServer API

history_server_api_url=rm1.test.com:18080

[[[ha]]]

# Enter the host on which you are running the ResourceManager

resourcemanager_host=rm2.test.com

# The port where the ResourceManager IPC listens on

resourcemanager_port=8032

# Resource Manager logical name (required for HA)

logical_name=rm2

# Un-comment to enable

submit_to=True

# URL of the ResourceManager API

resourcemanager_api_url=rm2.testcom:8088

And because of this error, livy will fail to create a Spark session:

The Spark session could not be created in the cluster: at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:607) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:705) at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:368) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1521) at org.apache.hadoop.ipc.Client.call(Client.java:1438) ... 33 more

I googled a lot, still no answer. Any clue guys?

Shady

unread,

Jun 20, 2016, 2:54:57 AM6/20/16

to Hue-Users

Has anyone come across this problem?

在 2016年6月17日星期五 UTC+8下午4:40:21，Shady写道：

Shady

unread,

Jun 20, 2016, 10:52:24 PM6/20/16

to Hue-Users

walk through the source code, seems livy does not support Resource Manager HA, you have to configure the exact active resource manager in yarn-site.xml.