Titan + HBase on a MapR Hadoop Cluster is it possible and how?

[Blagovest Trenchev]

Hi ,

I'm trying to run Titan on a MapR Hadoop Cluster. However, I'm getting this:

gremlin> g = TitanFactory.open('conf/titan-hbase-es.properties')
08:43:28 WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
gremlin> g = TitanFactory.open('conf/titan-hbase-es.properties')
08:43:33 WARN  org.apache.zookeeper.ClientCnxn  - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
08:43:33 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper  - Possibly transient ZooKeeper, quorum=127.0.0.1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
08:43:34 WARN  org.apache.zookeeper.ClientCnxn  - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
08:43:34 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper  - Possibly transient ZooKeeper, quorum=127.0.0.1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
08:43:35 WARN  org.apache.zookeeper.ClientCnxn  - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
08:43:37 WARN  org.apache.zookeeper.ClientCnxn  - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
08:43:37 WARN  org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper  - Possibly transient ZooKeeper, quorum=127.0.0.1:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid


I did some research and found out this:
http://mail-archives.apache.org/mod_mbox/hbase-user/201211.mbox/%3CCAPQV63WmTcaXfw7HoD=1TequQHOBf_3R=wUMxwf60...@mail.gmail.com%3E

In my case the problem might be that MapR are using a modified version of Apache HBase and thus the communication fails.

I also checked the ZooKeeper configuration and found that the quorum property is set properly to the ip of the machine.

Any ideas/suggestions?

Best,
Blago

[Yuriy]

I don't remember the details, but if you specify zookeeper with an ip (127.0.0.1) in your config file that might not work - you need to provide a host name instead (e.g. localhost). At least, that was the problem on my Hadoop cluster (Hortonworks).

http://mail-archives.apache.org/mod_mbox/hbase-user/201211.mbox/%3CCAPQV63WmTcaXfw7HoD=1TequQHOBf_3R=wUMxwf60qD_Gxq...@mail.gmail.com%3E

[Blagovest Trenchev]

Hi Yuriy,

Thanks for the reply!

You are correct for that the communication with Zookeeper is the problem. It turns out that the MapR distribution of Zookeeper listens to port 5181 instead of the default 2181. This was my problem.

For any future readers, here's the configuration for Titan that worked for me for Titan 0.5.5 , HBase 0.98+ and MapR Hadoop 2.5.smth

========== Titatn Config =======

storage.backend=hbase
storage.hostname=127.0.0.1
# Must use for MapR Hadoop Distributions ( Tells Titan to contact Zookeeper on port 5181 instead of 2181 )
storage.hbase.ext.hbase.zookeeper.property.clientPort=5181
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.5

index.search.backend=elasticsearch
# property introduced in Titan 0.5.5 ( but I'm not sure if I'm using the correct value )
index.search.elasticsearch.interface=NODE
index.search.hostname=127.0.0.1
index.search.elasticsearch.client-only=true

storage.index.search.backend=elasticsearch
storage.index.search.directory=/tmp/searchindex
storage.index.search.client-only=false
storage.index.search.local-mode=false

==========================================

Hope it helps someone someday.

All the best,
Blago