Titan 0.4.3 on HDP 2 / HBase - can't open the titan db

180 views
Skip to first unread message

Chung Chen

unread,
Feb 21, 2014, 10:11:01 PM2/21/14
to aureliu...@googlegroups.com
Hi,

My batchGraph program is running fine in my local HBase (hbase-0.94.16).  But, we need to run it in a cluster installed with HDP2 (HBase 0.96).  We are able to resolve the HBase issue per https://github.com/thinkaurelius/titan/issues/526.  That is running GraphOfTheGodsFactory.load(g) is working fine.  Unfortunately, the batchGraph program ran into the following issues in connecting to the same Titan/HBase:

Specifically, it failed on 

    TitanGraph graph = TitanFactory.open(configFile);

as enclosed below.

Can anyone shed some light on what could go wrong here and how to resolve the issues?

Thanks so much for all the help!
Chung

Enclosed:

0    [main] ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper  - ZooKeeper exists failed after 3 retries
2    [main] ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher  - hconnection-0x40dbf4b5 Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:200)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479)
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:794)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:627)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:381)
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:360)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:244)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:182)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.getAdminInterface(HBaseStoreManager.java:364)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureColumnFamilyExists(HBaseStoreManager.java:261)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:215)
at com.thinkaurelius.titan.diskstorage.Backend.getStore(Backend.java:231)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:243)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1164)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.create(BatchGraphLoad.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.main(BatchGraphLoad.java:269)






15585 [main] ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper  - ZooKeeper exists failed after 3 retries
15585 [main] ERROR org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher  - catalogtracker-on-hconnection-0x40dbf4b5 Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/meta-region-server
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:200)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:425)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77)
at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:195)
at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:220)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:268)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:284)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:247)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureColumnFamilyExists(HBaseStoreManager.java:262)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:215)
at com.thinkaurelius.titan.diskstorage.Backend.getStore(Backend.java:231)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:243)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1164)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.create(BatchGraphLoad.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.main(BatchGraphLoad.java:269)
15586 [main] FATAL org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation  - Unexpected exception during initialization, aborting
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/meta-region-server
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:200)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:425)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77)
at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:195)
at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:220)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:268)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:284)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:247)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureColumnFamilyExists(HBaseStoreManager.java:262)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:215)
at com.thinkaurelius.titan.diskstorage.Backend.getStore(Backend.java:231)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:243)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1164)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.create(BatchGraphLoad.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.main(BatchGraphLoad.java:269)
Exception in thread "main" com.thinkaurelius.titan.core.TitanException: Could not initialize backend
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:290)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1164)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.create(BatchGraphLoad.java:29)
at com.paypal.bds.datapipeline.titan.BatchGraphLoad.main(BatchGraphLoad.java:269)
Caused by: com.thinkaurelius.titan.diskstorage.TemporaryStorageException: Temporary failure in storage backend
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:254)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureColumnFamilyExists(HBaseStoreManager.java:262)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:215)
at com.thinkaurelius.titan.diskstorage.Backend.getStore(Backend.java:231)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:243)
... 6 more
Caused by: java.io.IOException: Attempt to start meta tracker failed.
at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:199)
at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:220)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:268)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:284)
at com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:247)
... 10 more
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/meta-region-server
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:200)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:425)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:77)
at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:195)
... 14 more


Chung Chen

unread,
Feb 26, 2014, 8:40:19 PM2/26/14
to aureliu...@googlegroups.com
At the end, we decided to use HBase 0.94.6 with HDP1 instead for the cluster deployment (BTW, the data was batch-loaded smoothly), and wait for HBase / HDP 2 in the future Titan releases.

Thanks,
Chung
Reply all
Reply to author
Forward
0 new messages