HBaseClient: Lost connection with the -ROOT- region
John Pitney
I'm trying to install OpenTSB for the first time on a Fedora 16 64-bit
system. After following the steps for getting started, I'm not able
to get tsd to launch. I've probably got something mis-configured, but
I'd appreciate any pointers on where I might have gone wrong.
I am able to run through the HBase shell quick tutorial, but it takes
several seconds just to show a prompt. Is it possible that the
connection is simply timing out?
Here's what I see:
$ ./build/tsdb tsd --port=4242 --staticroot=build/staticroot --
cachedir=$tsdtmp
2011-12-16 22:35:00,952 INFO [main] TSDMain: Starting.
2011-12-16 22:35:00,968 INFO [main] TSDMain: net.opentsdb built at
revision 698b2f6 (MINT)
2011-12-16 22:35:00,969 INFO [main] TSDMain: Built on 2011/12/17
04:17:12 +0000 by jo...@dogbert.localdomain:/home/john/Downloads/
opentsdb/build
2011-12-16 22:35:01,229 INFO [main] HBaseClient: Need to find the -
ROOT- region
2011-12-16 22:35:01,317 INFO [main] ZooKeeper: Client
environment:zookeeper.version=3.3.1-942149, built on 05/07/2010 17:14
GMT
2011-12-16 22:35:01,317 INFO [main] ZooKeeper: Client
environment:host.name=dogbert
2011-12-16 22:35:01,318 INFO [main] ZooKeeper: Client
environment:java.version=1.6.0_22
2011-12-16 22:35:01,318 INFO [main] ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
2011-12-16 22:35:01,318 INFO [main] ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/
jre
2011-12-16 22:35:01,319 INFO [main] ZooKeeper: Client
environment:java.class.path=/home/john/Downloads/opentsdb/build/
third_party/hbase/hbaseasync-1.0.jar:/home/john/Downloads/opentsdb/
build/third_party/slf4j/jcl-over-slf4j-1.6.1.jar:/home/john/Downloads/
opentsdb/build/third_party/slf4j/log4j-over-slf4j-1.6.1.jar:/home/john/
Downloads/opentsdb/build/third_party/logback/logback-
classic-0.9.24.jar:/home/john/Downloads/opentsdb/build/third_party/
logback/logback-core-0.9.24.jar:/home/john/Downloads/opentsdb/build/
third_party/netty/netty-3.2.3.Final.jar:/home/john/Downloads/opentsdb/
build/third_party/slf4j/slf4j-api-1.6.1.jar:/home/john/Downloads/
opentsdb/build/third_party/suasync/suasync-1.1.0.jar:/home/john/
Downloads/opentsdb/build/third_party/zookeeper/zookeeper-3.3.1.jar:/
home/john/Downloads/opentsdb/build/tsdb-1.0.jar:/home/john/Downloads/
opentsdb/build/../src
2011-12-16 22:35:01,319 INFO [main] ZooKeeper: Client
environment:java.library.path=/usr/lib/jvm/java-1.6.0-
openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-
openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-
openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/
usr/lib64:/lib64:/lib:/usr/lib
2011-12-16 22:35:01,320 INFO [main] ZooKeeper: Client
environment:java.io.tmpdir=/tmp
2011-12-16 22:35:01,320 INFO [main] ZooKeeper: Client
environment:java.compiler=<NA>
2011-12-16 22:35:01,320 INFO [main] ZooKeeper: Client
environment:os.name=Linux
2011-12-16 22:35:01,321 INFO [main] ZooKeeper: Client
environment:os.arch=amd64
2011-12-16 22:35:01,321 INFO [main] ZooKeeper: Client
environment:os.version=3.1.4-1.fc16.x86_64
2011-12-16 22:35:01,321 INFO [main] ZooKeeper: Client
environment:user.name=john
2011-12-16 22:35:01,322 INFO [main] ZooKeeper: Client
environment:user.home=/home/john
2011-12-16 22:35:01,327 INFO [main] ZooKeeper: Client
environment:user.dir=/home/john/Downloads/opentsdb
2011-12-16 22:35:01,330 INFO [main] ZooKeeper: Initiating client
connection, connectString=localhost sessionTimeout=5000
watcher=org.hbase.async.HBaseClient$ZKClient@3b75078b
2011-12-16 22:35:01,415 INFO [main-SendThread()] ClientCnxn: Opening
socket connection to server localhost/127.0.0.1:2181
2011-12-16 22:35:01,466 INFO [main-SendThread(localhost:2181)]
ClientCnxn: Socket connection established to localhost/127.0.0.1:2181,
initiating session
2011-12-16 22:35:01,494 INFO [main-SendThread(localhost:2181)]
ClientCnxn: Session establishment complete on server localhost/
127.0.0.1:2181, sessionid = 0x1344a1478050008, negotiated timeout =
5000
2011-12-16 22:35:01,530 INFO [main-EventThread] HBaseClient:
Connecting to -ROOT- region @ 127.0.0.1:54983
2011-12-16 22:35:01,667 INFO [main-EventThread] ZooKeeper: Session:
0x1344a1478050008 closed
2011-12-16 22:35:01,712 INFO [New I/O client worker #1-1]
HBaseClient: Added client for region RegionInfo(table=".META.",
region_name=".META.,,1", stop_key=""), which was added to the regions
cache. Now we know that RegionClient@839710315(chan=[id: 0x651dba45, /
127.0.0.1:47127 => /127.0.0.1:54983], #pending_rpcs=0, #edits=0,
#rpcs_inflight=1) is hosting 1 region.
2011-12-16 22:35:01,727 INFO [main] HBaseClient: Lost connection with
the -ROOT- region
Exception in thread "main" java.lang.RuntimeException: Initialization
failed
at net.opentsdb.tools.TSDMain.main(TSDMain.java:148)
Caused by: org.hbase.async.TableNotFoundException: "tsdb"
at org.hbase.async.HBaseClient$7.call(HBaseClient.java:1155)
at org.hbase.async.HBaseClient$7.call(HBaseClient.java:1152)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1260)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1239)
at com.stumbleupon.async.Deferred.callback(Deferred.java:987)
at org.hbase.async.HBaseRpc.callback(HBaseRpc.java:307)
at org.hbase.async.RegionClient.decode(RegionClient.java:1084)
at org.hbase.async.RegionClient.decode(RegionClient.java:87)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:
471)
at
org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:
444)
at
org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:
80)
at org.hbase.async.RegionClient.handleUpstream(RegionClient.java:921)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:
545)
at
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:
540)
at org.hbase.async.HBaseClient
$RegionClientPipeline.sendUpstream(HBaseClient.java:1861)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:
274)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:
261)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:
350)
at
org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:
281)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:
201)
at
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:
108)
at
org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:
46)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
1110)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
$ time ./bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
hbase(main):001:0> exit
real 0m39.148s
user 0m16.571s
sys 0m0.950s
Best regards,
John
John Pitney
On Dec 16, 10:54 pm, John Pitney <john.pit...@gmail.com> wrote:
> Hi,
>
> I'm trying to install OpenTSB for the first time on a Fedora 16 64-bit
> system. After following the steps for getting started, I'm not able
> to get tsd to launch. I've probably got something mis-configured, but
> I'd appreciate any pointers on where I might have gone wrong.
Sorry for the noise. I started over, and now I've got a running tsd.
I'm not sure what I did wrong, but it was likely a mistake on my
part.
Time to start collecting data!
Best regards,
John
tsuna
> Caused by: org.hbase.async.TableNotFoundException: "tsdb"
Looks like you had forgotten to create the tables :)
> $ time ./bin/hbase shell
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 0.90.4, r1150278, Sun Jul 24 15:53:29 PDT 2011
>
> hbase(main):001:0> exit
>
> real 0m39.148s
> user 0m16.571s
> sys 0m0.950s
Yeah the shell isn't particularly fast. Java apps have always been
slow to start, but can you imagine JRuby? :)
--
Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com
John Pitney
> On Fri, Dec 16, 2011 at 8:54 PM, John Pitney <john.pit...@gmail.com> wrote:
> > Caused by: org.hbase.async.TableNotFoundException: "tsdb"
>
> Looks like you had forgotten to create the tables :)
I remember doing the table-creation step before this issue cropped up,
but I tried enabling LZO, and maybe that failed in a way I didn't
notice. Since the "lost connection" message showed up first, I took
that as the cause and the org.hbase.async.TableNotFoundException:
"tsdb" as the effect.
Anyway, it's working now after my do-over. Thanks!
Cheers,
John