coordinator node exception

332 views
Skip to first unread message

xiaoxiaoluo

unread,
Aug 8, 2014, 11:32:57 AM8/8/14
to druid-de...@googlegroups.com
Hi, All:

When I load the date using indexing, the coordinator always have the exception logs, I have no idea why cause it.Can somebody help me?

Here is the log:

2014-08-08 14:49:23,110 ERROR [Master-PeonExec--0] io.druid.server.coordinator.LoadQueuePeon - Server[/druid/loadQueue/hostname:8081], throwable caught when submitting [SegmentChangeRequestLoad{segment=DataSegment{size=4462, shardSpec=NoneShardSpec, metrics=[count, added, deleted, delta], dimensions=[anonymous, city, continent, country, language, namespace, newpage, page, region, robot, unpatrolled, user], version='2014-08-08T10:46:55.206Z', loadSpec={type=hdfs, path=/druid/segments/wikipedia/20130831T000000.000Z_20130901T000000.000Z/2014-08-08T10_46_55.206Z/0/index.zip}, interval=2013-08-31T00:00:00.000Z/2013-09-01T00:00:00.000Z, dataSource='wikipedia', binaryVersion='9'}}].
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /druid/loadQueue/hostname:8081/wikipedia_2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z_2014-08-08T10:46:55.206Z
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
        at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:676)
        at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:660)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
        at org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:656)
        at org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:441)
        at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:431)
        at org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
        at io.druid.server.coordinator.LoadQueuePeon$4.run(LoadQueuePeon.java:248)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
2014-08-08 14:50:07,329 INFO [DatabaseSegmentManager-Exec--0] io.druid.db.DatabaseSegmentManager - Polled and found 1 segments in the database
2014-08-08 14:50:09,073 INFO [DatabaseRuleManager-Exec--0] io.druid.db.DatabaseRuleManager - Polled and found rules for 1 datasource(s)
2014-08-08 14:50:23,109 INFO [Coordinator-Exec--0] io.druid.server.coordinator.ReplicationThrottler - [_default_tier]: Replicant create queue is empty.
2014-08-08 14:50:23,109 INFO [Coordinator-Exec--0] io.druid.server.coordinator.ReplicationThrottler - [_default_tier]: Replicant terminate queue is empty.

I use zookeeper client to connect zookeeper to check this path, I found it existed, but why it has "NodeExistsException"

Here is the config:

druid.host=localhost
druid.service=druid/v1/coordinator
druid.port=8082

druid.zk.service.host=localhost

druid.db.connector.connectURI=jdbc\:mysql\:/hostname\:3306/druid
druid.db.connector.user=druid
druid.db.connector.password=diurd

druid.coordinator.startDelay=PT70s

Thanks

Fangjin Yang

unread,
Aug 8, 2014, 11:41:30 AM8/8/14
to druid-de...@googlegroups.com
Hi XiaoXiao, this is a known issue that we recently fixed. You can find a fix for it in 0.6.121.1 (http://static.druid.io/artifacts/releases/druid-services-0.6.121.1-bin.tar.gz) or with the new RC.


--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/c5c6d16e-885b-4c10-a76b-a2e70e1ec778%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

xiaoxiaoluo

unread,
Aug 8, 2014, 11:47:00 AM8/8/14
to druid-de...@googlegroups.com
Hi, Fangjin:

I used this stable version to build up this cluster.I found the segments have been in deep storage(hdfs). It seems the coordinator inform historical node to load index failed?

Thanks

在 2014年8月8日星期五UTC+8下午11时41分30秒,Fangjin Yang写道:

xiaoxiaoluo

unread,
Aug 8, 2014, 12:35:53 PM8/8/14
to druid-de...@googlegroups.com
Hi, Fangjin:

Have been fixed, It caused by hadoop version confilcts. I used hadoop-2.2 as deep storage. 
I pick up jars of hadoop-2.2 in a dir, then include it in java start command. It's ok now.

Thanks.

在 2014年8月8日星期五UTC+8下午11时47分00秒,xiaoxiaoluo写道:
Reply all
Reply to author
Forward
0 new messages