Lots of ZK errors I guess - NoNode for /druid/announcementsPath etc

[David Montgomery]

Allmost there:)  I hope

Below is my config file.  I had to add druid.zk.paths.discoveryPath=/druid/discoveryPath.  It was not in the sample on the wiki along with druid.database.ruleTable

Do I have to do some zookeeper bootstrap on my own?  How do I resolve the below?

# ZK quorum IPs; if empty, the use demo mode
druid.zk.service.host=127.0.0.1:2181
# msec; high value means tolerate slow zk nodes, default is to wait about 3 weeks
druid.zk.service.connectionTimeout=1000000

# ZK path for service discovery within the cluster
druid.zk.paths.announcementsPath=/druid/announcementsPath

# Legacy path, must be set, but can be ignored
druid.zk.paths.indexesPath=/druid/indexesPath

druid.zk.paths.indexer.tasksPath=/druid/tasksPath
druid.zk.paths.indexer.statusPath=/druid/statusPath

# ZK path for load/drop protocol between Master/Compute
druid.zk.paths.loadQueuePath=/druid/loadQueuePath

# ZK path for Master leadership election
druid.zk.paths.masterPath=/druid/masterPath

# ZK path for publishing served segments
druid.zk.paths.servedSegmentsPath=/druid/servedSegmentsPath

druid.zk.paths.discoveryPath=/druid/discoveryPath



2013-02-12 16:28:07,227 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void com.metamx.druid.master.DruidMaster.start()] on object[com.metamx.druid.master.DruidMaster@180a1d7b].
2013-02-12 16:28:07,272 INFO [main] com.metamx.druid.master.DruidMaster - I am the master, all must bow!
2013-02-12 16:28:07,290 WARN [DatabaseSegmentManager-Exec--0] com.metamx.druid.db.DatabaseSegmentManager - No segments found in the database!
2013-02-12 16:28:07,326 ERROR [main] com.metamx.druid.http.ServerMain - Error when starting up.  Failing.
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:108)
    at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:54)
    at com.metamx.druid.http.MasterMain.main(MasterMain.java:198)
Caused by: org.I0Itec.zkclient.exception.ZkNoNodeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /druid/announcementsPath
    at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
    at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
    at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413)
    at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
    at com.metamx.druid.client.ZKPhoneBook$InternalPhoneBook.registerListener(ZKPhoneBook.java:315)
    at com.metamx.phonebook.BasePhoneBook.registerListener(BasePhoneBook.java:176)
    at com.metamx.druid.client.InventoryManager.start(InventoryManager.java:102)
    at com.metamx.druid.master.DruidMaster.becomeMaster(DruidMaster.java:426)
    at com.metamx.druid.master.DruidMaster.start(DruidMaster.java:376)
    ... 7 more
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /druid/announcementsPath
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
    at org.I0Itec.zkclient.ZkConnection.getChildren(ZkConnection.java:99)
    at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416)
    at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413)
    at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
    ... 14 more

[Hagen Rother]

Druid doesn't create the paths in zookeeper, you need to use zkCli.sh and create them yourself.

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Hagen Rother

Senior Software Architect | madvertise

madvertise Mobile Advertising GmbH Paul-Lincke-Ufer 39-40

twitter |  facebook | xing  |  LinkedIn
Phone:	+49 176 15 00 38 77
Fax:	+49 30 69 53 54 52
E-Mail:	hagen....@madvertise.com
Internet:	www.madvertise.com

Meet the madvertise team @ MWC
Hall 8.1 Booth C15. See you in Barcelona!

Monetise your app: Sign up with madvertise
madvertise is hiring, check: madvertise Career Center
Check out our WORK & PLAY MAGAZINE

Managing Director | Christof Wittig
Jurisdiction | Local Court Berlin-Charlottenburg HRB 116329 B

The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination, distribution, forwarding, or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited without the express permission of the sender. If you received this communication in error, please contact the sender and delete the material from any computer.

[Eric Tschetter]

Another option, is that there is a DruidSetup main in the Druid code base which will setup ZK and the database for you.

com.metamx.druid.utils.DruidSetup

It takes a couple of verbs.  One of which is "put" which will create the relevant znodes for you.  Also, "dbprep" will create relevant database tables.

In order to run, if you just replace the "com.metamx.druid.http.ComputeMain" with "com.metamx.druid.utils.DruidSetup" it should tell you about the verbs.  Then you just run it again and add the various command line parameters it asks for at the end of your java invocation.

--Eric

--Eric

[James Estes]

FWIW, the behavior I've seen is if i have a completely clean install of everything, all I need to do is create the db (not tables) and db user, and only have to add /druid/announcements to zk.  Firing up the master creates everything else (the db tables and the other zk paths) automagically.

James

[Jay Wang]

I have 2 questions: 

1. What do you put for "druid.zk.paths.announcementsPath=   " the announcementsPath? Do you need to figure that out using the zkCli (as Hagen mentioned?) 

2. I have the db and db user (from when I ran the druid-test-harness, I'm guessing I can just use those?). Yet, when I run Master I get "Exception in thread "main" com.metamx.common.ISE: Property[druid.database.ruleTable] not specified" I don't see a druid.database.ruleTable in runtimeProperties? Do I need to create one? 

Thanks,

Jay

[Jay Wang]

Hey David, 

I'm having the same error. Did you manage to get it fixed? 

I'm trying Eric's solution to little avail... 

Jay

[Eric Tschetter]

1. What do you put for "druid.zk.paths.announcementsPath=   " the announcementsPath? Do you need to figure that out using the zkCli (as Hagen mentioned?) 

This can basically be anything, it just has to exist on Zookeeper.  You can choose to create it manually via zkCli, or you should be able to just set the property to something in your runtime config file and then you should be able to use DruidClient to create the "right" paths in ZK.  Basically, you just need whatever paths you specify in the config to exist.

 

2. I have the db and db user (from when I ran the druid-test-harness, I'm guessing I can just use those?). Yet, when I run Master I get "Exception in thread "main" com.metamx.common.ISE: Property[druid.database.ruleTable] not specified" I don't see a druid.database.ruleTable in runtimeProperties? Do I need to create one? 

Yes, you need to create a "druid.database.ruleTable" property in your runtime.properties.  This is a new property that we added recently...  You can name it to whatever you want, but it does need a name.

--Eric

 

To view this discussion on the web visit https://groups.google.com/d/msg/druid-development/-/QVtDiSmC3R8J.