Error when running realtime node with kafka 0.8

[Kyaw Zin Hein]

Hi,

When I try to run the realtime node with kafka 0.8, I've got the following error. 

2014-01-26 13:03:11,098 INFO [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=192.168.2.22 sessionTimeout=30000 watcher=org.apache.curator.ConnectionState@49b96770

2014-01-26 13:03:11,110 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.curator.announcement.Announcer.start()] on object[io.druid.curator.announcement.Announcer@5d2a2ab8].

2014-01-26 13:03:11,111 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.client.ServerInventoryView.start() throws java.lang.Exception] on object[io.druid.client.SingleServerInventoryView@dd2cc4a].

2014-01-26 13:03:11,113 INFO [main-SendThread(192.168.2.22:2181)] org.apache.zookeeper.ClientCnxn - Opening socket connection to server 192.168.2.22/192.168.2.22:2181. Will not attempt to authenticate using SASL (unknown error)

2014-01-26 13:03:11,118 INFO [main-SendThread(192.168.2.22:2181)] org.apache.zookeeper.ClientCnxn - Socket connection established to 192.168.2.22/192.168.2.22:2181, initiating session

2014-01-26 13:03:11,122 INFO [main] com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler - Invoking start method[public void io.druid.segment.realtime.RealtimeManager.start() throws java.io.IOException] on object[io.druid.segment.realtime.RealtimeManager@2e218ba2].

2014-01-26 13:03:11,122 INFO [main] io.druid.segment.realtime.RealtimeManager - Calling the FireDepartment and getting a Firehose.

2014-01-26 13:03:11,293 ERROR [main] io.druid.cli.CliBroker - Error when starting up.  Failing.

java.lang.reflect.InvocationTargetException

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at com.metamx.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:331)

        at com.metamx.common.lifecycle.Lifecycle.start(Lifecycle.java:250)

        at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:136)

        at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:72)

        at io.druid.cli.ServerRunnable.run(ServerRunnable.java:40)

        at io.druid.cli.Main.main(Main.java:91)

Caused by: java.lang.IllegalArgumentException: requirement failed: Missing required property 'zookeeper.connect'

        at scala.Predef$.require(Predef.scala:214)

        at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:158)

        at kafka.utils.ZKConfig.<init>(ZkUtils.scala:831)

        at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:73)

        at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:77)

        at io.druid.firehose.kafka.KafkaEightFirehoseFactory.connect(KafkaEightFirehoseFactory.java:76)

        at io.druid.segment.realtime.FireDepartment.connect(FireDepartment.java:90)

        at io.druid.segment.realtime.RealtimeManager$FireChief.init(RealtimeManager.java:150)

        at io.druid.segment.realtime.RealtimeManager.start(RealtimeManager.java:87)

        ... 10 more

I have changed 'druid.extensions.coordinates' to 'io.druid.extensions:druid-kafka-eight:0.6.52' in realtime runtime.properties, and Kafka is running with no error. I would appreciate if anyone can guide me what is causing that error. Thanks. ;)

Best Regards,

Zin Hein

[Fangjin Yang]

Hi Zin,

You need to set the kafka 8 configurations as part of your firehose consumer props.

If you look through the exception stack trace, you'll see:

 java.lang.IllegalArgumentException: requirement failed: Missing required property 'zookeeper.connect'

Thx,

FJ