'no brokers found when trying to rebalance'

[Neil Walsh]

Hello,

We use znodes in zookeeper and have one configured on kafka1. 

My secor.prod.properties looks like this (it's not clear if I need to add the port onto the kafka.seed.broker.host option)

# Name of one (random) Kafka broker host that is used to retrieve metadata.
# TODO(pawel): use a list of nodes or even better, extract active brokers from zookeeper.
kafka.seed.broker.host=$kafka_host


# List of Kafka Zookeeper servers with ports <zookeeper-address>:<zookeeper-port>.
zookeeper.quorum=$zk_host_1;2181,$zk_host_2:2181,$zk_host_3:2181

When I start secor, I get the following error

2017-07-25 21:49:13,750 [Thread-8] (kafka.utils.Logging$class:85) WARN  [$hostname_27804_17], no brokers found when trying to rebalance.

If I add /kafka1 to the zookeeper.quorum option 

# List of Kafka Zookeeper servers with ports <zookeeper-address>:<zookeeper-port>.
zookeeper.quorum=$zk_host_1;2181,$zk_host_2:2181,$zk_host_3:2181/kafka1

the error changes to

java.lang.RuntimeException: Failed to initialize the consumer
 at com.pinterest.secor.consumer.Consumer.run(Consumer.java:90)
Caused by: java.lang.IllegalArgumentException: Path must not end with / character

Does anyone know how to resolve this?

Thanks for any help.

Neil

[hc...@pinterest.com]

What's the full stack trace when you have 'XXX/kafka1' as quorum?

[Neil Walsh]

With quorum set as follows:

zookeeper.quorum=$zk_host_1;2181,$zk_host_2:2181,$zk_host_3:2181/kafka1

the full stacktrace is

2017-07-28 15:33:45,337 [Thread-3] (com.pinterest.secor.main.ConsumerMain$1:65) ERROR Thread Thread[Thread-3,5,main] failed
java.lang.RuntimeException: Failed to initialize the consumer
 at com.pinterest.secor.consumer.Consumer.run(Consumer.java:90)

Caused by: java.lang.IllegalArgumentException: Path must not end with /

character
 at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:58)
 at org.apache.zookeeper.client.ConnectStringParser.<init>(ConnectStringParser.java:57)
 at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:443)
 at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
 at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:70)
 at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1228)
 at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:157)
 at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:131)
 at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:79)
 at kafka.utils.ZkUtils$.apply(ZkUtils.scala:61)
 at kafka.consumer.ZookeeperConsumerConnector.connectZk(ZookeeperConsumerConnector.scala:192)
 at kafka.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:140)
 at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:66)
 at kafka.javaapi.consumer.ZookeeperConsumerConnector.<init>(ZookeeperConsumerConnector.scala:69)
 at kafka.consumer.Consumer$.createJavaConsumerConnector(ConsumerConnector.scala:120)
 at kafka.consumer.Consumer.createJavaConsumerConnector(ConsumerConnector.scala)
 at com.pinterest.secor.reader.MessageReader.<init>(MessageReader.java:69)
 at com.pinterest.secor.consumer.Consumer.init(Consumer.java:69)
 at com.pinterest.secor.consumer.Consumer.run(Consumer.java:88)

This is how I am running it too

java -ea -Dsecor_group=secor_backup   -Dlog4j.configuration=log4j.prod.properties   -Dconfig=secor.prod.backup.properties   -cp secor-0.25-SNAPSHOT.jar:lib/*   com.pinterest.secor.main.ConsumerMain

Thanks,

Neil