kafka.common.ConsumerRebalanceFailedException ... Can't rebalance after retries

[tony....@vungle.com]

Hi,

We built an older version of secor and it was working perfectly. We are trying to update to the latest code base with the latest configs to take advantage of some of the s3 timestamp granularity features but continuously met with this error:

2015-08-11 02:45:09,761 [Thread-5] (com.pinterest.secor.main.ConsumerMain$1:65) ERROR Thread Thread[Thread-5,5,main] failed
java.lang.RuntimeException: Failed to initialize the consumer
at com.pinterest.secor.consumer.Consumer.run(Consumer.java:85)
Caused by: kafka.common.ConsumerRebalanceFailedException: secor_backup_secor-directattributionraw01_15669_14 can't rebalance after 5 retries
at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:633)
at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:905)
at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:939)
at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:160)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:101)
at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:108)
at com.pinterest.secor.reader.MessageReader.<init>(MessageReader.java:70)
at com.pinterest.secor.consumer.Consumer.init(Consumer.java:68)
at com.pinterest.secor.consumer.Consumer.run(Consumer.java:83)

We are on kafka 0.8.2.1 and everything appears to be working fine on the kafka side.

Thanks for your help.

Tony

[Pawel Garbacki]

Hi Tony,

thanks for reporting it. Could you narrow this down to a specific code version? We should notify the author of the offending PR.

Thanks,

-pawel

--
You received this message because you are subscribed to the Google Groups "secor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to secor-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[tony....@vungle.com]

Just to follow up, turns out I had to increase the

zookeeper.session.timeout.ms=

to a bigger number.