Unable to startup MirrorMaker on CP 2.0.0

[Stuart Wong]

Hello,

I'm trying to startup a MirrorMaker service from the CP 2.0.0 download. I'm getting the below error:

Feb 16 23:23:05 stlukafkaprd01.monsanto.com systemd[1]: Started Kafka MirrorMaker cor service.

Feb 16 23:23:06 stlukafkaprd01.monsanto.com sh[26539]: [2016-02-16 23:23:06,000] stlukafkaprd01.monsanto.com [INFO] Processing variables...

Feb 16 23:23:06 stlukafkaprd01.monsanto.com sh[26539]: [2016-02-16 23:23:06,000] stlukafkaprd01.monsanto.com [INFO] Processing variables...

Feb 16 23:23:06 stlukafkaprd01.monsanto.com systemd[1]: Stopping Kafka MirrorMaker cor service...

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: [2016-02-16 23:23:09,009] WARN block.on.buffer.full config is deprecated and will be removed soon. Please use max.block.ms (org.apache.kafka.clients.producer.KafkaProducer)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Exception in thread "mirrormaker-thread-2" org.I0Itec.zkclient.exception.ZkNodeExistsException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.utils.ZKCheckedEphemeral.create(ZkUtils.scala:1090)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$registerConsumerInZK(ZookeeperConsumerConnector.scala:272)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:1000)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:163)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerOldConsumer.init(MirrorMaker.scala:477)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:388)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: ... 6 more

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Exception in thread "mirrormaker-thread-3" org.I0Itec.zkclient.exception.ZkNodeExistsException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.utils.ZKCheckedEphemeral.create(ZkUtils.scala:1090)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$registerConsumerInZK(ZookeeperConsumerConnector.scala:272)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:1000)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:163)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerOldConsumer.init(MirrorMaker.scala:477)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:388)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: ... 6 more

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Exception in thread "mirrormaker-thread-1" org.I0Itec.zkclient.exception.ZkNodeExistsException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:55)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.utils.ZKCheckedEphemeral.create(ZkUtils.scala:1090)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$registerConsumerInZK(ZookeeperConsumerConnector.scala:272)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector$WildcardStreamsHandler.<init>(ZookeeperConsumerConnector.scala:1000)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:163)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerOldConsumer.init(MirrorMaker.scala:477)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:388)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)

Feb 16 23:23:09 stlukafkaprd01.monsanto.com sh[26539]: ... 6 more

The consumer and producer configurations are below respectively:

consumer.properties:

auto.offset.reset=largest

client.id=mirrormaker-cor-1

consumer.id=mirrormaker-cor-1

group.id=mirrormaker-cor-1

offsets.storage=kafka

zookeeper.connect=zk01.company.com:2181,zk02.company.com:2181,zk03.company.com:2181,zk04.company.com:2181,zk05.company.com:2181/kafka/prd

producer.properties:

acks=all

client.id=mirrormaker-cor-1

compression.type=snappy

bootstrap.servers=kafka1.internal:9092,kafka2.internal:9092,kafka3.internal:9092

Any thoughts on why this error is occuring and how I get around it? Really appreciate any insights.

Thanks.

[万浩然]

Hi,

 Did you manage it later?

 in my case I remove the consumer.id=...  and that works.

 but notice the exception line number is different from mine (nevertheless the method is the same), I wonder if your  kafka version is another one than 0.10.0. that is what I use.

在 2016年2月17日星期三 UTC+8上午7:27:32，Stuart Wong写道：