Kafka mirrormaker is not working, seems the process is running but is doing nothing

[Koen Vantomme]

Hello,
Since a few days I'm trying to figure out why my mirriormaker is not working.
I work on kafka 0.8.2 .

We are not yet working on 0.9 because there are issues with spark integration for 1.6

My mirrormaker command :

./bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config config/consumer_mirror1.properties --producer.config config/producer_mirror1.properties --whitelist mirrorSource20160808_3

The output :
[root@comlkd13 kafka_2.11-0.8.2.2]# ./bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config config/consumer_mirror1.properties --producer.config config/producer_mirror1.properties --whitelist mirrorSource20160808_3
[2016-08-08 08:23:16,078] WARN Property consumer.timeout is not valid (kafka.utils.VerifiableProperties)
[2016-08-08 08:23:16,283] WARN Property bootstrap.servers is not valid (kafka.utils.VerifiableProperties)
[2016-08-08 08:23:16,283] WARN Property queue.time is not valid (kafka.utils.VerifiableProperties)

(Fixing the warnings is easy)

config file for mirrorconsumer :

zookeeper.connect=xxx1:2181,xxx2:2181,xxx3:2181

zookeeper.connection.timeout.ms=100000

group.id=test-mirror-group-1

consumer.timeout=-1

auto.offset.reset=smallest

config file for mirrorproducer :

metadata.broker.list=comlkd13.us.db.com:9092,comlkd13.us.db.com:9093,comlkd23.us.db.com:9092,comlkd23.us.db.com:9093
bootstrap.servers=comlkd13.us.db.com:9092,comlkd13.us.db.com:9093,comlkd23.us.db.com:9092,comlkd23.us.db.com:9093

producer.type=async

compression.codec=none

serializer.class=kafka.serializer.DefaultEncoder

queue.enqueue.timeout.ms=-1
queue.time=1000000

# the number of messages batched at the producer
#batch.num.messages

queue.time=1000000

There is not really much more log information  I can find at the moment.

Maybe there are network limitations ?

Any suggestions are much appreciated  and will be very helpful . Thanks !!

[gerard...@dizzit.com]

Seems like you using a mirror maker witch defaults to using the new consumer, using bootstrap.servers instead of zookeeper connect. Because the clients are not backwards compatible, you should use a 0.8.2 mirrormaker.

[Koen Vantomme]

Hello,

If anyone is reading this message by any chance.

I reduced the log level to debug and I have now a much better overview what is happening.

The output of mirrormaker is attached and also an overview of the settings of mirrormaker.

If someone could have a look too the files and maybe give a suggestion.

If I read it correct it seems I can consume messages but that there is an issue with the producer.

Any help / suggestion much appreciated !

Op maandag 8 augustus 2016 14:31:16 UTC+2 schreef Koen Vantomme:

[gerard...@dizzit.com]

I just took a quick look, but saw your still using mirror maker version 0.9.0.1, which will not work with 0.8.2 brokers.

[Koen Vantomme]

Thanks Gerard for your reaction.
I switched my installation to kakfa 0.9.1 and I used the appropriate properties and I got it to work after a lot try and error.
Now I will swicht back to kafka 0.8.8 and get it to work in the same way.

For me the trick was I had to create the topic on the target cluster, although I have the property auto create topic enabled.

consumer.

I also changed the log level to debug so I could better understand what was happening
changing the consumer file :
zookeeper.connection.timeout.ms=1000000
consumer.timeout.ms=-1
changing the producer file:
producer.type=async
queue.enqueue.timeout.ms=-1
queue.time=1000

changing the broker properties files:
advertised.host.name=xxx (this could be overkill)
advertised.port=9092

zookeeper.connection.timeout.ms=1000000:w
auto.create.topics.enable=true
controller.socket.timeout.ms=90000
controller.message.queue.size=20
auto.leader.rebalance.enable=true
queued.max.requests=20
auto.leader.rebalance.enable=true

Once I realized that the topic was adding messages in the log files for the topic in the target system after producing messages to the consumer I realized that

 mirrormaker is working fine :)

Thanks for your support Gerard

Op maandag 8 augustus 2016 14:31:16 UTC+2 schreef Koen Vantomme:

Hello,

[Viswanathan J]

Hi Gerard/Koen,

I'm trying to implement mirrormaker using Kafka version v0.8.2, and when I'm trying to create topics it is created in target cluster. But I produce data in source cluster I'm unable to consume any data in target cluster.

Please help on this.

[Mohan Kumar]

Just want to check if you got the MM working. I am running into issues with MM 0.9 as i am unable to see the message replication on the destination cluster.

I did not see any errors during execution of MM. However the target and source cluster are not exchanging messages. Not sure if this is known behaviour.

Please share the Producer and Consumer configs and i can cross check. Thanks and Regards,

Mohan

[karan alang]

Hi I'm getting similar errors on running Mirror Maker on Kafka 0.9

Here are the details :

Any ideas on how to fix these warnings ?

$KAFKA_HOME1/bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config $KAFKA_HOME1/config/mmConsumer.config --num.streams 3 --producer.config $KAFKA_HOME1/config/mmProducer.config --whitelist="mmtopic" --abort.on.send.failure true

[2017-06-20 14:49:08,190] WARN block.on.buffer.full config is deprecated and will be removed soon. Please use max.block.ms (org.apache.kafka.clients.producer.KafkaProducer)

[2017-06-20 14:49:08,244] WARN The configuration queue.time = 1000000 was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig)

[2017-06-20 14:49:08,244] WARN The configuration queue.enqueue.timeout.ms = -1 was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig)

[2017-06-20 14:49:08,244] WARN The configuration producer.type = async was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig)

Kafka version -> 0.9

Producer config :

bootstrap.servers=localhost:7092,localhost:7093,localhost:7094,localhost:7095

producer.type=async

queue.time=1000000

queue.enqueue.timeout.ms=-1

Consumer config :

zookeeper.connect=localhost:2181

#zk.connectiontimeout.ms=1000000

zookeeper.connection.timeout.ms=1000000

#bootstrap.servers=localhost:9092,localhost:9093,localhost:9094,localhost:9095

consumer.timeout.ms=-1

#security.protocol=PLAINTEXTSASL

group.id=kafka-mirror

[Mohan]

Hi,

I cannot see the prefix setting 

In the Producer config :

bootstrap.servers=localhost:7092,localhost:7093,localhost:7094,localhost:7095

are you pointing to the IP address/host name of the Destination Kafka on which you are running Mirror maker?

In the Consumer config :

#bootstrap.servers=localhost:9092,localhost:9093,localhost:9094,localhost:9095

are you pointing to the Primary ZK IP address/host name?

Please provide me the config/server.properties config and can cross check. 

I had similar MM error messages on 0.9.0.1, when the local.host setting was set to IP address, so had to force it to localhost. Also on the Consumer and Producer configs, had bare minimum config entries ( 2 entries for the consumer (pointing to Primary ZK and GroupID config) and 1 entry on the producer pointing to the destination Kafka ) and got it working. I was able to get large DB tables backed up on the destination Kafka and it works great now. 

Also, try running the MM command with the wildcard ".*' on the topic. 

Hope this helps.

Thanks and Regards,

Mohan

--
You received this message because you are subscribed to a topic in the Google Groups "Confluent Platform" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/confluent-platform/tX2WWUF8o20/unsubscribe.
To unsubscribe from this group and all its topics, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/2afecd02-141e-442e-81a3-ec83a903b90d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks and Regards,
Mohan