Kafka Connect Replicator not writing messages to the Destination replcia topic

580 views
Skip to first unread message

Krishna Marupudi

unread,
Jul 25, 2017, 5:22:29 PM7/25/17
to Confluent Platform
Hi 

I am trying to replicate a topic from source - destination. followed the documentation http://docs.confluent.io/current/connect/connect-replicator/docs/connect_replicator.html

I can see the test-topic.replica in my destination node but no data,noticed test-topic.replica topic got created in the source node and data written to it. I am not sure where i am getting this wrong

Source Node Details

zookeeper  running on 2182
kafka running on 9093

Destination Node Details

zookeeper  running on 2181
kafka running on 9092

Topic :- 
----
test-topic with replication factor 1 and partition 4

quickstart-replicator.properties

name=replicator-source                                                   
connector.class=io.confluent.connect.replicator.ReplicatorSourceConnector
tasks.max=4                                                              
                                                                         
key.converter=io.confluent.connect.replicator.util.ByteArrayConverter    
value.converter=io.confluent.connect.replicator.util.ByteArrayConverter  
                                                                         
src.kafka.bootstrap.servers=localhost:9093                               
                                                                         
src.zookeeper.connect=localhost:2182                                     
dest.zookeeper.connect=XXXXXXXXX:2181                                  
                                                                         
topic.whitelist=test-topic                                     
#topic.blacklist=                                                        
#topic.regex=                                                            
topic.rename.format=${topic}.replica             


                        
                                                                         

Ewen Cheslack-Postava

unread,
Jul 25, 2017, 5:31:11 PM7/25/17
to Confluent Platform
It looks like you have the same configuration for both src.zookeeper.connect and dest.zookeeper.connect. Given the notes on your setup I think you need to change to

src.zookeeper.connect=localhost:2182

You might also need to check your worker configuration since those configs are used by the framework to instantiate the producer that writes data into the destination cluster.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/8ce6cea1-e26a-4b79-a8f0-aeebbea5dc95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Krishna Marupudi

unread,
Jul 25, 2017, 10:08:38 PM7/25/17
to Confluent Platform
Sorry my bad it i was running the connector on the source machine instead of destination. Is this only available with the confluent licensed version ? 


To post to this group, send email to confluent...@googlegroups.com.

Ewen Cheslack-Postava

unread,
Jul 26, 2017, 2:47:18 AM7/26/17
to Confluent Platform
Replicator is a Confluent Enterprise feature. You'll get a 30 day trial period, but yes, it is a subscription feature.

-Ewen

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages