Replication works to RabbitMQ but ignored to Kafka

[Brent Herridge]

I started Debezium, and then updated records in the database to verify Debezium picks it up. 

Log entries and Offset Tool both showed that nothing happened after I updated the rows. I tried this with a single updated row, and then with over 200 updated rows.

I then switched to RabbitMQ as my sink, and it worked as expected. The records flowed through immediately. Oddly, logs said "4 records sent" yet only a single message was in Rabbit, which matched the single row that was updated.

Here's my Rabbit and Kafka configs:

# Sink connector config - Kafka
debezium.sink.type=kafka
debezium.sink.kafka.producer.bootstrap.servers=the-whole-server-list
debezium.sink.kafka.producer.key.serializer=org.apache.kafka.common.serialization.StringSerializer
debezium.sink.kafka.producer.value.serializer=org.apache.kafka.common.serialization.StringSerializer
debezium.sink.kafka.producer.sasl.mechanism=GSSAPI
debezium.sink.kafka.producer.security.protocol=SASL_SSL
debezium.sink.kafka.producer.ssl.truststore.location=/path/to/truststore.jks
debezium.sink.kafka.producer.ssl.truststore.password=ThePWD
debezium.sink.kafka.producer.ssl.truststore.type=JKS
debezium.sink.kafka.producer.enable.idempotence=false

# Sink connector config - RabbitMQ
debezium.sink.type=rabbitmq
debezium.sink.rabbitmq.connection.host=RabbitHost
debezium.sink.rabbitmq.connection.port=5682
debezium.sink.rabbitmq.connection.username=RMQUser
debezium.sink.rabbitmq.connection.password=RMQPwd
debezium.sink.rabbitmq.connection.virtual.host=/
debezium.sink.rabbitmq.ackTimeout=3000
debezium.sink.rabbitmq.exchange=flex-cdc
# Autocreate creates the Q, but does not bind it to the exchange. Once I manually bound it, it routed properly.
debezium.sink.rabbitmq.autoCreateRoutingKey=true
debezium.sink.rabbitmq.routingKeyFromTopicName=true

[jiri.p...@gmail.com]

Hi,

could you please share the log for Kafka?

WRT RabbitMQ, maybe you saw https://debezium.io/documentation/reference/3.0/connectors/mysql.html#mysql-schema-change-topic

Jiri

[Brent Herridge]

Today i deleted the offset file to start over from scratch. After doing that, the process picked up the changes. The only odd thing I noticed this time was that when I updated just a single record in each table, it took nearly 20 minutes for Debezium to pick it up and replicate it to Kafka. The instances feeding RabbitMQ picked them up immediately.

The log entries showing how many records were affected were wildly off. But the target Q and Topics seemed to have the right number.