Running JDBC Sink connector from latest offset setting

[Mukesh Kumar]

Hi,

I am going to create a JDBC sink connector which will read data from kafka to Postgres Tables. Just wanted to know if there is any setting which will sink records from  from kafka topics which "populated" after the connector creation, not the historical data  (if any data exist) from kafka. 

Regards
Mukesh

[Chris Cranford]

Hi Mukesh -

This isn't Debezium specific, but instead has to do with how the sink connector is configured to read from the topic using Kafka Connect's configuration option, "auto.offset.reset". When this is set to "earliest", it will read from the beginning of the topic, capturing all changes that currently exist in the topic. If this is set to "latest", then it will only begin to consume new events after the connector's creation.  Is it possible you've set this to "latest" rather than "earliest"?  

Thanks,
-cc

Mukesh --
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/9ec5c2e1-c344-43f2-8466-2f181d64faaan%40googlegroups.com.