Question about the JdbcOffsetBackingStore

[Bob Daly]

Hello!

We are considering the io.debezium.storage.jdbc.offset.JdbcOffsetBackingStore for debezium.source.offset.storage, running Debezium Server w/ PostgreSQL connector (as well as the k8s operator).

After a bit of testing, I couldn't determine an obvious pattern in how often the offset is updated - sometimes it seemed immediate in response to write events on monitored tables, but other times no update for many minutes or longer (despite write events continuing). 

Is offset.flush.interval.ms not relevant when using the JdbcOffsetBackingStore?  And if not, is there any control on the update cycle?

Thank you!

[Chris Cranford]

Hi -

Offsets flush based on the `debezium.source.offset.flush.interval.ms` in Debezium Server. 

However, what's important is that the connector must be emitting events to your target platform for the interval to expire and trigger flushes. If you have periods of where there is low or no activity on any captured table, the offsets won't flush as regularly. If you take a look at this issue [1], there is a recent change that will be in Debezium 3.6 to make these flush operations more consistent.

In the meantime, the best solution is to enable `debezium.source.heartbeat.action.query` and `debezium.source.heartbeat.interval.ms` to guarantee there is periodic traffic from the source database.

-cc

[1]: https://github.com/debezium/dbz/issues/571

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/043c02e4-92f7-420d-9344-14397e8e5051n%40googlegroups.com.