What is the recommended cleanup.policy for schema.history.internal.kafka.topic?

[shani.kim]

Hello Team,

I’m currently running Debezium 2.5 with a MySQL connector. I understand that the database schema history topic (schema.history.internal.kafka.topic) must be configured with:

• partitions=1 (to guarantee ordering),

• retention.ms=-1 or at least 5 years (to prevent data loss).

However, I couldn’t find a clear recommendation in the documentation regarding the cleanup.policy for this history topic.

• Should it always be set to compact?

• Is it safe to leave the default (usually delete)?

• What are the potential risks of using delete versus compact for this internal topic?

I want to make sure that schema history events will never be lost and that the connector can always rebuild its schema state after a restart.

Could someone clarify the best practice for cleanup.policy on the history topic in production?

Thanks in advance!

[Chris Cranford]

Hi, the database schema history topic should not have compaction [1].

Hope that helps.
-cc

[1]: https://debezium.io/documentation/reference/stable/install.html#configuring-debezium-topics

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/5e64f422-e419-4dc8-a9f5-aea552780bd2n%40googlegroups.com.