Debezium with Citus Postgres on multiple machines

[Andrea Marchi]

We are trying to add a CDC system to our Citus database to dump data into another sink.

However we noticed that running a normal configuration, using the postgres plugin, no changes are streamed (the snapshot works, the stream doesn't).

Is it possible to achieve this kind of CDC for systems such as Citus or Greenplub?

Do you have any example of the configuration that would be suitable?

Thank you!

[Chris Cranford]

That's hard to say Andrea without more information.

What connector configuration are you using?
Can you share your logs of the snapshot & streaming?

Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/b05068cd-3874-42c1-b9a1-26a6da13f193n%40googlegroups.com.

[Andrea Marchi]

Hi Chris,

I tried "wal2json" and "connector.class": "io.debezium.connector.postgresql.PostgreConnector". Streaming only, no snapshot

The Postgres extension (pg_partman compatible) I use is Citus: https://www.citusdata.com/

My current set up is

• I have 6 postgres Citus-worker instances and 1 master, and I set replication to 3
• I insert a new record in the master
• The debezium configuration is pointing to the master

Something I thought about experimenting is, creating a debezium connection to each one of the workers, but my suspect is that, having the replication set to 3, I would receive the same message 3 times (if the WAL is read once per replica)

I'll try to find some logs I can share.

Thank you