Incremental Snapshot via KafkaSignalChannel with PostgreSQLConnector

[Abubakr Mirzoev]

Hi
I want to get incremental snapshot of one existing table by sending message to the kafka topic (KafkaChannelSignal approach)
Here is my connector configs:

{

"connector.class": "io.debezium.connector.postgresql.PostgresConnector",

"topic.creation.default.partitions": "1",

"slot.name": "cdc_mobidb_test",

"tasks.max": "1",

"publication.name": "mobi_cdc_publication",

"include.schema.changes": "true",

"tombstones.on.delete": "false",

"topic.prefix": "mobidb_test",

"topic.creation.default.replication.factor": "2",

"database.user": "debezium_user",

"database.dbname": "mobidb",

"topic.creation.default.compression.type": "lz4",

"topic.creation.default.cleanup.policy": "delete",

"incremental.snapshot.enabled": "true",

"database.server.name": "mobidb",

"snapshot.max.threads": "4",

"database.port": "5432",

"plugin.name": "pgoutput",

"database.hostname": "tmp-cdc-debezium-kafka.amazonaws.com",

"database.password": "****",

"name": "mobi_connector",

"table.include.list": "public.user_config,public.orders,public.debezium_signal", 

"signal.enabled.channels": "kafka", 

"signal.kafka.topic": "debezium-signals",

"signal.data.collection": "public.debezium_signal", 

"signal.kafka.bootstrap.servers": "ip:9092", 

} 

When I send message execute-snapshot for orders table to the debezium-signals topic I receive logs like:
 
[2024-06-05 08:57:00,274] INFO [mobi_connector|task-0] Requested 'INCREMENTAL' snapshot of data collections '[public.orders]' with additional conditions '[]' and surrogate key 'PK of table will be used' (io.debezium.pipeline.signal.actions.snapshotting.ExecuteSnapshot:64)
[2024-06-05 08:57:01,274] INFO [mobi_connector|task-0] Incremental snapshot for table 'public.orders' will end at position [129873] (io.debezium.pipeline.source.snapshot.incremental.AbstractIncrementalSnapshotChangeEventSource:381)

Nothing happens after that.
The orders table size is 50MB

[Chris Cranford]

Hi -

There have been a few regressions recently with incremental snapshots starting prematurely.  Can you please try, if you are not already, with Debezium 2.6.2.Final.

Thanks,
Chris

Nothing happens after that.
The orders table size is 50MB --
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/5c442b09-3a80-4577-ad9b-34d32b16cd36n%40googlegroups.com.