Repull message pubsub from debezium

70 views
Skip to first unread message

Rahadian Djati

unread,
Jun 28, 2022, 11:44:42 AM6/28/22
to pubsub-discuss
Hi all, we are currently testing the debezium cdc in postgres. this time the message has made it to pubsub message. but when we integrate into bigquery using dataflow, messages that are already in pubsub can't be pulled into bigquery. the incoming data is only the most recent data when we add the dataflow template. How do we repull pubsub messages to bigquery?

Eduardo Ortiz Caraveo

unread,
Jun 29, 2022, 7:25:15 PM6/29/22
to pubsub-discuss

The subscriber has a configurable, limited amount of time, known as the ackDeadline, to acknowledge the outstanding message. After the deadline passes, the message is no longer considered outstanding, and Pub/Sub attempts to redeliver the message.

You can also use synchronous pull with lease management since an individual message's processing may exceed the preconfigured acknowledgement deadline, also known as the lease. A good use case for synchronous pull with lease management is when you are processing hundreds of messages in parallel, and each message takes a long time to process.

Reply all
Reply to author
Forward
0 new messages