Debezium Server with Pub/Sub Sink: TimeoutException - Futures stuck in PENDING state after 30 seconds

[pramod choudhari]

Hi Debezium Community,

I'm experiencing timeout issues with Debezium Server publishing to Google Cloud Pub/Sub. The engine fails with a TimeoutException where some Pub/Sub publish futures remain in PENDING state after 30 seconds.

Issues #1:

The Debezium Engine fails with below error:

{"timestamp":"2025-10-08T16:37:41.171319652Z",

"sequence":246696,

"loggerClassName":"org.slf4j.impl.Slf4jLogger",

"loggerName":"io.debezium.embedded.async.AsyncEmbeddedEngine",

"level":"ERROR","message":"Engine has failed with ",

"threadName":"pool-7-thread-1",

"threadId":35,

"mdc":{},

"ndc":"",

"hostName":"debezium-cdc-server-production-db2-85976cd8cb-zwxwh",

"processName":"io.debezium.server.Main",

"processId":1,

"exception":

{"refId":1,

"exceptionType":"java.util.concurrent.ExecutionException",

"message":"io.debezium.DebeziumException: java.util.concurrent.TimeoutException: Waited 30000 milliseconds (plus 133188 nanoseconds delay) for ListFuture18e3d19d[status=PENDING,....

...

}

}

This causes the entire engine to fail and stop. and k8s pod restarts because of this 

**PostgreSQL Connector:**

{

"connector.class": "io.debezium.connector.postgresql.PostgresConnector",

"plugin.name": "pgoutput",

"snapshot.mode": "no_data",

"max.batch.size": 4096,

"max.queue.size": 16384,

"poll.interval.ms": 500,

"offset.flush.interval.ms": 1000

}

**Pub/Sub Sink:**

{

"sink.type": "pubsub",

"pubsub.project.id": "fintech-app",

"pubsub.ordering.enabled": true,

"pubsub.batch.delay.threshold.ms": 500,

"pubsub.batch.element.count.threshold": 1000,

"pubsub.batch.request.byte.threshold": 10000000

}

## Environment

- Debezium Server running in Kubernetes (production)

- High-throughput PostgreSQL database[20TB]

- Multiple schemas with large table lists

- Google Cloud Pub/Sub with ordering keys enabled

Question:

Is there a way to increase the 30-second timeout[Or Any other solution to fix this pod restart]? Is there a Debezium Server configuration property to control this wait time?

Issue #2:
Also, I am facing another issues[Slow Startup]

After each restart, Debezium Server takes ~1 hr before it starts consuming messages. During this time, it appears to be fetching metadata from PostgreSQL about the relevant tables.

# Key Details:

117 schemas in the include list

16 table names to monitor across all schemas (so potentially up to ~1,872 table combinations)

Column-level filtering configured for each table

snapshot.mode: "no_data" - no initial snapshot, streaming only

publication.autoconfigure.mode: "disabled" - using pre-configured publication

Questions:

Is this startup delay normal for large databases?

----------------

Both issues are causing significant production challenges, the timeout causes crashes and the slow startup creates long downtime windows during restarts.

Any guidance on understanding and fixing this would be extremely helpful!

Thanks!

[Nathan Smit]

hey pramod,

On your first issue: this 30000 limit comes from this "hard" limit:  debezium.sink.pubsub.wait.message.delivery.timeout.ms

You can find all the configs and timeouts here:  https://debezium.io/documentation/reference/stable/operations/debezium-server.html#_google_cloud_pubsub I believe a user added this because they were having some issue where pubsub would get stuck but I've never experienced it.  In our environments we always just set it to some arbitrarily high number.

On your second issue, how many tables are there in your database overall?  Unfortunately I don't know enough about working with Postgres to help on this one but that does seem like a long time to wait.  It would help also to know what DBZ version you're on.

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/7c6900f5-8c19-4cd2-910c-bcec3b47c1f0n%40googlegroups.com.

--

Nathan Smit nath...@pepkor-nextech.com www.pepkor-nextech.com Creating Value - Simplified

_{This email and all contents are subject to the following disclaimer: https://www.pepkor.co.za/email-disclaimer}

_{IMPORTANT: This electronic message may contain information that is considered confidential. If received in error, please notify the sender immediately and then delete the original and all subsequent/related messages.}

[pramod choudhari]

Hi Nathan,
Thank you so much for the quick response,

Configuration Changes:

I will be updating the below configs with higher timeouts[Just to be on safe side],
debezium.sink.pubsub.wait.message.delivery.timeout.ms = 120000 (120 seconds)
debezium.sink.pubsub.retry.total.timeout.ms = 240000 (240 seconds)

Env Details:

- Debezium Server Version: 3.0.7.Final (quay.io/debezium/server)

- Postgres Version: 17.5

we have a total of 120+ schemas, total data size is 20TB across for the DB.