Debezium stops sending heartbeats when postgres is updated from 14 to 15

[shivam fialok]

Hi,

We updated postgres from 14 to 15 and we noticed that WAL started to grow upon upgrade. On inspection, I found that debezium wasn't sending heartbeats, because of which the heartbeat.action.query was also not being fired.

Then i manually fired heartbeat.action.query in the DB and I noticed the WAL usage drop, and debezium also resumed heartbeats and the issue was resolved.

Does anyone know what might be going on and how to fix this?

Let me know if any other info would be helpful.

WAL usage was only growing because of one DB which was possibly not having any updates during the day. And we have both heartbeat.interval.ms and heartbeat.action.query set on the connectors.

Thanks.

[Chris Cranford]

Hi -

If you have heartbeat.action.query and heartbeat.interval.ms configured in your configuration, there shouldn't be any change in behavior between the two database versions.  I'd check the connector logs to see if its reporting any failure with writing the heartbeat action query to the database, perhaps its permissions related?

Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/0f5051fa-b6ed-460e-b0e0-8b668091ef94n%40googlegroups.com.

[shivam fialok]

Thanks for replying.

Actually it is not sending heartbeats to the heartbeats topic as well. So, I am imagining it is problem with heartbeats themselves and not with applying heartbeat.action.query.

What do you think?

[Chris Cranford]

Hi -

Well, if the connector is unable to write to the heartbeat table or any reason; that would explain it.
Again I think the logs here may share some useful details, either the logs of the connector or the logs of the database.
I'd suggest setting the heartbeat.interval.ms to something very tiny & look at those two logs to see if you find any detail.

Thanks,
Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/c3e0c74e-d2d9-4fb8-b588-0a316a479638n%40googlegroups.com.

[shivam fialok]

Also I checked the logs and didn't see any such errors.

One thing to note is that Heartbeats are able to resume once i manually apply the heartbeat.action.query in the DB.

[shivam fialok]

Currently we have heartbeats interval set as 10s.

I will see the connector and DB logs.

Note that we are only printing INFO+ logs and not DEBUG logs.

Hopefully that would be sufficient to debug this?

[Chris Cranford]

Hi -

If there is an error happening, you shouldn't need to increase the log verbosity in general; however, if the current level isn't sufficient, the next step would be to get the connector logs with at least DEBUG logging enabled to see what may be happening during the streaming phase.

Thanks,
Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/3025bc48-af32-43d1-a578-4f2ebaf50273n%40googlegroups.com.