Kafka connect hangs...

[Mahesh Vedantam]

Hi,

Kakfa connect & debezium hangs & not responding when received the transaction logs of about weeks data. Is there anyway to over come this? Is this related snapshots or offset pointers, please let me know. Please let me know how to troubleshoot.

Thanks,

UMV

[Chris Cranford]

Hi -

I believe you're going to need to share quite a bit more detail, such as:

    - Debezium version
    - Kafka Connect version
    - What is your source database
    - What is the amount of a week's transaction logs of data? 
    - What is the maximum heap you've given Kafka Connect?

Perhaps the issue is that you're consuming the maximum memory available to the process and the JVM is constantly firing garbage collection?

Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CAFxchOqKH%2BLk90X_Y_CJCX48KSSb7qZRNPb2rnj41rrkznACNw%40mail.gmail.com.

[Mahesh Vedantam]

    - Debezium version

2.6.1
    - Kafka Connect version

3.7.0

    - What is your source database

postgres

    - What is the amount of a week's transaction logs of data? 

Not sure, but i was showing committed/ack 16000 messages

    - What is the maximum heap you've given Kafka Connect?

Default.

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/824c257a-eb59-4b0f-b754-e422b33af85a%40gmail.com.

[Chris Cranford]

Hi -

Is it possible the connector is traversing the WAL and gives the impression that it is stuck?  Would you be able to share the full connector logs, preferably setting the log level to DEBUG for io.debezium.

Thanks,
Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CAFxchOpKGg6za55oJn20%2BeKo5Pq4bggDENb1pyvWFCu3mVXQ0g%40mail.gmail.com.