Debezium Server - Memory Consumption

[Adam Whitmore]

Is it expected that memory consumption would differ when changing snapshot mode from initial to schema_only?

We are running Debezium Server 1.6.2.Final with SQL Server connector and seeing memory consumption climb when setting snapshot mode to schema_only, particularly when processing large transactions.

I've run tests with one particular table, inserting 1.5M records in one transaction, which resulted in no increase in memory consumption when snapshot mode is set to initial.  Running that same transaction on an instance with snapshot mode set to schema_only results in an increase in memory consumption of around 200 MiB.

The increase in memory consumption is there with smaller transactions,  too, but to a lesser degree.  The increase was 70 MiB when inserting 1.5M records in 10 separate transactions of 150K records.

Thanks,
Adam

[jiri.p...@gmail.com]

Hi,

thise should not be the case. The snapshot phase should consume more memory for initial snapshot but then the consumption should be more or less same. Could you please share some charts containing multiple GC runs?

Thanks

J.

[Adam Whitmore]

Here's what I have from the runs I've recorded so far:

[Adam Whitmore]

Adding to this - also ran a test with v1.8.0.Final today in schema_only mode, had a similar increase from 1.52 GiB to 1.73 GiB when processing a 1.5M record insert transaction.