--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/ab7d7541-f6c6-4734-857f-57d9873b5790n%40googlegroups.com.
Thank you for the quick response. We are already testing the fix you proposed. We began this project with version 3.0.0, but we switched to 3.4.0 prior to starting our performance tests.
I have another question regarding the low watermark update. It appears to be tied to the oldest transaction's start SCN. This maintains a larger reading window, and long-running transactions force this window to remain wide.
Since changes are buffered in memory (or via Infinispan), why is it necessary to anchor the start SCN to the beginning of the oldest transaction?
Now, not all or in some cases none of these corner cases were seen by a subset of users.
There is currently a PR in the works [2] that introduces a new windowing technique that aims to avoid long running transactions having the same impact on mining like they currently do; however, the new algorithm has the potential to reintroduce the above corner cases. So this feature is purely opt-in and the risks are documented that users should be aware.
Unfortunately, this is all based on LogMiner limitations, so we're trying to provide solutions for environments that need absolute data reliability versus others that need higher performance at the risk of data reliability.
Let us know if you have any questions.
-cc
To view this discussion visit https://groups.google.com/d/msgid/debezium/b5942445-0f9c-423a-b897-e39e66004f71n%40googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/61db8ee5-cfb9-4f34-b1ee-300df0cf65c1n%40googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/f4ab6e64-42a2-49e6-b61c-25f2d47ae5e7n%40googlegroups.com.
Hi Chris,
I will follow up tomorrow morning once our DBA provides more feedback.
In the meantime, I’ve received a log analysis from my colleagues with the following definitions:
Setup(s): The duration of the start mining session calls.
Batch SCN: The size of the query window.
Window SCN: The total size of the mining session window.
My observations:
Fetch speed: Data fetching seems quite slow, especially considering we are on optimized hardware (Exadata).
Scaling issues: It is clearly visible that query times increase as the Window SCN grows, likely due to long-running transactions. When the window exceeds log.mining.batch.size.max, I see the mining session starting from the same SCN repeatedly.
You mentioned in a previous comment that you know of several very large databases (millions of changes per hour) using a single connector successfully.
Could you share some specific metrics on achievable performance with LogMiner? I’m particularly interested in TPS and redo log volume/sec.
I’m trying to understand how close we are to the architectural limits, as we will eventually need to mine significantly more data in production (approx. 100–200 MB/s).
73 | 2026-02-04 13:17:12.692 | 0,006 | 25,479 | 25,485 | 114 784 | 129 284 | 22 661 | 2 | OKWhat I would do is take these timings, and have the DBA share the alert logs with you, if possible, and see what might have been happening at the times you see spikes. In some environments, I've seen similar scenarios where LogMiner is reading the archive logs just fine, and then all of a sudden the read of an archive log takes 2, 4, or 10x longer than it had been. In the cases we saw, this was due to materialized view refreshes, which creates a tremendous amount of contention on the Oracle redo system. Do you have something similar?
74 | 2026-02-04 13:17:41.206 | 0,006 | 41,865 | 41,871 | 154 088 | 283 371 | 27 177 | 7 | OK
75 | 2026-02-04 13:18:26.106 | 0,006 | 71,925 | 71,931 | 200 000 | 483 370 | 45 077 | 2 | OK
To view this discussion visit https://groups.google.com/d/msgid/debezium/cf2d15ff-58bb-410c-8c50-e93efacec71fn%40googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/99843d7b-e3fe-43c4-b8be-b2f394be40d8n%40googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/9862f64d-adda-4c31-9130-aff196c546a3n%40googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/3f9f57ac-6d9c-4abc-86cd-774023cdcab7n%40googlegroups.com.