incremental snapshot prod

Vladislav P

unread,

Nov 7, 2025, 2:52:06 AM (6 days ago) Nov 7

to debezium

Hi. I'm using incremental snapshots, and everything works fine on the test bench. But as soon as I run them on PROD, I start getting errors:

cannot open archived log
it takes a very long time to read information
If I enable snapshot.mod: "initial", it reads all the data without any errors with the logs. (The only issue is ORA-01555 on large tables).

How can I properly work with incremental snapshots on the PROD database? I asked about clearing the log, and they told me that it happens twice a day.

Logs error1:
```
2025-11-05T20:14:05,125 INFO Oracle||streaming Streaming metrics at shutdown: LogMinerStreamingChangeEventSourceMetrics{connectorConfig=io.debezium.connector.oracle.OracleConnectorConfig@6863b9b3, startTime=2025-11-05T14:27:49.903421226Z, clock=SystemClock[Z], currentScn=6324525349055, offsetScn=6324135244612, commitScn=6324140546520, oldestScn=6324135244613, oldestScnTime=2025-11-05T15:08:17.143Z, currentLogFileNames=[+FRA/RAC_RCDB/redo_1_05.log, +FRA/RAC_RCDB/redo_2_13.log], redoLogStatuses=[+FRA/RAC_RCDB/redo_2_12.log | ACTIVE, +FRA/RAC_RCDB/redo_1_04.log | ACTIVE, +FRA/RAC_RCDB/redo_2_13.log | CURRENT, +FRA/RAC_RCDB/redo_1_05.log | CURRENT, +FRA/RAC_RCDB/redo_2_11.log | INACTIVE, +FRA/RAC_RCDB/redo_1_02.log | INACTIVE, +FRA/RAC_RCDB/redo_1_01.log | INACTIVE, +FRA/RAC_RCDB/redo_2_14.log | INACTIVE, +FRA/RAC_RCDB/redo_1_03.log | INACTIVE, +FRA/RAC_RCDB/redo_2_08.log | INACTIVE, +FRA/RAC_RCDB/redo_2_15.log | INACTIVE], databaseZoneOffset=+03:00, batchSize=100000, logSwitchCount=462, logMinerQueryCount=479, sleepTime=1000, minimumLogsMined=2, maximumLogsMined=96, maxBatchProcessingThroughput=24056, timeDifference=-48858, processedRowsCount=405491566, activeTransactionCount=18, rolledBackTransactionCount=755245, oversizedTransactionCount=0, changesCount=32113258, scnFreezeCount=1, batchProcessingDuration={min=PT2.24936652S,max=PT1M38.270860463S,total=PT5H38M2.803800116S}, fetchQueryDuration={min=PT0.951903371S,max=PT4.144294881S,total=PT16M32.602922222S}, commitDuration={min=PT0.000000556S,max=PT34.15229874S,total=PT1M34.496830578S}, lagFromSourceDuration={min=PT7.494671792S,max=PT5H4M39.908129806S,total=PT83328813H28M12.189646289S}, miningSessionStartupDuration={min=PT0.002943306S,max=PT0.009020702S,total=PT2.144448743S}, parseTimeDuration={min=PT0.00001856S,max=PT0.018188603S,total=PT10M6.977323698S}, resultSetNextDuration={min=PT0.000000078S,max=PT6.382167955S,total=PT4H36M48.481456134S}, userGlobalAreaMemory={value=68654640,max=93036440}, processGlobalAreaMemory={value=75359232,max=139453440}, abandonedTransactionIds=[], rolledBackTransactionIds=[220111004e561b01, 14000a009c76f101, 110015008a831a03, 4b001b00dbbc1901, 390020006eeb7e01, 42000400534a0101, 20010c009f7e1f01, 2c00070099ec9501, 0d000e00a4a4be01, 26010300f3840b01], lastMiningSessionScnRange=[6324135244612, 6324140691197], lastMiningFetchScnRange=[6324135244612, 6324140691197]} [io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource]
2025-11-05T20:14:05,151 INFO Oracle||streaming Offsets as shutdown: OracleOffsetContext [scn=6324135244612, txId=e200030076e6d600, txSeq=1, commit_scn=["6324140546478:1:e200030076e6d600","6324140546520:2:75001e00fcf15201"], lcr_position=null] [io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource]
2025-11-05T20:14:05,153 INFO Oracle||streaming Finished streaming [io.debezium.pipeline.ChangeEventSourceCoordinator]
2025-11-05T20:14:05,153 INFO Oracle||streaming Connected metrics set to 'false' [io.debezium.pipeline.ChangeEventSourceCoordinator]
2025-11-05T20:14:05,333 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 1794 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-05T20:14:05,339 ERROR || WorkerSourceTask{id=wpms-source-connector-avro-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted [org.apache.kafka.connect.runtime.WorkerTask]
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:67) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:217) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute
nerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:330) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:208) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:148) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at io.debezium.util.Threads$3.lambda$newThread$0(Threads.java:314) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: java.sql.SQLException: ORA-00308: cannot open archived log '+FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599'
ORA-17503: ksfdopn:2 Failed to open file +FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599
ORA-15012: ASM file '+FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599' does not exist

at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:630) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:564) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1231) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:772) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:299) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:512) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:163) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:1010) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.prepareDefineBufferAndExecute(OracleStatement.java:1271) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1149) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.executeSQLSelect(OracleStatement.java:1661) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1470) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3761) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3936) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1102) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.executeAndProcessQuery(AbstractLogMinerStreamingChangeEventSource.java:394) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.process(BufferedLogMinerStreamingChangeEventSource.java:245) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:156) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
... 10 more
Caused by: oracle.jdbc.OracleDatabaseException: ORA-00308: cannot open archived log '+FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599'
ORA-17503: ksfdopn:2 Failed to open file +FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599
ORA-15012: ASM file '+FRA/RCDB/ARCHIVELOG/2025_11_05/thread_1_seq_514810.632.1216404599' does not exist

at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:637) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:564) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1231) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:772) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:299) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:512) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:163) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:1010) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.prepareDefineBufferAndExecute(OracleStatement.java:1271) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1149) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.executeSQLSelect(OracleStatement.java:1661) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1470) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3761) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3936) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1102) ~[ojdbc11-21.15.0.0.jar:21.15.0.0.0]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.executeAndProcessQuery(AbstractLogMinerStreamingChangeEventSource.java:394) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.process(BufferedLogMinerStreamingChangeEventSource.java:245) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:156) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
... 10 more
2025-11-05T20:14:05,342 INFO || Stopping down connector [io.debezium.connector.common.BaseSourceTask]
2025-11-05T20:14:05,345 INFO || SignalProcessor stopped [io.debezium.pipeline.signal.SignalProcessor](AbstractLogMi

```

Logs error2:
```

2025-11-06T08:06:14,558 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 4 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-06T08:06:15,096 INFO || [AdminClient clientId=connector-adminclient-wpms-source-connector-avro-0] Rebootstrapping with Cluster(id = null, nodes = [apache-kafka:19092 (id: -1 rack: null isFenced: false)], partitions = [], controller = null) [org.apache.kafka.clients.admin.internals.AdminMetadataManager]
2025-11-06T08:06:42,960 INFO Oracle||streaming Redo Thread 2 is inconsistent; failed to find log with sequence 439680 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 2 is inconsistent; failed to find log with sequence 439680
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:330) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:208) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:148) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at io.debezium.util.Threads$3.lambda$newThread$0(Threads.java:314) ~[debezium-core-3.4.0.Alpha1.jar:3.4.0.Alpha1]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-06T08:06:42,962 INFO Oracle||streaming No logs available yet (attempt 1)... [io.debezium.connector.oracle.logminer.LogFileCollector]

```

Chris Cranford

unread,

Nov 10, 2025, 9:38:22 AM (3 days ago) Nov 10

to debe...@googlegroups.com

Hi,

If we look at the "Streaming Metrics" entry here from the logs, there is quite some useful information.

* Connector has been up for approximately 5 hours, 41 minutes.
* Gap between CurrentScn (database write position) and OffsetScn (connector resume position) is 390M.
* Gap between CurrentScn (database write position) and CommitScn (last transaction that was seen as committed) is 385M.
* However you have an in-progress transaction that started at 15:08, it's now 20:04, so it's been active for over 5 hours <=== this is the problem

If you have LOB enabled, you should really focus on addressing why you have long-running transactions like this. Unfortunately when LOB is enabled, the connector will continue to read from 15:08 onward until that transaction commits or is rolled back.

If you do not have LOB enabled (the default), then you're in luck, as DBZ-8747 [1] introduced a performance regression that under this exact scenario would cause the connector's performance to potentially suffer. We're currently actively working on a fix in DBZ-9664 [2] to address fixing the regression. You can track the progress of the pull request [3] if you'd like to be notified when the change is merged.

Do you have `lob.enabled` set to `false` (the default) ?

[1]: https://issues.redhat.com/browse/DBZ-8747
[2]: https://issues.redhat.com/browse/DBZ-9664
[3]: https://github.com/debezium/debezium/pull/6880

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/debezium/4244ee4b-d605-46c6-b0a9-ded69fdfd256n%40googlegroups.com.

Chris Cranford

unread,

Nov 10, 2025, 9:45:43 AM (3 days ago) Nov 10

to debe...@googlegroups.com

I should have also included the following.

Because you have a long running transaction since 15:08, even with the DBZ-9664, that won't change the fact that the OffsetScn will continue to point to that log position, so if the connector is restarted due to some other issue such as a rebalance or reconfiguration, you will find that the SCN is no longer available. This is because on restarts, we must resume reading from the eldest in-progress transaction when the connector stopped, regardless of how the connector manages the transient state while it's active.

You have a couple options here to remedy the issue:

1. Make sure the database sets idle connection timeouts if this was simply an idle user transaction.
2. Enable `log.mining.transaction.retention.ms` to discard transactions that do not complete after the specified time window.

With (1), the database will automatically terminate the user session and rollback the transaction. This is good as it will keep the replicated state aligned with the database state. However with (2), you potentially risk the transaction eventually being committed on the source database, but not replicated because it is discarded.

But in either case, it's important that the transaction retention or the idle timeout be set to a value that is reasonably smaller than your archive log retention policy.

Thanks,
-cc

Vladislav P

unread,

Nov 10, 2025, 12:06:10 PM (2 days ago) Nov 10

to debezium

Hi, Chris.
1) My "lob.enabled" parameter is set to "false". Can setting the value of "lob.enabled" to true help me?
2) Do I understand correctly that all these errors relate to versions 3.4, 3.2, 3.3. Can I work around them if I start using version 2.7?

[1]: https://issues.redhat.com/browse/DBZ-8747
[2]: https://issues.redhat.com/browse/DBZ-9664
[3]: https://github.com/debezium/debezium/pull/6880

3) Now I'm waiting for answers from the database administrator. Because we archive and delete logs very quickly. According to him, this period is less than an hour.
4) I will try to get an answer to this question: "Make sure the database sets idle connection timeouts if this was simply an idle user transaction".

понедельник, 10 ноября 2025 г. в 18:45:43 UTC+4, Chris Cranford:

Vladislav P

unread,

Nov 11, 2025, 3:20:21 AM (yesterday) Nov 11

to debezium

Chris, you were right.
The database administrator just replied to me:
1) Regarding "idle time", the profiles used by users in the database are "unlimited". The database itself does not stop sessions due to inactivity. Ask them to set this parameter for at least 3 hours ?
2) Logs are backed up every 12 hours using RMAN. To delete it, you need to create 1 backup copy and apply it to the replica (backup database). There is no time reserve right now. I will ask you to leave logs at least 1 hour in advance. Will this improve the situation?

понедельник, 10 ноября 2025 г. в 21:06:10 UTC+4, Vladislav P:

Chris Cranford

unread,

Nov 11, 2025, 11:34:04 AM (yesterday) Nov 11

to debe...@googlegroups.com

Hi -

Your first point should definitely help; however my concern here is how (2) is worded.

If the DBA team removes all archive logs on the system every 12 hours, and they do so without any grace period you will continue to face the problem. I come back to my classic example of the following, you run the removal at 00:00 and 12:00 every day and if an archive log is created at 11:59 or 23:59, and there is no grace period, it gets removed. The script needs to account for only removing logs older than its execution frequency.

-cc

To view this discussion visit https://groups.google.com/d/msgid/debezium/8d7507f1-0c5e-42c9-a850-9af2f73082a8n%40googlegroups.com.

Vladislav P

unread,

Nov 11, 2025, 5:46:43 PM (yesterday) Nov 11

to debezium

Yes, now I'm faced with the fact that the deletion script was triggered at 00:00. And at 00:40, my connector went off. The error is still the same: "The archived log cannot be opened."

I want to write to the administrators so that when deleting them, they take into account only those logs that are older than 1 hour, but will this change the situation? Or do I need to ask for a lot more hours?

вторник, 11 ноября 2025 г. в 20:34:04 UTC+4, Chris Cranford:

Chris Cranford

unread,

Nov 11, 2025, 8:50:36 PM (yesterday) Nov 11

to debe...@googlegroups.com

Hi -

I am afraid I cannot say with certainty that one hour would be sufficient for your environment. Please take a look at the LagFromSourceInMilliseconds and MilliSecondsBehindSource JMX metrics and see how these values trend. If these never exceed one hour and if you never expect any downtime window for longer than 1 hour, then that might be sufficient. But in practice, most Oracle environments we've seen or talk with users about generally retain their logs for up to 6+ hours just to sustain a reasonable buffer if they need a downtime or if there happens to be a situation where an unusual activity pattern causes unexpected latency.

-cc

To view this discussion visit https://groups.google.com/d/msgid/debezium/4c40161c-c3a8-4410-a5dc-f677b6a3fdf7n%40googlegroups.com.

Vladislav P

unread,

9:40 AM (14 hours ago) 9:40 AM

to debezium

Hi, Chris.

We have configured the log deletion only after 6 hours. I'll check if there's an error today.

Now I would like to ask for help with optimizing the config. I start the connector with "snapshot.mode": "no_data". After that, I launched an incremental snapshot for 16 tables. Many of them are not very big, but even they take a very long time to pour. For example, a table with 169 records was moved for 1+ hour. And on another larger table (1.7 million records) The connector has been running for 3 hours and has transferred only 450 records. Is this normal?

And is it possible to start sinc-connector immediately after starting source-connector? Or do I have to wait for the incremental snapshot to work?

{
"name": "source-connector",
"config": {
"connector.class": "io.debezium.connector.oracle.OracleConnector",
"tasks.max": "1",
"database.hostname": "{{sourceDatabaseHost}}",
"database.port": "{{sourceDatabasePort}}",
"database.user": "{{sourceDatabaseUser}}",
"database.password": "{{sourceDatabasePassword}}",
"database.dbname": "{{sourceDatabaseName}}",
"table.include.list": "SCHEMA1.DEBEZIUM_SIGNAL,...",
"column.include.list": "SCHEMA1\\.DEBEZIUM_SIGNAL\\.(ID|TYPE|DATA),...",
"topic.prefix": "{{topicPrefix}}",
"database.server.name": "{{topicPrefix}}",
"schema.history.internal.kafka.topic": "dbz_oracle_wpms_history",
"schema.history.internal.kafka.bootstrap.servers": "{{kafkaBootstrapServers}}",
"log.mining.strategy": "hybrid",
"log.mining.query.filter.mode": "in",

"message.key.columns": "...",

"signal.enable.channels": "source",
"signal.data.collection": "{{sourceDatabaseName}}.SCHEMA1.DEBEZIUM_SIGNAL",

"incremental.snapshot.chunk.size": 50000,
"incremental.snapshot.allow.schema.changes": "true",
"topic.creation.enable": "true",
"topic.creation.default.replication.factor": 1,
"topic.creation.default.partitions": 1,
"topic.creation.default.cleanup.policy": "delete",

"snapshot.mode": "no_data",
"log.mining.transaction.retention.ms": "10800000",
"schema.history.internal.store.only.captured.tables.ddl": "true",
"snapshot.database.errors.max.retries": 2,

"heartbeat.interval.ms": "10000",
"heartbeat.action.query": "MERGE INTO SCHEMA1.DEBEZIUM_HEARTBEAT t USING (SELECT 1 id, CURRENT_TIMESTAMP ts FROM dual) s ON (t.id = s.id) WHEN MATCHED THEN UPDATE SET t.ts = s.ts WHEN NOT MATCHED THEN INSERT (id, ts) VALUES (s.id, s.ts)"
}
}

среда, 12 ноября 2025 г. в 05:50:36 UTC+4, Chris Cranford:

Chris Cranford

unread,

10:09 AM (13 hours ago) 10:09 AM

to debe...@googlegroups.com

Hi, incremental snapshot speed is directly proportional to the speed that the streaming operates. I suspect if you are seeing these tables to be snapshot so slowly, chances are there are performance issues with streaming. What is your current LagFromSourceInMilliseconds?

-cc

To view this discussion visit https://groups.google.com/d/msgid/debezium/bca3c755-15b5-42da-952a-3e2f1680190dn%40googlegroups.com.

Vladislav P

unread,

6:25 PM (5 hours ago) 6:25 PM

to debezium

Hi.
The connector was started using "snapshot.mode": "no_data". Incremental snapshots have worked, and several tables have already been read. But then the backup and deletion time came at 00:00, and at 00:40 the connector crashed with this error. (this error often appears in logs, but only as a WARNING 1 time, and the connector continues to work, but now it has crashed).

Today, the database administrator has set a new policy for deleting archived logs. We only delete those that are over 6 hours old. According to lagFromSourceDuration, it can be seen that the delay ranges from 6 seconds to 2 hours, everything is fine.
After such an error, how can I restart the connector to continue reading (if possible)? Are there any suggestions why this could happen and how to get around it?

```
2025-11-12T20:10:12,990 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 20576 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:11:12,998 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 6590 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:11:16,266 INFO || [AdminClient clientId=connector-adminclient-wpms-source-connector-avro-0] Rebootstrapping with Cluster(id = null, nodes = [apache-kafka:19092 (id: -1 rack: null isFenced: false)], partitions = [], controller = null) [org.apache.kafka.clients.admin.internals.AdminMetadataManager]
2025-11-12T20:12:13,006 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 9693 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:13:13,015 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 10561 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:14:13,023 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 10140 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:15:13,030 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 3625 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:16:13,037 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 5751 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:17:13,044 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 3596 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:18:13,051 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 16970 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:19:01,972 INFO || [AdminClient clientId=1-shared-admin] Rebootstrapping with Cluster(id = null, nodes = [apache-kafka:19092 (id: -1 rack: null isFenced: false)], partitions = [], controller = null) [org.apache.kafka.clients.admin.internals.AdminMetadataManager]
2025-11-12T20:19:13,058 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 2 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:20:13,065 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 13187 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:20:49,059 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:20:49,060 INFO Oracle||streaming No logs available yet (attempt 1)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:20:50,430 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:20:50,431 INFO Oracle||streaming No logs available yet (attempt 2)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:20:52,773 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:20:52,774 INFO Oracle||streaming No logs available yet (attempt 3)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:20:57,139 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]2025-11-12T20:20:57,140 INFO Oracle||streaming No logs available yet (attempt 4)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:21:05,541 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:21:05,542 INFO Oracle||streaming No logs available yet (attempt 5)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:21:13,072 INFO || WorkerSourceTask{id=wpms-source-connector-avro-0} Committing offsets for 13756 acknowledged messages [org.apache.kafka.connect.runtime.WorkerSourceTask]
2025-11-12T20:21:16,572 INFO || [AdminClient clientId=connector-adminclient-wpms-source-connector-avro-0] Rebootstrapping with Cluster(id = null, nodes = [apache-kafka:19092 (id: -1 rack: null isFenced: false)], partitions = [], controller = null) [org.apache.kafka.clients.admin.internals.AdminMetadataManager]
2025-11-12T20:21:21,880 INFO Oracle||streaming Redo Thread 1 is inconsistent; failed to find log with sequence 516702 [io.debezium.connector.oracle.logminer.LogFileCollector]
io.debezium.DebeziumException: Redo Thread 1 is inconsistent; failed to find log with sequence 516702
at io.debezium.connector.oracle.logminer.LogFileCollector.logException(LogFileCollector.java:567) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollectorsOpenThreadConsistent(LogFileCollector.java:324) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.isLogFileListConsistent(LogFileCollector.java:197) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:21:21,880 INFO Oracle||streaming No logs available yet (attempt 6)... [io.debezium.connector.oracle.logminer.LogFileCollector]
2025-11-12T20:21:53,881 ERROR Oracle||streaming LogMiner session stopped due to an error. [io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource]
io.debezium.connector.oracle.logminer.LogFileNotFoundException: None of the log files contain offset SCN: 6337779057442, re-snapshot is required.
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:96) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2025-11-12T20:21:53,882 ERROR Oracle||streaming Producer failure [io.debezium.pipeline.ErrorHandler]
io.debezium.connector.oracle.logminer.LogFileNotFoundException: None of the log files contain offset SCN: 6337779057442, re-snapshot is required.
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:96) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?].i2025-11-12T20:21:53,883 INFO Oracle||streaming Streaming metrics at shutdown: LogMinerStreamingChangeEventSourceMetrics{connectorConfig=io.debezium.connector.oracle.OracleConnectorConfig@1a2380ef, startTime=2025-11-12T17:26:12.353344441Z, clock=SystemClock[Z], currentScn=6337890563064, offsetScn=6337779057442, commitScn=6337779224148, oldestScn=6337779057443, oldestScnTime=2025-11-12T18:08:50.399Z, currentLogFileNames=[+FRA/RAC_RCDB/redo_2_11.log, +FRA/RAC_RCDB/redo_1_02.log], redoLogStatuses=[+FRA/RAC_RCDB/redo_1_01.log | ACTIVE, +FRA/RAC_RCDB/redo_1_02.log | CURRENT, +FRA/RAC_RCDB/redo_2_11.log | CURRENT, +FRA/RAC_RCDB/redo_2_13.log | INACTIVE, +FRA/RAC_RCDB/redo_2_14.log | INACTIVE, +FRA/RAC_RCDB/redo_1_05.log | INACTIVE, +FRA/RAC_RCDB/redo_2_15.log | INACTIVE, +FRA/RAC_RCDB/redo_2_08.log | INACTIVE, +FRA/RAC_RCDB/redo_1_04.log | INACTIVE, +FRA/RAC_RCDB/redo_1_03.log | INACTIVE, +FRA/RAC_RCDB/redo_2_12.log | INACTIVE], databaseZoneOffset=+03:00, batchSize=100000, logSwitchCount=497, logMinerQueryCount=695, sleepTime=1000, minimumLogsMined=2, maximumLogsMined=37, maxBatchProcessingThroughput=23480, timeDifference=-66601, processedRowsCount=174069709, activeTransactionCount=11, rolledBackTransactionCount=282696, oversizedTransactionCount=0, changesCount=9865859, scnFreezeCount=0, batchProcessingDuration={min=PT2.557922533S,max=PT48.162976753S,total=PT2H42M45.656929595S}, fetchQueryDuration={min=PT0.598436457S,max=PT3.47725636S,total=PT24M23.387220733S}, commitDuration={min=PT0.000000474S,max=PT1.112903003S,total=PT21.884785163S}, lagFromSourceDuration={min=PT6.141506169S,max=PT2H14M21.416279818S,total=PT16197077H48M44.099175949S}, miningSessionStartupDuration={min=PT0.002903802S,max=PT0.009329573S,total=PT2.975295687S}, parseTimeDuration={min=PT0.000014792S,max=PT0.009811698S,total=PT3M32.98757225S}, resultSetNextDuration={min=PT0.000000074S,max=PT1.960188341S,total=PT2H2.026501629S}, userGlobalAreaMemory={value=69965352,max=93036536}, processGlobalAreaMemory={value=77849600,max=134079488}, abandonedTransactionIds=[], rolledBackTransactionIds=[bd000700fcd5e600, c8001a00c816fd00, 520015002f832701, e1001c0097a29a00, 14000500b4e2f301, 220012009f5b3902, 06000500b2029701, 4a000e008f97c700, 130017007f47c802, 43000e00073fab01]} [io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource]
2025-11-12T20:21:53,906 INFO Oracle||streaming Offsets as shutdown: OracleOffsetContext [scn=6337779057442, txId=2b001400cdd4a501, txSeq=7, commit_scn=["6337779224148:1:04001500ab548e01","6337779223932:2:590010004c4cc800"], lcr_position=null] [io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource]
2025-11-12T20:21:53,907 INFO Oracle||streaming Finished streaming [io.debezium.pipeline.ChangeEventSourceCoordinator]
2025-11-12T20:21:53,907 INFO Oracle||streaming Connected metrics set to 'false' [io.debezium.pipeline.ChangeEventSourceCoordinator]
2025-11-12T20:21:54,363 ERROR || WorkerSourceTask{id=wpms-source-connector-avro-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted [org.apache.kafka.connect.runtime.WorkerTask]

org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.

at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:67) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:217) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:88) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:329) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline

or.executeChangeEventSources(ChangeEventSourceCoordinator.java:207) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:147) ~[debezium-core-3.3.1.Final.jar:3.3.1.Final]

at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) ~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]

at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]

Caused by: io.debezium.connector.oracle.logminer.LogFileNotFoundException: None of the log files contain offset SCN: 6337779057442, re-snapshot is required.
at io.debezium.connector.oracle.logminer.LogFileCollector.getLogs(LogFileCollector.java:96) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.prepareLogsForMining(AbstractLogMinerStreamingChangeEventSource.java:1142) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.buffered.BufferedLogMinerStreamingChangeEventSource.executeLogMiningStreaming(BufferedLogMinerStreamingChangeEventSource.java:146) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
at io.debezium.connector.oracle.logminer.AbstractLogMinerStreamingChangeEventSource.execute(AbstractLogMinerStreamingChangeEventSource.java:212) ~[debezium-connector-oracle-3.3.1.Final.jar:3.3.1.Final]
... 9 more
2025-11-12T20:21:54,365 INFO || Stopping down connector [io.debezium.connector.common.BaseSourceTask]
2025-11-12T20:21:54,367 INFO || SignalProcessor stopped [io.debezium.pipeline.signal.SignalProcessor]
2025-11-12T20:21:54,368 INFO || Requested thread factory for component JdbcConnection, id = JdbcConnection named = jdbc-connection-close [io.debezium.util.Threads]
2025-11-12T20:21:54,368 INFO || Creating thread debezium-jdbcconnection-JdbcConnection-jdbc-connection-close [io.debezium.util.Threads]
2025-11-12T20:21:54,371 INFO || Connection gracefully closed [io.debezium.jdbc.JdbcConnection]
2025-11-12T20:21:54,372 INFO || Debezium ServiceRegistry stopped. [io.debezium.service.DefaultServiceRegistry]
2025-11-12T20:21:54,374 INFO || Requested thread factory for component JdbcConnection, id = JdbcConnection named = jdbc-connection-close [io.debezium.util.Threads].ChangeEventSourceCoordinat
```

среда, 12 ноября 2025 г. в 19:09:55 UTC+4, Chris Cranford:

Reply all

Reply to author

Forward