Oracle XStream source - ORA-26824: user-defined XStream callback error

[Sagar Kapare]

Hi All,

I am using the Oracle XStream connector with Debezium version 1.5.2.Final in Embedded mode. I configured the XStream outbound server to receive the events from tables HR.Countries and HR.LONGTABLE. 

Now HR.LONGTABLE has a column (HR.LONGTABLE.COMMENTS) with data type as LONG which is not supported by debezium 1.5.2.Final. 

I configured the connector to only read from HR.Countries, however we still get LCR events from HR.LONGTABLE in debezium which is expected since XOut server is configured that way. 

However I am seeing connector keep stopping with following error:

2021-10-26 22:56:10,429 - ERROR [debezium-oracleconnector-dummy-change-event-source-coordinator:i.d.p.ErrorHandler@31] - Producer failure
oracle.streams.StreamsException: ORA-26824: user-defined XStream callback error
XStreamOut DeliverChunk: exception thrown in processChunk callback.


	at oracle.streams.XStreamOut.XStreamOutReceiveLCRCallbackNative(Native Method) ~[xstreams.jar:na]
	at oracle.streams.XStreamOut.receiveLCRCallback(XStreamOut.java:465) ~[xstreams.jar:na]
	at io.debezium.connector.oracle.xstream.XstreamStreamingChangeEventSource.execute(XstreamStreamingChangeEventSource.java:116) ~[debezium-connector-oracle-1.5.2.Final.jar:1.5.2.Final]
	at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:152) [debezium-core-1.5.2.Final.jar:1.5.2.Final]
	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:119) [debezium-core-1.5.2.Final.jar:1.5.2.Final]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_275]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_275]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_275]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_275]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_275]

This happens just after it processes the LCR for HR.LONGTABLE which has a LONG column data type. 

I even see the following log line just before this exception, indicating that the table is filtered correctly.

2021-10-26 22:56:10,392 - TRACE [debezium-oracleconnector-dummy-change-event-source-coordinator:i.d.p.EventDispatcher@187] - Filtered data change event for ORCL.HR.LONGTABLE

I tried passing the connector config to explicitly exclude that column as follows, however it still failed with the same error.

2021-10-26 22:52:07,293 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    database.server.name = dummy
2021-10-26 22:52:07,293 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    offset.flush.interval.ms = 1000
2021-10-26 22:52:07,293 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    column.exclude.list = HR.LONGTABLE.COMMENTS
2021-10-26 22:52:07,293 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    database.serverTimezone = UTC
2021-10-26 22:52:07,294 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    database.password = ********
2021-10-26 22:52:07,294 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    name = delta
2021-10-26 22:52:07,294 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    database.out.server.name = dbzxout
2021-10-26 22:52:07,294 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    table.include.list = HR.COUNTRIES
2021-10-26 22:52:07,294 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    ddl_event_sent = 
2021-10-26 22:52:07,295 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    snapshot.mode = schema_only
2021-10-26 22:52:07,295 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    snapshot = 
2021-10-26 22:52:07,295 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    database.history = io.cdap.delta.plugin.common.DBSchemaHistory
2021-10-26 22:52:07,295 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    snapshot_completed = 
2021-10-26 22:52:07,295 - INFO  [pool-9-thread-1:i.d.c.c.BaseSourceTask@126] -    scn = 

Any idea why it would still fail when the table is not in the include list in connector config? 

We are planning to use the connector to stream CDC events for HR.LONGTABLE as well with excluding column COMMENTS (since it is LONG and unsupported in 1.5.2.Final). Any workaround to achieve this?

Thanks and Regards,

Sagar

[Chris Cranford]

Hi Sagar -

I believe we adjusted the LCR callback handler in Debezium 1.6.0.Final to support chunk processing.  Prior to 1.6, these methods explicitly threw an exception and so for the version you're using there is not a workaround.  It would be interesting to know if you used 1.6 or 1.7 instead whether or not the excluded column configuration works as you expect without a failure.  I would suggest upgrading and testing that and report back if you encounter any problems.

Thanks,
CC

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CANRorfeoqNggQg194AvxPNSJqo2iJX_MMEzPnBjZ5_UrmX6dww%40mail.gmail.com.

[Sagar Kapare]

Hi Chris,

Thanks a lot for quick response :) I updated the library version to 1.6.3.Final and can confirm that I no longer see this issue. I did not have to pass the exclude column configuration column.exclude.list = HR.LONGTABLE.COMMENTS.

As a side note, I also tried streaming CDC events with table having LONG column. For snapshot events I see the values for LONG column, however for streaming events I see them as NULL. I guess its expected as LONG is not yet supported, right?

Thanks again for help!

Best Regards,

Sagar

[Chris Cranford]

Sagar -

The reason you see the values during the snapshot is because we're using raw JDBC queries to fetch the column values and the driver is capable of providing those directly; however during streaming we're limited to the capabilities of Xstream or LogMiner depending on the adapter you're using.  These frameworks don't support every Oracle data type and that's why those are streamed with values of NULL; as they're technically unsupported at this time.

CC

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/377fc48a-43e0-45d0-901f-2457ad21659en%40googlegroups.com.

[Sagar Kapare]

Makes sense! Thanks Chris for clarification.

Regards,

Sagar