NPE while creating oracle source connector

[Vijay Shankar]

Hi,

I am creating Oracle source connector with following curl post command. but getting null pointer exception, not able to fins where could be issue.

curl command

curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:8083/connectors/ -d ‘{
 "name": "debez_ora_cdc",
 "config": {
 "connector.class": "io.debezium.connector.oracle.OracleConnector",
 "tasks.max": "1",
 "database.server.name": "server1",
 "database.tablename.case.insensitive": "true",
 "database.hostname": "HOST_NAME",
 "database.port": "1522",
 "database.user": "logminer_user",
 "database.password": "logminer_user",
 "database.dbname": "test",
 "database.history.kafka.bootstrap.servers": "localhost:9092",
 "database.history.kafka.topic": "server1.oracle.history",
 "database.history.skip.unparseable.ddl": "true",
 "include.schema.changes": "true",
 "table.include.list": "logminer_user.users",
 "snapshot.mode": "initial",
 "errors.log.enable": "true"
 }

 }’

Exception 

{"name":"debez_ora_cdc","connector":{"state":"RUNNING","worker_id":"null:-1"},"tasks":[{"id":0,"state":"FAILED","worker_id":"null:-1","trace":"org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.\n\tat io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:116)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: io.debezium.DebeziumException: java.lang.NullPointerException\n\tat io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)\n\tat io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)\n\t... 5 more\nCaused by: java.lang.NullPointerException\n\tat io.debezium.relational.TableEditorImpl.columnWithName(TableEditorImpl.java:46)\n\tat io.debezium.relational.TableEditorImpl.hasColumnWithName(TableEditorImpl.java:50)\n\tat io.debezium.relational.TableEditorImpl.lambda$updatePrimaryKeys$0(TableEditorImpl.java:103)\n\tat java.util.ArrayList.removeIf(ArrayList.java:1413)\n\tat io.debezium.relational.TableEditorImpl.updatePrimaryKeys(TableEditorImpl.java:102)\n\tat io.debezium.relational.TableEditorImpl.create(TableEditorImpl.java:267)\n\tat io.debezium.relational.Tables.lambda$overwriteTable$2(Tables.java:192)\n\tat io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:84)\n\tat io.debezium.relational.Tables.overwriteTable(Tables.java:186)\n\tat io.debezium.jdbc.JdbcConnection.readSchema(JdbcConnection.java:1209)\n\tat io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:181)\n\tat io.debezium.connector.oracle.OracleSnapshotChangeEventSource.readTableStructure(OracleSnapshotChangeEventSource.java:35)\n\tat io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:114)\n\tat io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)\n\t... 8 more\n"}],"type":"source"} 

[Chris Cranford]

Hi Vijay -

What version of the Oracle connector are you using?  If you are not yet on 1.9.5.Final, I would suggest that you upgrade.  The error seems to indicate there was a problem resolving the primary key of one of the tables and we've made some improvements when primary keys are based on function-based or hidden columns using specific types of indices.

Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/19ba1e32-da37-4ec1-91ab-017d74d371d5n%40googlegroups.com.

[Vijay Shankar]

Hi Chris,

I am using debezium 1.9.5.Final. I have to create a connector for 1 table which is USERS table.  I have created a logminer user as  'logminer_user' and I am creating connection with logminer user only.

I am passing  "table.include.list": "logminer_user.users", is it right? because USERS table is available different schema and I have given grant to select to logminer user.

while creating connector, I observed that in log it is scanning all the schema table. is that my connector script is correct? 

[Chris Cranford]

Hi Vijay -

That is normal and expected behavior.  The scan is preparing the database history topic.  If you want the history topic to only capture schemas for the tables you are capturing, you need to set "database.history.store.only.captured.tables.ddl" with a value of true.  Please see https://debezium.io/documentation/reference/2.0/connectors/oracle.html#oracle-property-database-history-store-only-captured-tables-ddl for details.

Thanks
Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/04e55c71-36c7-4540-becc-dc8b3cd41ad5n%40googlegroups.com.