Oracle Snapshot question

[Donald Lyon]

I'm having an issue where the snapshot process is looking at all of the tables in my schema rather than just tables in my tables.include.list.  This has the side effect of erroring on a table that's not supported, and also requiring supplemental logging on every table in the schema.  Is there a way to limit the schema tables to only the one(s) I want monitored?

Version: 1.5.0.Final

Running EmbeddedEngine

Config:

return io.debezium.config.Configuration.create()
.with("name", "my-oracle-connector")
.with("connector.class", "io.debezium.connector.oracle.OracleConnector")
.with("offset.storage", "org.apache.kafka.connect.storage.FileOffsetBackingStore")
.with("offset.storage.file.filename", "/Users/don/debezium/my-offset.dat")
.with("offset.flush.interval.ms", 60000)
.with("database.server.name", DBHost+"-"+DBName)
.with("database.hostname", DBHost)
.with("database.port", DBPort)
.with("database.user", DBUserName)
.with("database.password", DBPassword)
.with("database.dbname", DBName)
.with("database.history", "io.debezium.relational.history.FileDatabaseHistory")
.with("database.history.file.filename", "/Users/don/debezium/dbhistory.dat")
.with("snapshot.mode", "schema_only")
.with("schema.include.list", "MYSCHEMA")
.with("schema.exclude.list", "MYSCHEMA.BAD_ORACLEAQ_TABLE_Q")
.with("table.include.list", "MYSCHEMA.TABLE1")
.build();

[Donald Lyon]

Specifically, in LogMinerHelper.checkSupplementalLogging():402.  Is there a way to limit the check for supplemental logging to only be tables that are in the tables.include.list?

[Donald Lyon]

Some more information.  It appears that the combination of schema.include.list and schema.exclude.list was causing the schema tables to include the additional tables.  When I comment out the schema.excludes.list, it gets to LogMinerHelper.checkSupplementalLogging() with the single table I specified.  However, I wasn't able to complete the initial snapshotting without that setting since BAD_ORACLEAQ_TABLE_Q was not supported.  Is there a different way to exclude tables from being included in the schema structure snapshot?

I'm also now getting an issue in the supplemental logging check.  It fails because tableId.table() is lower case and the SQL isn't case insensitive.  I'm not sure if database version plays a role in that, but I'm using Oracle 11.2.0.4.

[Chris Cranford]

Hi Donald,

Regarding the case, for LogMiner you need to specify "database.tablename.case.insensitive=false" in your connector configuration.  That should resolve the problem with the tableId.table() case comparison failure you're experiencing. 

With respect to include/exclude list configuration, those settings regardless of connector are mutually exclusive.  In your connector configuration below, you can't combine schema.include.list with schema.exclude.list as that can lead to unexpected behavior and you've encountered.  The documentation for these options indicate this mutual exclusivity.  If you're only interested in a single table in your database, specifying just the "table.include.list" with that table or the small list of monitored tables is more than sufficient for the connector to only check those tables for logging configurations.

HTH,
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/da467e04-04e2-46ac-9b83-114244add03cn%40googlegroups.com.