Not able to stream MySQL existing tables: schema isn't known to connector

[Zhiqi Chen]

Hi,

We created our MySQL database 3+ years ago and now we would like to stream data using Debezium. We created and started a new connector for existing tables but they were not getting streamed, and we were getting below errors:

"trace": "org.apache.kafka.connect.errors.ConnectException:

Encountered change event for table db.tb

whose schema isn't known to this connector\n\tat io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:241)\n\tat io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:218)\n\tat io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:607)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1104)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:955)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:595)\n\tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:839)\n\tat java.lang.Thread.run(Thread.java:750)\nCaused by: org.apache.kafka.connect.errors.ConnectException: Encountered change event for table db.tb whose schema isn't known to this connector\n\tat io.debezium.connector.mysql.BinlogReader.informAboutUnknownTableIfRequired(BinlogReader.java:875)\n\tat io.debezium.connector.mysql.BinlogReader.handleUpdateTableMetadata(BinlogReader.java:849)\n\tat

io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:590)\n\t... 5 more\n"

But if we created new table after the connector started, it was working fine. We come up with a temporary solution: 1. we renamed the existing tables and created one more time with same name and same schema, 2. we dropped the newly created ones and renamed back the old ones. Then those tables started to stream fine.

However, after running for some time, again we are getting the same error. Any suggestion and solution will will be appreciated!

Debezium version we are using: 1.8. MySQL version we are using: 8.0.28. Kafka version we are using: 2.5.0.

Also attached our debezium configuration and log info.

Thanks!

[jiri.p...@gmail.com]

Hi,

you are using legacy implementation of MySQL connector that is no longer supported (accroding to stacktrace as the classes are not used in new implementation). Are you sure you are running version1.8?

J.

[Zhiqi Chen]

Hi,

Thanks for response! We get the version number from API connector-plugins:

{

"class": "io.debezium.connector.mysql.MySqlConnector",

"type": "source",

"version": "1.8.1.Final"

}

Previously we were using 1.4 but we have upgraded that. The trace and log we got after upgrade.

[jiri.p...@gmail.com]

Hi, but the runtime stacktrace really does not match to the new implementation unless you explictly disabled it.

J.

[Zhiqi Chen]

Hi,

We have double checked and upgraded to version 1.9.5, but unfortunately the issue still exists. Here's the latest log:

[2022-08-02 16:23:36,183] WARN Encountered change event 'Event{header=EventHeaderV4{timestamp=1659457416000, eventType=TABLE_MAP, serverId=73079, headerLength=19, dataLength=77, nextPosition=840944952, flags=0}, data=TableMapEventData{tableId=1862, database='db', table='tb', columnTypes=8, 15, 18, 3, 3, 3, 3, 3, 15, 15, 18, 18, 18, 18, 3, 15, columnMetadata=0, 300, 0, 0, 0, 0, 0, 0, 30, 300, 0, 0, 0, 0, 0, 300, columnNullability={2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, eventMetadata=TableMapEventMetadata{signedness={}, defaultCharset=33, charsetCollations=null, columnCharsets=null, columnNames=null, setStrValues=null, enumStrValues=null, geometryTypes=null, simplePrimaryKeys=null, primaryKeysWithPrefix=null, enumAndSetDefaultCharset=null, enumAndSetColumnCharsets=null,visibility=null}}}' at offset {transaction_id=null, file=mysql-bin.000269, pos=840944760, gtids=3a731eec-e201-11ec-8685-0050569f37ec:1-258,5293ceac-994a-45cd-bf1b-f80f2492c46c:1-105150061,ee4f745d-d3b6-4dc9-8753-7c53c352d9f1:1-141240804, server_id=73079, event=1} for table db.tb whose schema isn't known to this connector. One possible cause is an incomplete database history topic. Take a new snapshot in this case. The event will be ignored.

We have no idea how to resolve it. Is there any suggestion?

Thank you so much!

[Chris Cranford]

Hi Zhiqi Chen -

So the incomplete database history topic error can happen for a variety of reasons.  Some of the most common reasons are:

    * Topic retention is not set to infinite
    * Topic isn't configured with 1 partition
    * Topic has gotten corrupted in Kafka
    * A table was added to "table.include.list" after the snapshot has completed

If there have been no DDL changes since you added the table that would impact existing binlog entries that could be based on older table structures, you could use "snapshot.include=schema_recovery" to recover the database history topic and continue streaming without warnings.  You can also prepare a signal (if you have signaling enabled) to perform a snapshot of the newly added table.  If none of that works, you can also consider redeploying the connector or clearing the history topic and offsets so that the connector starts new, performing the snapshot, and then streaming changes.

Hope that helps
CC

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/2ad32abe-5f59-4e91-bfba-fac7eb782cdan%40googlegroups.com.