debezium Grab a table without a primary key.

[铅笔小新]

This error occurred. 

2018-09-20 01:12:42,360 ERROR  ||  Producer failure   [io.debezium.pipeline.ErrorHandler]

org.apache.kafka.connect.errors.ConnectException: java.lang.NullPointerException: key must not be null

at io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:149)

at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:68)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

[John Psoroulas]

Hi,

try to exclude the table form debezium monitoring by setting the schema.blacklist connector property

https://debezium.io/docs/connectors/postgresql/#connector-properties

regards,

John

[John Psoroulas]

On Thursday, 20 September 2018 09:22:05 UTC+3, John Psoroulas wrote:

Hi,

try to exclude the table form debezium monitoring by setting the schema.blacklist connector property

https://debezium.io/docs/connectors/postgresql/#connector-properties

The link for sqlserver

https://debezium.io/docs/connectors/sqlserver/#connector-properties

[Gunnar Morling]

I suppose the OP actually wants to capture changes to that table. I've created https://issues.jboss.org/browse/DBZ-916 for tracking this.

If possible we should employ the same behaviour as e.g. for MySQL. There we'll take the first unique key found in the table definition as message key, if there's no PK. And then, quoting the docs:

> If the table does not have a primary or unique key, then the change event’s key will be null. This makes sense since the rows in a table without a primary or unique key constraint cannot be uniquely identified.

In any case, we shouldn't run into an NPE.

--Gunnar

[Gunnar Morling]

Gotcha. We'll try to get this fixed soon. Of course any help with it will be highly welcomed :)

--Gunnar

Am Donnerstag, 20. September 2018 09:13:33 UTC+2 schrieb 铅笔小新:

hi

for example：my table struct is

                    OutQualityID int

                    QCCheckID int

                    Memo varchar

table without a primary key

    "name": "mes-QC_OutCheckReason",

    "config": {

        "connector.class" : "io.debezium.connector.sqlserver.SqlServerConnector",

        "tasks.max" : "1",

        "database.server.name" : "mes",

        "database.server.id":"QC_OutCheckReason",

        "database.hostname" : "172.16.7.2",

        "database.port" : "1433",

        "database.user" : "bl",

        "database.password" : "bl@MES",

        "database.dbname" : "RCollarGarmentMES",

        "table.whitelist": "RCollarGarmentMES.dbo.QC_OutCheckReason",

        "database.history.kafka.bootstrap.servers" : "kafka-0.kafka.default.svc.cluster.local:9092",

        "database.history.kafka.topic": "mes-schema-changes-QC_OutCheckReason",

        "include.schema.changes": "true" ,

        "snapshot.mode": "initial_schema_only"

error：2018-09-20 01:12:42,360 ERROR  ||  Producer failure   [io.debezium.pipeline.ErrorHandler]