io.debezium.transforms.ByLogicalTableRouter seems to conflict with topic creation settings

[Vera van Mondfrans]

Hi,

I'm trying to set the following settings:

• topic.creation.default.replication.factor:3
• topic.creation.default.partitions:25

But if I set them, I get the following error:

 2021-01-08 17:38:31,809 ERROR Failed to start task mysql-connector-0 (org.apache.kafka.connect.runtime.Worker) [StartAndStopExecutor-connect-1-2]
 org.apache.kafka.connect.errors.ConnectException: org.apache.kafka.common.config.ConfigException: Unknown configuration 'transforms.Reroute.type'
     at org.apache.kafka.connect.runtime.ConnectorConfig.transformations(ConnectorConfig.java:296)
     at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:605)
     at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:555)
 ...

Any help would be most welcome :)

This is my full config:

database.port: "3306"

database.hostname: localhost

snapshot.mode: when_needed

snapshot.locking.mode: none

topic.creation.default.replication.factor: 3

topic.creation.default.partitions: 25

database.user: "${file:/opt/kafka/external-configuration/kafka-mysql-connector-config/__encrypt__debezium-mysql-credentials.properties:mysql_username}"

database.password: "${file:/opt/kafka/external-configuration/kafka-mysql-connector-config/__encrypt__debezium-mysql-credentials.properties:mysql_password}"

database.history.kafka.bootstrap.servers: "kafka-cluster-kafka-bootstrap:9092"

database.history.kafka.topic: "debezium.database-changes"

database.server.id: 184054

database.server.name: cloudsql

table.whitelist:  // removed due to size

transforms: Reroute

transforms.Reroute.type: io.debezium.transforms.ByLogicalTableRouter

transforms.Reroute.topic.regex: cloudsql\..+\.(.*)

transforms.Reroute.topic.replacement: debezium.$1

[jiri.p...@gmail.com]

Hi,

this is a known bug in Kafka 2.6.0 - see https://issues.apache.org/jira/browse/KAFKA-10387

J.

[Vera van Mondfrans]

Hi,

Thank you! Reverting to 2.5.1 did indeed fix it. We'll upgrade to 2.6.1 or 2.7.0 some later date. However, the topics are still created with only 1 partition and a replication factor of 1. Am I missing something?

Thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "debezium" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/debezium/Nt9mwvY1FQI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/6ff5658f-71cf-44f3-b537-f1c08506171en%40googlegroups.com.

[Vera van Mondfrans]

And hi again,

I just realized this is a kafka 2.6 feature, so I guess I'll be upgrading now anyways. Thanks!