Debezium Server Configuration of binary.handling.mode for SQL Server

204 views
Skip to first unread message

Adam Whitmore

unread,
Oct 9, 2021, 9:33:51 AM10/9/21
to debezium
Hi - 

I'm looking to decode values for a column that is defined as `numeric(19,4)` in the source SQL Server db.  A record with a value of 1000000138.0000 in the source will come through as "CRhOh66g" in the payload from Debezium whether I try to configure the `binary.handling.mode` as `base64` or leave it at default setting.

I'm either mistaken on how to configure `binary.handling.mode` (should it have prefix of `debezium.source.`?) or I am on the wrong path of how to handle the values in the field, or both.  My ultimate goal is to use Python to retrive the numeric value in the payload.

Any help is appreciated.  Below are the values I have in an application.properties file for a POC I am running.  DBZ version is 1.6.2.Final.

debezium.sink.type=pubsub
debezium.source.connector.class=io.debezium.connector.sqlserver.SqlServerConnector
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.database.hostname=db-demo-container-host
debezium.source.database.port=1433
debezium.source.database.user=sa
debezium.source.database.password=<pass>
debezium.source.database.dbname=testDB
debezium.transforms=RerouteData, RerouteSchema
debezium.transforms.RerouteSchema.type=io.debezium.transforms.ByLogicalTableRouter
debezium.transforms.RerouteSchema.topic.regex=localhost
debezium.transforms.RerouteSchema.topic.replacement=debezium.server.poc.schema
debezium.transforms.RerouteData.type=io.debezium.transforms.ByLogicalTableRouter
debezium.transforms.RerouteData.topic.regex=localhost.([^.]+)\.([^.]+)
debezium.transforms.RerouteData.topic.replacement=debezium.server.poc.data
debezium.source.database.history=io.debezium.relational.history.FileDatabaseHistory
debezium.source.database.history.file.filename=data/testDB.history


Thanks,
Adam

Adam Whitmore

unread,
Oct 11, 2021, 4:13:09 PM10/11/21
to debezium
I see now the impact of the `debezium.source.binary.handling.mode` setting on the data message schema, in field strings in `schema.fields[0].fields`.  The default value of `binary` results in
  {"type": "bytes", "optional": false, "field": "Data"}

whereas a value of `base64` results in
{"type": "string", "optional": false, "field": "Data"}

As for decoding the numeric value, this SO post had the solution I was looking for.
Reply all
Reply to author
Forward
0 new messages