Kafka returning integers as "almost" Base64 ?

[Emanuele Meazzo]

Hello Everybody,

I have a strange issue with the Oracle connector that is blocking a POC I'm creating.

I was able to set everything up, and get the events posted in Kafka using the Logminer configuration, however, while strings are displayed ok, using the Kafka console consumer the Json message shows the integer fields in the Database in some kind of "almost" BASE64 string.

I tried to configure the connector specifying  "decimal.handling.mode": "precise" , but nothing changes.

I see something like this:

ORACLE

• INSERT INTO TESTTABLE
• VALUES (1,'hey hey hey hey');
• COMMIT;

Console Output:
"payload":{"before":null,"after":{"ID":"AQ==","VALUE":"hey hey hey hey"}

"AQ==" really seems like base64, but it's definitely not the base64 equivalent of 1 (which is "MQ==")

Here are the details of my environment:

1. Local Docker connector built with compose
2. Oracle 19c hosted on AWS RDS

docker-compose.yaml is attached

This is the connector configuration I'm deploying:

{

  "name": "inventory-connector",

  "config": {

    "connector.class" : "io.debezium.connector.oracle.OracleConnector",

    "tasks.max" : "1",

    "database.server.name" : "XXX",

    "database.hostname" : XXXXXXXXXXXX",

    "database.port" : "1521",

    "database.user" : "LOGMINER",

    "database.password" : "XXXXXXXXX",

    "database.dbname" : "XXX",

"database.schema" : "XXX",

    "database.connection.adapter" : "LOGMINER",

    "database.history.kafka.bootstrap.servers" : "kafka:9092",

    "database.history.kafka.topic": "schema-changes.inventory",

"snapshot.mode": "schema_only"

}

}

Any idea of what's going on?

Regards

[Chris Cranford]

Hi Emanuele -

What is the DDL of the TESTTABLE, I suspect this has to do with the scale/precision of the ID field, but just want to check.

Thanks,
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/50f2eb58-f00a-46e4-90af-442f7e428f22n%40googlegroups.com.

[Emanuele Meazzo]

Hi Chris,

the ID it's a simple integer:

CREATE TABLE LOGMINER.TESTTABLE

(

  ID     INTEGER,

  VALUE  VARCHAR2(50 BYTE),

  SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS,

  SUPPLEMENTAL LOG DATA (ALL) COLUMNS

)

TABLESPACE LOGMINER_TBS

PCTUSED    0

PCTFREE    10

INITRANS   1

MAXTRANS   255

STORAGE    (

            INITIAL          64K

            NEXT             1M

            MINEXTENTS       1

            MAXEXTENTS       UNLIMITED

            PCTINCREASE      0

            BUFFER_POOL      DEFAULT

           )

LOGGING 

NOCOMPRESS 

ILM ADD POLICY ROW STORE COMPRESS ADVANCED ROW AFTER 1 MONTH OF NO MODIFICATION

NOCACHE;

[Chris Cranford]

Emanuele -

From what I can tell for a simple INTEGER data type, that would be emitted as a Decimal schema type with precision=38 and scale=0 with the value being a BigDecimal.  We get the values from LogMiner's REDO SQL, so just to cover all bases, could  you enable TRACE logging for the connector and perform the insert and see what the connector detects as the value for "ID"? 

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/689dcc5c-12ba-4638-9afa-714ab10981bbn%40googlegroups.com.

[Emanuele Meazzo]

Hey Chris,

I'm not sure I know how to do it, I used the following command as in the docs but I see no additional logging in the console output:

curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/admin/loggers/io.debezium.connector.oracle -d '{"level": "TRACE"}'

Am I doing something wrong? How can I get you the log?