debezium altering dates format

[harby ariza]

 Hi everyone,

I got a postgres debezium connector running and I can see even the data on database looks like :

effectivedates_effectivestartdate | date

 

effectivedates_effectiveenddate 

---------------------------------+--------
 9999-12-31                     

when I check the CDC topics the data appears to be like :

}, "effectivedates_effectiveenddate": { "string": "10000-12-31" } 

and data is landing on S3 buckect like :

max(after.effectivedates_effectiveenddate AS `effectivedates_effectiveenddate`)|
+-------------------------------------------------------------------------------+
|10000-12-31                                                                    |
+-------------------------------------------------------------------------------+

so does anyone knows that is causing this behaviour. Is there a configuration parameter that might help to fix this. thanks. 

[Gunnar Morling]

What type has that column?

[harby ariza]

date

effectivedates_effectivestartdate | date

--
You received this message because you are subscribed to a topic in the Google Groups "debezium" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/debezium/jMU_Tl-GLIU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/6a38f3f5-b36a-401d-976e-4de711bd9fc8n%40googlegroups.com.

[Gunnar Morling]

Hum, are you applying some transformation somewhere? DATE columns are exported as int32 (days since epoch) by Debezium. So I suppose the issue may be some custom SMT you've applied? Can you share you complete connector config?

[harby ariza]

The only transformation we are applying is decimal.handling.mode = string . Would that cause a problem?

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/398d0dcf-a7c2-4451-a9ec-03da063f71b8n%40googlegroups.com.

[Gunnar Morling]

No, this won't impact temporal data types.

Can you provide steps for reproducing based on our tutorial set-up (https://github.com/debezium/debezium-examples/tree/main/tutorial)?

[Gunnar Morling]

Am Fr., 6. Mai 2022 um 17:56 Uhr schrieb harby ariza <harby...@gmail.com>:
>
> just in case this is the entire connector config but I am just omitting the table list:
>

> "timestampConverter.type": "oryanmoshe.kafka.connect.util.TimestampConverter",

This looks like the likely cause. I suppose there's an issue with this
converter.

> You received this message because you are subscribed to the Google Groups "debezium" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/c55fca27-03d9-482b-aaa7-9f88f5eb6861n%40googlegroups.com.