Are there any ways to change the field's data type of an existing STREAM/TABLE ?

[kumar....@gmail.com]

I have an existing stream where I want to change the field's data type from INT to BIGINT, but when I tried with the latest 0.19.0 version using 'CREATE OR REPLACE' syntax ended up with the following error.

ksql> CREATE STREAM profile (profileId INT, name VARCHAR)

>  WITH (kafka_topic='profiles', value_format='json', partitions=1);

 Message

----------------

 Stream created

----------------

ksql> CREATE OR REPLACE STREAM profile (profileId BIGINT, name VARCHAR)

>  WITH (kafka_topic='profiles', value_format='json');

Cannot upgrade data source: DataSource '`PROFILE`' has schema = `PROFILEID` INTEGER, `NAME` STRING which is not upgradeable to `PROFILEID` BIGINT, `NAME` STRING. (The following columns are changed, missing or reordered: [`PROFILEID` INTEGER])

Please suggest how to handle this scenario.

Thanks,

Subash P

[Sergio Pena Anaya]

Hey Subash, schema type promotion is not supported yet. I think there are some conflicts on how to make it compatible with other consumers that are still using an old schema. For example, you change INT -> BIGINT in a stream, but you may have an external consumer (out of KSQL) that still reads your topic using the INT schema. This consumer will fail. Even if Schema Registry supports this, I'm not sure how this behaves in real scenarios.

The current solution is to recreate the stream with the new schema (drop + create). We're still figuring out how to solve these compatibility problems, but in the meantime schema changes are still limited. Btw, could you tell us a little bit about your use case? When schema types are changed and what do you do to change it in all your environments?

- Sergio

--
You received this message because you are subscribed to the Google Groups "ksqldb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ksql-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ksql-users/b9e93f56-3e76-40dd-9f60-dff88f885c15n%40googlegroups.com.