table.include.list regex expression

[Srinivas Vangari]

We are planning to replicate all db tables into Kafka as topics however we dont want to include all table names in the connector file.

Looking to pull required table names using regex. Can someone give some examples.

Thank you.

[Chris Cranford]

Hi Srinivas -

Something as simple as "table.include.list=<schema-name>.*" would match all tables within that schema. 

Hope that helps
Chris

--
You received this message because you are subscribed to the Google Groups "debezium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to debezium+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/af056abb-2025-42b5-9988-88301cb7b818n%40googlegroups.com.

[Chris Cranford]

Hi Srinivas -

Unfortunately your choices are to define "schema.include.list" / "schema.exclude.list" or "table.include.list" / "table.exclude.list".  If the number of tables you wish to exclude is a shorter list than the number of tables you wish to include within a specific schema, you could define the connector configuration as follows:

schema.include.list=MYSCHEMA
table.exclude.list=<...TABLE1>,<...TABLE2>

In this case all tables from MYSCHEMA would get captured except TABLE1 and TABLE2.

The other alternative is to reduce the explicit fully qualified table names down to a set of regular expressions that are comma-delimited if possible.  If neither of these solutions are possible, then your only option is to list each on separately.

Chris

On 3/28/23 10:56, Srinivas Vangari wrote:

Hey Chris,

We do not want to include all tables from the schema. Is there any way we can have an expression to limit to include specific tables instead of keeping each table name.

Regards,

Srinivas

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/f7b4ef97-8215-4c8c-abc6-87679485ffe1n%40googlegroups.com.

[srinivas vangari]

Thank you Chris for the response.

I am working on a regular expression to include all tables starting with rcu using "rcu_[A-Za-z_]+". This expression is creating topics only for 8 tables. I tested this expression in Postgres and it is working well. Am I missing something?

Regards,

Srinivas

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/571e25ec-81a6-136d-35e3-65ed4f8d9480%40gmail.com.

--

Regards,
Srinivas

[Chris Cranford]

Hi Srinivas -

I can't necessarily speak to whether that will miss any potential tables; but assuming all tables are always starting with "rcu_" and match that provided pattern, then it should work.  Are there other tables it should be picking up but isn't? 

Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/debezium/CAFmQ2MvPW3KhMzYA4RE9p7XVZojwSHJAn0sU-8BiVnG5cpv90Q%40mail.gmail.com.