Suggestions adding new databases/tables to MySql CDC monitoring

[Revanth P]

Hi All,             
  Environment:                                                                                                                        
  - Debezium 3.4.1.Final, MySQL connector                                                                                   
  - Schema history + offset storage: JDBC (MySQL)                                                                      - MySQL cluster: ~800 databases, ~24,000 tables          
  - snapshot.mode=no_data                                                                                                              
   

  We frequently need to add new databases and tables to table.include.list. This requires a connector restart with the updated list.

                                                                                                                            
  Issue 1 — store.only.captured.tables.ddl=true (current setting):                                                 When the connector restarts with new tables added, we get:                                           Encountered change event for table <unmonitored_table> whose schema isn't known to this connecto. This happens because unmonitored tables with active DML appear in the binlog. Since their DDL was never stored (due to store.only.captured.tables.ddl=true), the connector crashes.                                                                                                                                                                                      
  Issue 2 — store.only.captured.tables.ddl=false (attempted fix):                                                Setting this to false resolves the crash, but on startup (especially with snapshot.mode=recovery to rebuild schema history), Debezium reads and stores DDL for all ~24,000 tables across ~800 databases. This takes hours and causes MySQL memory pressure during the process
                                                                                                    

 Any recommended patterns for managing table.include.list changes in large MySQL clusters?                                        

Thanks in advance,
Revanth P