Automatic Topic Creation for Kafka Connect Source Connector

[Micah Whitacre]

I have a Kafka cluster where the brokers have the settings

auto.create.topics.enable = false

When standing up the JDBCSourceConnector with a topic.prefix, is there a way to configure the Kafka Connector instance to create the new topics automatically?  Since the connector will detect new tables it'd be nice if it could create the topics once the table appears without an extra step of creating the topic out of band.  Is this possible?  Looking through the WorkerSourceTask, I do not see where it would attempt to create the topic so I wasn't sure if there was a property I could set to make this happen.

Thanks,

Micah

[Ryan Pridgeon]

Michah, 

I think you are correct, outside of the global auto.create.topics.enable there is no way to allow the connector instance to create it's own topics automatically. It looks like Kafka's Authorization scheme will be adding a 'CREATE' operation to the permissions may get you where you need to be in the future. This way you could keep auto.create.topics.enable=true while maintaining control over when topics are created. 

http://docs.confluent.io/3.1.1/kafka/authorization.html

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/f9e53c77-23e4-43c3-ad94-0df636f35823%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.