Control Center - Transform Data Between 2 Topics

[kev.s...@gmail.com]

Hi,

I would like to know if anyone have an idea on how to use the Control Center to transform streaming data before exporting it to another system using one of the Sink connectors. A good example of a transformation would be to replace the first 12 numbers of a credit card by Xs.

Ideal Workflow (using Control Center GUI):

1. Use one of the Source Connectors to import data from another system to a Kafka Topic

2. Consume & transform the data (with Kafka Streams) from the Kafka Topic and store the aggregated/normalized data in a second topic

3. Use one of the Sink Connectors to export data to another system from the second topic

What would be the best way to achieve this? I already have some code using Kafka Streams (w/ Kafka 0.10.1) that transforms the data but I can't find a way to use it, monitor it and to make it HA using the Control Center.

My current setup is to use Control Center for the Source & Sink connectors then start my Kafka Streams processes by hand so it uses the data imported from the Source connector but that's not really ideal since I have to monitor my own process.

Thanks!