How might I sink JSON to a Mysql DB using Kafka Connect?

[andrewdb]

I've a Kafka Streams app that's outputting JSON to a topic in Kafka, and I'd like to save the output JSON to a mysql table.

Here's how I'd like to configure my data flow:

Input JSON -> input topic -> Kafka Stream transformer -> output topic (JSON) -> Kafka connect -> mysql DB.

I've implemented the above up until Kafka connect, as I can't find much documentation on using Kafka Connect to save from JSON to a mysql database.

The examples I've seen so far with Kafka Connect seem to work with the avro data format, not JSON.

Can anyone point my in the right direction?  

[Robin Moffatt]

Kafka Connect lets you specify the Converter to use. So if your data is in JSON, you can use the JsonConverter :) 

You can specify this in your connector configuration, or at the worker level if you want. 

                "key.converter": "org.apache.kafka.connect.json.JsonConverter",

                "key.converter.schemas.enable":"false",

                "value.converter": "org.apache.kafka.connect.json.JsonConverter",

                "value.converter.schemas.enable": "false",

If you have a schema embedded in your JSON (as schema and payload root objects, per the JSON generated by Connect if you specify the JsonConverter as the source) then set enable=true in the above config. This is an example of such a format: https://gist.github.com/rmoff/2b922fd1f9baf3ba1d66b98e9dd7b364

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/6dd0991d-a2a0-45b1-83b7-7cb2ee7a4570%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.