When using InfluxDB as a data sink, should Telegraf be omitted from the TICK stack when used w KC?

[Baden]

I am new to the InfluxDB landscape and am in the process of planning out a data ingestion infrastructure that will consume search and time-series data.

Using Confluent Kafka Connect to consume incoming data is the basic model I am following for search-related data.  However the TICK stack uses Telegraf for ingesting data.

I would like to know form those with experience with this if I should exclude Kafka for this time-series data and/or what the best implementation solution would be?

I wanted to use Kafka as the micro-services broker for all comms.

[Ewen Cheslack-Postava]

If you use Connect to move data from Kafka into InfluxDB (e.g. via this connector http://docs.datamountaineer.com/en/latest/influx.html), then you wouldn't need Telegraf since Connect would handle that step for you.

As for the choice of the stack, I am obviously biased, but Connect has a bunch of features that make it nice for this use case such as easy scalability, fault tolerance, delivery guarantees, pluggable serialization formats, and more listed in http://docs.confluent.io/3.1.1/connect/intro.html Kafka also works really well for real time data integration, so you'll probably have more uses that Telegraf (which targets metrics) may not be a great fit for. Preferring Connect may reduce the number of technologies you need to learn & the number of systems you need to monitor.

-Ewen

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/a2b5fd0f-3073-4ca9-b5ba-b63c0d486638%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.