Currently Recommended Modules

[Greg Saxton]

Hey guys,

Looking though some of past questions in the forum concerning the real time nodes I found several answers suggesting that we should be using the Kafka Indexing service instead and that is why no spec file is provided in the 0.10+ releases.  Can I get a confirmation on this and an update as to any other druid components that should be preferred over other existing druid components?  Could this information also be provided in the documentation on Druid.io?

Thank you.
Gregory

[Gian Merlino]

Hey Greg,

Realtime ingestion in Druid has gone through a few iterations (realtime nodes -> tranquility -> kafka indexing service with some detours along the way). Right now (Druid 0.10 timeframe) we are in a transitionary period between recommending tranquility and recommending the kafka indexing service. I would say that if the kafka indexing service meets your needs and you are ok with a "rapidly evolving" ingestion scheme, you should go with that. It is where most of the effort is going now, I think it's the best approach we have taken so far, and I don't see any reason on the horizon to want to replace it with something else. It has nice operational properties compared to the other two.

If you prize stability above all else then I would go with tranquility at this point in time. Although I expect this advice will change before too long, to just going with the kafka indexing service always.

Definitely agree the docs on druid.io could explain this better. I blame the transitionary period I mentioned earlier :)

Gian

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/bb7acc52-b333-49c0-bacd-5e7a73d24969%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Greg Saxton]

Thanks Gian, This was very helpful!

On Thursday, November 2, 2017 at 1:35:20 AM UTC-4, Gian Merlino wrote:

Hey Greg,

Realtime ingestion in Druid has gone through a few iterations (realtime nodes -> tranquility -> kafka indexing service with some detours along the way). Right now (Druid 0.10 timeframe) we are in a transitionary period between recommending tranquility and recommending the kafka indexing service. I would say that if the kafka indexing service meets your needs and you are ok with a "rapidly evolving" ingestion scheme, you should go with that. It is where most of the effort is going now, I think it's the best approach we have taken so far, and I don't see any reason on the horizon to want to replace it with something else. It has nice operational properties compared to the other two.

If you prize stability above all else then I would go with tranquility at this point in time. Although I expect this advice will change before too long, to just going with the kafka indexing service always.

Definitely agree the docs on druid.io could explain this better. I blame the transitionary period I mentioned earlier :)

Gian

On Wed, Nov 1, 2017 at 7:46 AM, Greg Saxton <sdgrego...@gmail.com> wrote:

Hey guys,

Looking though some of past questions in the forum concerning the real time nodes I found several answers suggesting that we should be using the Kafka Indexing service instead and that is why no spec file is provided in the 0.10+ releases.  Can I get a confirmation on this and an update as to any other druid components that should be preferred over other existing druid components?  Could this information also be provided in the documentation on Druid.io?

Thank you.
Gregory

--
You received this message because you are subscribed to the Google Groups "Druid User" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.