Kafka MassTransit

[Ian Patterson]

Hi,

What does the group think about the suitability of Kafka as a transport for MassTransit ? I was looking at writing code to help integrate Kafka Producers and Consumers and it started to look a bit like MassTransit. So I had the idea to look at implementing a Kafka Transport for MassTransit. Does anyone have an opinion on whether this would be feasible, or why Kafka and MassTransit would not work well together ? Currently it is just an idea, but leveraging an existing library like MassTransit would save time in the long run and also could be useful for other people using Kafka and MassTransit. 

Thanks 

IanP

[Chris Patterson]

Kafka is definitely on my radar, as well as EventHub support in Azure. There are some different patterns to address with both of these transports, in terms of how to track the progress through the stream of messages in the broker. It's likely that a common function set could be harvested into the main assembly for working with streams instead of queues, but most of the work around threading, dispatching, etc. is completely transferrable to the stream-based approach.

So, yeah, if you want to get started that would be great. I know it's on my list of things to do as well.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/75a063f0-2e4e-4ae2-98ef-10061770cdcc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Clay Anderson]

Has there been any more thought or work on this? We are weighing the options of continuing to roll our own or build a Kafka transport for mass transit.

[Chris Patterson]

Kakfa (and event hub) are still on the radar. Nothing has been started yet of which I am aware, however.

If you're willing to be an early adopter and contributor, I'd gladly help with design and implementation work to get it done.

On Thu, Aug 18, 2016 at 8:31 AM, Clay Anderson <superso...@gmail.com> wrote:

Has there been any more thought or work on this? We are weighing the options of continuing to roll our own or build a Kafka transport for mass transit.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/66589528-a926-451c-a6dc-5bc227b578cd%40googlegroups.com.

[Alex Wheat]

Hi guys,

do you have any update on Kafka?

On Thursday, August 18, 2016 at 11:14:09 AM UTC-4, Chris Patterson wrote:

Kakfa (and event hub) are still on the radar. Nothing has been started yet of which I am aware, however.

If you're willing to be an early adopter and contributor, I'd gladly help with design and implementation work to get it done.

On Thu, Aug 18, 2016 at 8:31 AM, Clay Anderson <superso...@gmail.com> wrote:

Has there been any more thought or work on this? We are weighing the options of continuing to roll our own or build a Kafka transport for mass transit.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Mike Moser]

Hey guys,

Wondering about the Kafka transport and where it stands? We are looking at being able to use it through MassTransit to stream some stuff through.

Thanks,

Mike 

[Alfredo Alvarez]

I would like to help on this

[Chris Patterson]

Event Hub support should be coming this year, and that consumption pattern should lead the way towards supporting Kafka as well.

It would be great to open an issue to starting planning out the expectations around how MT should support consuming from topics, since it's a much different model than the pub/sub broker model.

On Thu, Feb 22, 2018 at 11:37 AM, Alfredo Alvarez <alfredo....@gmail.com> wrote:

I would like to help on this

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/08b0f736-0308-4e07-abec-facd658583bc%40googlegroups.com.