pub/sub vs competing consumers

[Tomer Yoskovich]

Hello,

I want some messages to be consumed using the competing consumers - I do it by simply "connecting" multiple consumers to the same endpoint (=queue).

If I understood it correctly, it means that if I want *all* the consumers to receive a certain message, then I will need to have another endpoint (=queue) for each of them (and then MT will connect the publishing exchange to all the endpoints thereby pub/sub)

Is my understanding correct? is there another way of achieving this?

Thanks

[Wayne Brantley]

Correct.

Say you create a windows service.  In that service - you set the endpoint name to "MyService".

If you start 3 copies of this windows service on 3 machines....they will all listen to the same queue...and end up being competing consumers.

Now, if you create a new windows service with an endpoint of "MyNewService" and set that endpoint to use the same consumers as above - you will end up with the MyNewService getting a copy of every message...while the MyService installs compete for a single copy.

Typical design would be to create a new queue for every 'application/service' you are deploying.   Everyone gets their own copy of the messages.  If you want to scale out - you simply run more copies of your application/service.

[Tomer Yoskovich]

Hi Wayne, Thanks for your input, it was as I expected, maybe you can help me with a dilemma I'm facing related to this:

Currently I am using reflection over the assemblies in the executing assembly to find all the consumers and register them all to the same endpoint. (so if a class inherits from IConsumer)

Given this new consideration I will need to register them to more than one queue.

Do you have a recommendation as to what pattern I should use so that by reflection I will register them to the different endpoints ?

[Wayne Brantley]

We have this same issue!   I ended up registering each consumer manually.   For us our consumers handle many messages.  This is because our consumers take a single dependency and simply map messages into cqrs query/commands - so they are a simple proxy.   This means most consumers are one line of code.  So we have a single consumer per queue.  This make it easy to register one consumer per queue and done.

[Chris Patterson]

Yes, that is how publish/subscribe works. You create multiple receive endpoints (which maps to a queue), each with a separate queue name. You then register each consumer on a unique endpoint. Each consumer will get a copy of the message to process all by itself. This was if one consumer takes five seconds versus a quicker consumer than only takes one second, the processing does not back up behind the slowest consumer.

 

Competing consumer is where you run the same service (behavior, logic, etc.) with the same queue with multiple instances (across machines, typically). This provides both load balancing and higher availability.

 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/65ce7d9a-c4f3-4d5a-80a5-bed1f83006b8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.