competing consumers MT3

155 views
Skip to first unread message

Brian Weeres

unread,
Aug 19, 2015, 4:42:51 PM8/19/15
to masstransit-discuss
I just want to confirm something is working like it should because I think my understanding was incorrect.

I always thought consumers consuming the same message on the same queue were competing regardless of what bus instance they were registered on but that does not seem to be the case.

1. If I have two consumers consuming the same message and using the same queue and are registered as endpoints on the same BUS they will both get the message.
2. If I have two consumers consuming the same message and using the same queue and  are registered as endpoints on different BUS instances they compete for the message.
3. If I have two consumers consuming the same message and using different queues and  are registered as endpoints on different BUS instances they will both get the message.

I think what I am seeing makes sense and is okay because we never have scenario one but just wanted to make sure it is the expected behaviour. 

Thanks,
Brian


Chris Patterson

unread,
Aug 19, 2015, 6:45:03 PM8/19/15
to masstrans...@googlegroups.com
1. This seems wrong, I'll verify, but two separate ReceiveEndpoints on a single bus with the same queue name should compete. A single receive endpoint with two consumers for the same message type should both get the message.
2. Yes.
3. If you are publishing the message (versus sending directly to a queue), this is correct.


--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/813c0385-a25c-4b27-baa2-519db8e37f4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Brian Weeres

unread,
Aug 20, 2015, 9:42:32 AM8/20/15
to masstransit-discuss
In regards to number one I am registering the consumers under one ReceiveEndpoint. If I register each consumer under a separate ReceiveEndpoint but the same queue then they compete which is what you said you expect.
So to update the summary: 

1. Two consumers consuming the same message and using the same queue and are registered in the same endpoint on the same BUS they will both get the message.
1a. Two consumers consuming the same message and using the same queue and are registered on different endpoints on the same BUS they will compete for the message.
2. Two consumers consuming the same message and using the same queue and  are registered as endpoints on different BUS instances they compete for the message.
3. Two consumers consuming the same message and using different queues and  are registered as endpoints on different BUS instances they will both get the message.

This leaves me now wondering if we should change how we register our consumers. Should it be one ReceiveEndpoint, one consumer or should we continue registering multiple consumers under one ReceiveEndpoint?
We currently group consumers by functionality or domain into one assembly and that assembly has a plugin that creates a bus instance, a ReceiveEndpoint with its own queue and discovers and registers all consumers in that assembly in that endpoint.
I could easily change that to create multiple endpoints on that bus instance with one consumer per endpoint. I could also either have all the endpoints in this assembly use a different queue instead of the same one. Not sure what the tradeoff is. Beginning to think one endpoint, one consumer, one queue may not be so bad. Not sure if that causes a lot of overhead in Rabbit / MT or less overhead.

So many options.






















On Wednesday, August 19, 2015 at 5:45:03 PM UTC-5, Chris Patterson wrote:
1. This seems wrong, I'll verify, but two separate ReceiveEndpoints on a single bus with the same queue name should compete. A single receive endpoint with two consumers for the same message type should both get the message.
2. Yes.
3. If you are publishing the message (versus sending directly to a queue), this is correct.

On Wed, Aug 19, 2015 at 1:42 PM, Brian Weeres <bwe...@gmail.com> wrote:
I just want to confirm something is working like it should because I think my understanding was incorrect.

I always thought consumers consuming the same message on the same queue were competing regardless of what bus instance they were registered on but that does not seem to be the case.

1. If I have two consumers consuming the same message and using the same queue and are registered as endpoints on the same BUS they will both get the message.
2. If I have two consumers consuming the same message and using the same queue and  are registered as endpoints on different BUS instances they compete for the message.
3. If I have two consumers consuming the same message and using different queues and  are registered as endpoints on different BUS instances they will both get the message.

I think what I am seeing makes sense and is okay because we never have scenario one but just wanted to make sure it is the expected behaviour. 

Thanks,
Brian


--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

Chris Patterson

unread,
Aug 20, 2015, 11:10:51 AM8/20/15
to masstrans...@googlegroups.com
So, just a little guidance based on the experiences we have had in production.

For most consumers that are doing "a service style activity" we use a queue for each consumer. This prevents messages of one type from getting stacked behind messages of another type. For instance, if you had Order Update and New Order in the same queue, and there were 10x order updates to new orders, the new orders could get delayed behind order updates. There is no priority, so you manage consumers per queue to ensure that a high volume of potentially lower priority messages don't end up preventing lower frequency but higher priority messages from being processed quickly.

In the case of events, particularly those used for auditing or tracking, those often get dumped into a single queue with many consumers on it (one consumer for each set of event types). That way, a single queue just chews away and dumps those into MongoDB or SQL or whatever store is being used. The caveat to this, however, is that we ended up splitting this into separate queues because...

Some of the events were operational events (such as those published by Courier routing slips -- activity and routing slip events) and others were business level events (existing record matched, invalid data in message content, etc.). We ultimately split the business events from the operational events, a couple of times actually, because under heavy system load the non-critical events were drowning out the important business events that were needed to decision support. It also helped reduce the load on RabbitMQ. We made the operational event queue non-HA, but kept the business event queue HA, so that we would not lose business information if a broker failed but the operational information was fine to discard (it's actually not discarded, the queue is still there when the node comes back up as it isn't HA).

For routing slip activities, we definitely have separate execute and compensate queues, and they're unique for each activity. This is because Courier uses a single message type (RoutingSlip) and there is no way other than endpoint to dictate behavior.

For sagas, every saga should always have its own queue, period. Otherwise it's just crazy. Sagas typically subscribe to many different event types, so they really need their own queue. A coming feature in MT3 is the ability to ensure that sagas correlated to specific ids are only executed on specific nodes (ala consistent hashing) to improve node affinity for certain performance reasons (mainly around distributed state storage).

Does that help?




To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/b59b1cbd-2979-4ddf-a697-0d3ea509bd44%40googlegroups.com.

Brian Weeres

unread,
Aug 20, 2015, 12:01:46 PM8/20/15
to masstransit-discuss
It does help. I am going to change our default model to one queue per consumer since MT3 now supports multiple receiveendpoints on one servicebus . 

We did always put each saga in its own queue. One note regarding sagas is we find later steps of sagas get backed up behind messages that will start new sagas. In some cases we would like events that complete a saga to take priority over messages starting a new saga or be processed in parallel. We ended up using routing slips because of that issue. It would be nice to have the ability to have a different queue per event that a saga subscribes to.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

Chris Patterson

unread,
Aug 20, 2015, 2:36:32 PM8/20/15
to masstrans...@googlegroups.com
Ooo, that's a nice feature request. :)

I assume that this is for Automatonymous sagas, right?


To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/f8c3556f-59e6-4b18-9dd1-31c283f2554d%40googlegroups.com.

Brian Weeres

unread,
Aug 25, 2015, 10:17:30 AM8/25/15
to masstransit-discuss
Yes, we have switched to Automatonymous sagas.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages