Rabbit MQ with multiple workers & request/response questions?

[Josh Handel]

Howdy all,

   I have an environment with multiple services that coordinate to handled a rather large workflow.. I may want to independently scale various queues (for instance inventory management service may need 3 nodes, while order processing may only need 1)...  I am using RabbitMQ with MassTransit.. Is doing this just as simple as pointing all of the workers at the same queue for the given service (so point all of the Inventory management workers at the same queue path)....

Second question is about multiple workers where a worker may do a request from another service.. (example being say the Order Processing service has 4 workers, and one of them calls the Inventory Service via a Request/Response to determine if something is in stock)..  Is there any risk that Worker A would make the request, but Worker B would pick it up?   Or is Rabbit & Mass Transit smart enough to know that Worker A made the request (even though they are both using the same queue path) so only Worker A will pick up the response message?

*Really what I want to be able to do right now, is spin up 2 or 3 VMs across multiple failure points so that our services are resilient to failure, but these questions would answer if I can do that or not :-).

Thanks in advance,

Josh

[Travis Smith]

Pointing multiple consumers (that are exactly the same) at the same end point would give you load balancing in RabbitMQ. Easy. 

Allowing those consumers to do request/response is harder. Any endpoint could pickup the message and say "not mine, error queue for you". So likely what you need is a secondary bus, with a different queue backing it for each consumer, that supports the request/response.  

-Travis

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/fc683191-a7ea-4fa7-a0de-5af6b459ec9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Prothero]

Josh,

I recently wrote about the first part of your question...

http://looselycoupledlabs.com/2014/08/scaling-out-subscribers-with-masstransit/

The second part sounds interesting. Let us know if you try Travis' suggestion.

David

[Udi Dahan]

Alternatively, you could use a saga to preserve state which would then hold across multiple workers.

 

-- Udi

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/CAAqUTbkqi_a0ZV_z%3DWN4w81kd%2BoQVG9UmoEn7K9U%2B_VmqQehXg%40mail.gmail.com.

[Josh Handel]

sounds like all the services should share the same queues for when they subscribe for consumption.. but they should all have unique "send" queues so the path to respond is unique to each worker..

I'll let you guys know how this works for us :-).

Thanks

Josh

On Tuesday, August 12, 2014 12:36:40 PM UTC-5, Travis Smith wrote:

Pointing multiple consumers (that are exactly the same) at the same end point would give you load balancing in RabbitMQ. Easy. 

Allowing those consumers to do request/response is harder. Any endpoint could pickup the message and say "not mine, error queue for you". So likely what you need is a secondary bus, with a different queue backing it for each consumer, that supports the request/response.  

-Travis

On Tue, Aug 12, 2014 at 10:28 AM, Josh Handel <jha...@gmail.com> wrote:

Howdy all,

   I have an environment with multiple services that coordinate to handled a rather large workflow.. I may want to independently scale various queues (for instance inventory management service may need 3 nodes, while order processing may only need 1)...  I am using RabbitMQ with MassTransit.. Is doing this just as simple as pointing all of the workers at the same queue for the given service (so point all of the Inventory management workers at the same queue path)....

Second question is about multiple workers where a worker may do a request from another service.. (example being say the Order Processing service has 4 workers, and one of them calls the Inventory Service via a Request/Response to determine if something is in stock)..  Is there any risk that Worker A would make the request, but Worker B would pick it up?   Or is Rabbit & Mass Transit smart enough to know that Worker A made the request (even though they are both using the same queue path) so only Worker A will pick up the response message?

*Really what I want to be able to do right now, is spin up 2 or 3 VMs across multiple failure points so that our services are resilient to failure, but these questions would answer if I can do that or not :-).

Thanks in advance,

Josh

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

[Chris Patterson]

If you're need to capture state and stay on a "thread", the request/response will be fine if you have a unique queue for each worker (with RabbitMQ, just create another bus using rabbitmq://server/*?temporary=true and you get a temporary queue that disappears.

If you're just coordinating with the "response" - using a saga is a better approach as the next available worker will receive the response and read the saga from storage and "continue" where it left off. Using Automatonymous is a great way to model this situation and is part of the Riktig example (https://github.com/phatboyg/Riktig).

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/50b96db5-2e43-4d9d-b17e-2970878067ef%40googlegroups.com.