Re: [masstransit-discuss] Design and Implementation Advice

[Dru Sellers]

initial questions / thoughts inline

On Fri, May 3, 2013 at 11:20 AM, João Lourenço <blacks...@gmail.com> wrote:

Hi there,

I am currently working on choosing and implementing a service bus with a colleague at work. Our current systems design is as follows:

                                         Load Balancer

        ________________________|________________________

       |                                          |                                          |

Web server                           Web server                          Web server

       |________________________|________________________|

                                                  |

                                        Load Balancer----------------------------------------------------Applications

        ________________________|________________________

       |                                          |                                          |

App server                            App server                           App server

Our services all run on the app servers, and the load balancer in bold routes the various requests from web servers, applications and other services. We are investigating the addition a service bus that will possibly replace this load balancer.

So the web servers would publish events or send commands that would get processed by the app servers. Your goal is that the routing of these 'messages' would allow you to remove the 'Load Balancer'

 

Our current line of thinking is as follows: the various services on the app server all act as workers that will do work based on a message that is received i.e. once a message (command?) has been picked up off the message queue, it should not be available to any other workers to redundantly process again.

ok, good.

 

Once the work has been finished, the aforementioned services will need to publish an event that the work has been done. The other services and applications also need to act as subscribers and display results/alerts when applicable events are picked up i.e. the messages will need to be processed by all recipients, unlike the commands mentioned before.

sounds like these are 'events'

 

With the testing that we've done with MassTransit (with MSMQ), we have successfully built toy bus applications on separate computers that talk to each other and send messages between the two using the SetNetwork(...) and adding subscription URIs and receive URIs. This works well for the publish/subscribe model outlined above, but only temporarily as once a service [1] goes down and starts up again, the other service [2] stops picking up the messages when [1] comes back online. It does not, however, address the requirement of only one worker picking up a command message.

have you thought about using RabbitMQ instead of MSMQ?

I would recommend have competing consumers for your command messages - this would allow you to have N services but still only have one process each command.

 

One of the ideas behind this is that if one app server goes down, the others will easily pick up the messages and divvy up the messages between them until the server comes back up. In my mind, this suggests some sort of shared message queue (having a master queue kind of defeats the purpose, because that machine could go down).

With MSMQ this would indeed be a problem, but if you looked at RabbitMQ I think you can beat this very easily. I forget if its federation or what, but I know that this can be done.

 

If anyone has any advice about this, please would you respond? The deployment of this bus and how to connect client applications and web servers to the bus would also be invaluable.

So I would suggest a load clustered / federated (?) RabbitMQ server would be a great starting point. From there you can spin up N app servers to process the messages in the RabbitMQ Queues. I would start with the web apps sending commands using 'bus.GetEndpoint('command endpoint').Send(msg)' from there the N app servers would process them as needed. When the app servers are done with their work they would publish events 'bus.Publish(msg)' now other app servers can listen for that event and do the follow on work as necessary.

I hope that helps.

-d

 

Thanks in advance!

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/LKSKiYDFXe8J.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Tim Gebhardt]

Not to sound too dramatic, but if you can use RabbitMQ it'll solve all your problems :).

You can set up RabbitMQ in a clustered configuration in about 5 minutes:
http://coding-insomnia.com/2013/02/23/clustering-rabbitmq-on-windows/

Recommended configuration (since you don't really have too many servers): Setup RabbitMQ service on each server, cluster them, and each application hooks up to the localhost instance of RabbitMQ and let the RabbitMQ handle the routing and sending messages to where they need to go.

This solves the problem with a single point of failure "master queue" as you referred to it.  All you have to do is set the important queue as a highly available queue and you can specify the degree of highly available-ness (that's just a word I made up now) per policies -- replicate to all cluster nodes, replicate to N number of nodes, replicate to X, Y, Z nodes, etc.

The next step for load balancing would just be for your command consumers to point themselves to the same MassTransit endpoint URI.  That'll set them up in a competing consumers situation (as Dru mentioned) where they will each take the messages off the queue and distribute the work among themselves.

You still need a unique URL per application instance so you can direct messages to individual application instances or so they can each get their own copies of the events you publish.

So as an example your application servers would have at least two connections to RabbitMQ:

AppServer 1: rabbitmq://localhost/command_queue?ha=true, rabbitmq://localhost/app_server_1
AppServer 2: rabbitmq://localhost/command_queue?ha=true, rabbitmq://localhost/app_server_2

Wire up your app so that your command handler listens on the first endpoint, but your event handlers are hooked up to listen to the second endpoints and this should do what you explained in your post.

On Friday, May 3, 2013 11:20:28 AM UTC-5, João Lourenço wrote:

Hi there,

I am currently working on choosing and implementing a service bus with a colleague at work. Our current systems design is as follows:

                                         Load Balancer

        ________________________|________________________

       |                                          |                                          |

Web server                           Web server                          Web server

       |________________________|________________________|

                                                  |

                                        Load Balancer----------------------------------------------------Applications

        ________________________|________________________

       |                                          |                                          |

App server                            App server                           App server

Our services all run on the app servers, and the load balancer in bold routes the various requests from web servers, applications and other services. We are investigating the addition a service bus that will possibly replace this load balancer.

Our current line of thinking is as follows: the various services on the app server all act as workers that will do work based on a message that is received i.e. once a message (command?) has been picked up off the message queue, it should not be available to any other workers to redundantly process again. Once the work has been finished, the aforementioned services will need to publish an event that the work has been done. The other services and applications also need to act as subscribers and display results/alerts when applicable events are picked up i.e. the messages will need to be processed by all recipients, unlike the commands mentioned before.

With the testing that we've done with MassTransit (with MSMQ), we have successfully built toy bus applications on separate computers that talk to each other and send messages between the two using the SetNetwork(...) and adding subscription URIs and receive URIs. This works well for the publish/subscribe model outlined above, but only temporarily as once a service [1] goes down and starts up again, the other service [2] stops picking up the messages when [1] comes back online. It does not, however, address the requirement of only one worker picking up a command message.

One of the ideas behind this is that if one app server goes down, the others will easily pick up the messages and divvy up the messages between them until the server comes back up. In my mind, this suggests some sort of shared message queue (having a master queue kind of defeats the purpose, because that machine could go down).

If anyone has any advice about this, please would you respond? The deployment of this bus and how to connect client applications and web servers to the bus would also be invaluable.

Thanks in advance!

[João Lourenço]

Thanks so much for the detailed responses. I am glad that we were at least partly on the right track :). We have successfully got the messaging queueing working as we expected. We went with Tim's suggestion of having clustered nodes with mirrored queues. We are just going to write a few test consumers/producers and publishers/subscribers and do some tests to see if we have any further questions.