Advise on RabbitMQ clustering and load balancer

[4integ...@gmail.com]

Hi,

We will setup a RabbitMQ cluster to evaluate the solution.

Some notes on my understanding

Queues belongs to the node where they were created. That node is master of the queue but not necessary the master node in RabbitMQ cluster.

Thinking of placing a load balancer (F5) in front of the cluster to make it easy for applications and users to connect to the RabbitMQ cluster - without knowing the details of the cluster. 

There are performance benefits of connecting to the master node.

There are no benefit of running round robin load balancer in the cluster - better to have active/passive.

So I am thinking

- Have 2 RabbitMQ nodes in a cluster with HA of all artifacts (replication of queues, exchanges,messages,..etc)

- Create all artifacts in the RabbitMQ master

- Configure load balancer to checks heart beats of the master node

- If the master node is responding direct all traffic to that node (active). If the "master" node does not respond set the "slave" (now master) as active

Is this the best approach?

Other better alternatives?

/ Joacim

[Michael Klishin]

There are no master nodes in RabbitMQ. I'd recommend 3 nodes (or any odd number)

instead of 2.

Proxies or load balancers in front of clusters is a common thing to see. It has its own downsides

but is good enough for many users.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[4integ...@gmail.com]

Hi again,

Vacation got in the way for the discussion :)

What is the reason/best practice for choosing clusters with odd number of nodes instead of even numbers?

Ok, there is no master nodes as such but queues are created on one node and queues can be mirrored to other nodes. That what I meant by "That node is master of the queue..".

Since the documentation says "All queue operations go through the master first and then are replicated to mirrors. This is necessary to guarantee FIFO ordering of messages."; does that mean that load balancer should run all traffic to the queue master?

If we create all queues in one node (#1 being our "master node" ..even though there is no such thing :) ). In case of "master node" #1 have issues another node (e.g. #2 will be promoted as "master node" (or queue master) - how can the load balancer be aware that it should run traffic to #2 ? 

It might be stupid questions here but need to know.

In order to setup a good high available RabbitMQ solution - is there any blue print/reference setup/cookbook available? 

/ Joacim

On Tuesday, June 20, 2017 at 4:09:01 PM UTC+2, Michael Klishin wrote:

There are no master nodes in RabbitMQ. I'd recommend 3 nodes (or any odd number)

instead of 2.

Proxies or load balancers in front of clusters is a common thing to see. It has its own downsides

but is good enough for many users.

On Tue, Jun 20, 2017 at 3:42 PM, <4integ...@gmail.com> wrote:

Hi,

We will setup a RabbitMQ cluster to evaluate the solution.

Some notes on my understanding

Queues belongs to the node where they were created. That node is master of the queue but not necessary the master node in RabbitMQ cluster.

Thinking of placing a load balancer (F5) in front of the cluster to make it easy for applications and users to connect to the RabbitMQ cluster - without knowing the details of the cluster. 

There are performance benefits of connecting to the master node.

There are no benefit of running round robin load balancer in the cluster - better to have active/passive.

So I am thinking

- Have 2 RabbitMQ nodes in a cluster with HA of all artifacts (replication of queues, exchanges,messages,..etc)

- Create all artifacts in the RabbitMQ master

- Configure load balancer to checks heart beats of the master node

- If the master node is responding direct all traffic to that node (active). If the "master" node does not respond set the "slave" (now master) as active

Is this the best approach?

Other better alternatives?

/ Joacim

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Steve Smith]

Ok, there is no master nodes as such but queues are created on one node and queues can be mirrored to other nodes. That what I meant by "That node is master of the queue..".

Since the documentation says "All queue operations go through the master first and then are replicated to mirrors. This is necessary to guarantee FIFO ordering of messages."; does that mean that load balancer should run all traffic to the queue master?

My understanding is that the balancer will introduce the client to one of the nodes, not necessarily the 'master' for a given queue. RabbitMQ causes the slave queue to forward the message to the master queue. Beware unexpected amounts of extra network hops happening in this case in your solution.

 

If we create all queues in one node (#1 being our "master node" ..even though there is no such thing :) ). In case of "master node" #1 have issues another node (e.g. #2 will be promoted as "master node" (or queue master) - how can the load balancer be aware that it should run traffic to #2 ? 

All your messages will land on the master queue having been routed through the slave queue that introduced it (see above, loads of extra network hops.

In order to setup a good high available RabbitMQ solution - is there any blue print/reference setup/cookbook available? 

I long for a windoze based cookbook for RabbitMQ - I really hate everything assuming its unix/java based...

 

[4integ...@gmail.com]

Thanks for the reply.

Understood about the extra network hops.

To have a high available solution transparent to applications I see two alternatives (please add more alternatives if you have ideas)

1. RabbitMQ cluster with round robin

2. RabbitMQ cluster with active/passive

I created a few pictures about it and some pros/cons. I appreciate if you add pros/cons or other input.

https://www.dropbox.com/s/fh5xkg8gekffo0p/RabbitMQ%20-%20Load%20Balancing%20%26%20HA.pdf?dl=0 

/ Joacim

[4integ...@gmail.com]

Hi,

No comments/input on the previous post and attachment?

Also would like to know why it's recommended with odd number of RabbitMQ nodes in a cluster? 

/ Joacim

[Arnaud Cogoluègnes]

It's better to have an odd number of nodes in case of partition: there can't be any partition with the same number of nodes. This usually makes the work of the partition handling algorithm easier, as there's always a partitioned cluster with more nodes than the other. 

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

[V Z]

So, what happens when 9 node cluster splits 3 ways? How is it different from 4 node cluster splitting in half?

Another example: assume 5 node Rabbit cluster runs in a PaaS with two AZs -- 3 nodes in AZ1 and 2 nodes in AZ2. AZ1 goes down due to underlying IaaS problem. Will the remaining 2 nodes in AZ2 continue to work or shut down (they are a minority after all)?