Quorum queues specify master and member node

[Soumen K]

Hi, 

I have a 3 node RMQ cluster and I want to declare 3 queues such that each of them are on a different node for master. For example - 

Queue 1 : Node-1 (Master) + Node-2,Node-3 members

Queue 2 : Node-2 (Master) + Node-1,Node-3 members

Queue 3 : Node-3 (Master) + Node-1,Node-2 members

When I am doing it like the below, all the queues are having master on the same node - 

rabbitmqadmin declare queue name=QUEUE1 durable=true auto_delete=false arguments='{"x-queue-type": "quorum","x-delivery-limit": "3","x-quorum-initial-group-size":3}'

How do I specify the master node and member nodes for a particular queue ? 

Thanks in advanced !

[Michal Kuratczyk]

Hi,

1. Please don't post the same question multiple times.

2. Please use the correct names: leader and follower(s)

Can you explain why you want to do that? What do you expect to happen if you upgrade/restart the cluster? The whole point is that the leader's role can be moved to another node.

Best,

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/dc426c03-c678-4046-a64c-911bca450e78n%40googlegroups.com.

--

Michał

RabbitMQ team

[Soumen K]

Hi, 

The reason behind this is that when all the leaders are on the same node (NODE-1 in this case), the entire load is on the NODE-1 when the other nodes are idle. So if each queue has its own separate leader, the work load will be well distributed in terms of CPU and mem utilisation also. Thanks. 

[Michal Kuratczyk]

Rather than manually assigning leaders, you can rebalance the queues to spread the leaders between nodes:

https://www.rabbitmq.com/rabbitmq-queues.8.html#rebalance

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/d71d4d16-2c33-45b5-b417-d08f994d731fn%40googlegroups.com.

--

Michał

RabbitMQ team

[Karl Nilsson]

It is true that leaders do more work than followers but it isn't correct that all work is done by leaders. For example if you have a consumer connected to a node with a follower the follower will do the message delivery, not the leader.

As Kura pointed out occasional rebalancing is the correct approach as is not worrying too much unless you have tested a representative workload and found it causes an issue.

Cheers

Karl

[Soumen K]

OK Thanks for your suggestions and advises guys. appreciate it ! 

There is another reason why I want to do this. In the production system, we have 6 Rabbit VMs on 3 physical servers. Now if I cant specify the leaders and the replica's location for the queues myself, the system may not survive a physical machine being down (i.e. 2 underlying VMs will be gone). Is there a way to do this anyway for quorum queues ? Earlier, when we had classic queues, it was done via some policies and it was really simple. 

[Karl Nilsson]

You can use quorum queues with 5 members to support 2 failures.

https://www.rabbitmq.com/quorum-queues.html#replication-factor

[Luke Bakken]

Hello,

The best practice here is to have 3 RabbitMQ VMs, each pinned to a physical server. Clusters should have an odd number of nodes so that quorum can be calculated.

Thanks,

Luke