RabbitMQ Cluster Computing Resources Usage
Yuri Martins
I have an architecture (image below) with a RabbitMQ cluster and a kind of load balancer in front of the cluster to receive and distribute connections across nodes and the master of the cluster, Haproxy.
Each node has common queues (local queues) and doesn't have quorum queues or mirrored queues. The queues were distributed numerically across the nodes and the master. For example, the master node has 3 queues, node 1 has 3 queues, and node 2 has 3 queues. Each node, including the master node, is in a separate instance in AWS (EC2 Service), but clearly in the same network.
I've been having a problem with resource usage on T3 instances, such as CPU and consumption of credits (they are burstable instances). In random times, a specific node (or two nodes at the same time) begins to work harder (increasing CPU usage) than other nodes, and CPU consumption of the other nodes begins to decrease. Basically, I don't know why it happens.
The problem with this is instability; it is unpredictable to say which node will be stressed. This causes all instance credits to be spent while other nodes are working below common CPU usage.
Given the problems, how could I stabilize cluster CPU usage (after stabilizing the consumption of credits)?