Question on duplicate messages when multiple consumers run in RabbitMQ cluster

[bhalchandr...@loanbeam.com]

Hi

I have set up RabbitMQ cluster. There are multiple consumer processes running on different windows servers that read messages from nodes in the cluster. I am passing list of IP addresses for the nodes like below in each consumer process to create connection.

connection = factory.CreateConnection(rabbitMQNodesList)

I have set noAck = true while reading message from queue.

 BasicGetResult result = channel.BasicGet(QueueName, true);

My question is. Is it possible for two different consumers to read same message. i.e. duplication of message while reading. First consumer process reads a message, Second consumer reads the same message before RabbitMQ can remove that message from the queue. Is this scenario possible. How do you handle it.

Thanks
Bhal

[Luke Bakken]

Hi Bhal -

In the scenario you describe, the second consumer won't read the same message. Since you're not using acknowledgements, RabbitMQ will immediately consider the message successfully delivered and remove it from the queue.

There are cases when acknowledgements are used where messages can be re-delivered in error scenarios. You can read about it here - https://www.rabbitmq.com/reliability.html

Thanks,

Luke

[bhalchandr...@loanbeam.com]

Hi

 Thanks for the reply. You  mentioned message will be "Immediately" remove from the queue. What does

 “immediately” mean. The two queues are in the cluster with two nodes running on different machines and mirrored.

 

Machine1ß---à Machine 2

S1    S2                    S3    S4

 

 

S1 and S3 see the same massage ( because the cluster is replicating them) and pick them at the same time. How do we guard against it? At what granular level can RabbitMQ distinguish two consumer accessing messages at same time in mirrored queues.

Thanks

Bhalchandra

[Michael Klishin]

There is a fundamental flaw in your reasoning. You assume that mirroring means

that deliveries are also mirrored. As in, two consumers on two replicas of a queue will each get

a copy of every message flowing through that queue.That's NOT the case.

Queue *contents* are mirrored but every operation, be it enqueueing of a published message

or delivery, goes through the queue master node first. Which will make sure that only one consumer

gets a particular message at a time.

A couple of relevant doc links:

http://www.rabbitmq.com/clustering.html#cluster-membership

http://www.rabbitmq.com/ha.html#non-mirrored-queue-behavior-on-node-failure

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ