Publisher Confirm on queue vs lazy queue

[mprah...@gmail.com]

Suppose I have a 3 node cluster with ha and mirrored queues. I am sending persistent messages to the RabbitMQ node and I'm using publisher confirms for each message. From what I understand, the broker sends a confirm when the persistent message has been persisted to disk and mirrored to all queues.

My questions are - 

1) Does the behaviour remain consistent even with lazy queues? At what point exactly does the broker send a confirm to the producer for a default queue vs lazy queue?

2) Could you please answer the same with respect to using transactions? If I send 1 message and commit it, does the broker send back any ack to the producer, or does it just send message to the consumer on receiving the commit?

3) If there is a client listening to the broker, does the broker send the message to the consumer first without persisting it to the disk?

4) I used default queues and lazy queues, and I noticed that I had a much higher rate of message producer on lazy queues. I know it's difficult to say why given the vague information, but could you have any possible guesses why?

Thank you so much! :)

[Daniil Fedotov]

Hi,

1) Lazy queues confirm after a message was written and synced to disk. The same way as non-lazy persistent messages (with durable queues). The difference is that lazy queue will not create a copy of the message in memory.

2) Transactions follow the same behaviour, with the difference that transaction messages are published on commit all at once. Transaction mode and confirm mode on channel are mutualy exclusive. If you have transaction enabled on a channel - tx.commit-ok means that all persistent messages of this transaction were saved to disk.

If you have a channel with confirm mode - you need to track delivery tags.

3) The broker will try to send a message to consumers as soon as possible, even before it's saved to disk. If consumer is configured with `no-ack` - the message may be removed from the queue and not saved to disk. Otherwise - it will be saved to disk and wait to be acknowledged from the client (reported as unacknowledges messages).

4) Depending on the workload and memory available, a queue process may spend some time organising in-memory messages, while lazy queue does not have those. Lazy queues do less things in general compared to durable queued with persistent messages. This of course depends on your disk, memory and CPU performance.

[mprah...@gmail.com]

For routable messages, the basic.ack is sent when a message has been accepted by all the queues. For persistent messages routed to durable queues, this means persisting to disk. For mirrored queues, this means that all mirrors have accepted the message.

What about persistent messages sent to durable queues in a 3 node cluster with mirrored queues on all 3 nodes - when is basic.ack sent?

From what you say, lazy queue will confirm a message after it was written and synced to a disk. Now, if I have 3 mirrored queues with ha and there is an offline durable subscriber, and messages are being written to disk on Node1. After X minutes, node 1 goes down and the offline durable subscriber comes back online and connects to node 2. Will it receive the messages that were lost on node 1?

I have the above question because I fail to understand when exactly does the broker replicate messages across the mirrored queues in case of a lazy queue for my described setup. 

[Daniil Fedotov]

Hi,

Mirroring and persistence features are orthogonal. Messages sent to mirrored queues will be confirmed when all queue replicas confirm them. Queue replicas confirm messages based on message persistence settings. For lazy queues and persistent messages - after saving to disk.

If you have a mirrored queue - persistent messages will be written to disk on all nodes and confirmed after written on all nodes.

If replica nodes fail or disconnected - they are removed from the waiting list on the channel. If channels does not have any replicas to wait for - it confirms the message. This will work in most cases unless all replicas fail. 

If a queue replica (not master) fails - the queue contents on that node will be removed and recreated and needs to be resynced from the master. If a queue master fails - one of the replicas should be promoted to master (depends on configuration). https://www.rabbitmq.com/ha.html