Regarding message order guarantees

[Peter Billen]

Hi,

I have a question regarding message order. Please note I also posted this question on SO. See https://stackoverflow.com/questions/53596761/regarding-message-order-guarantees-in-rabbitmq-amqp.

One of the main characteristics of a message queue service, RabbitMQ included, is preserving message publication order. This is confirmed in the RabbitMQ documentation [1]:

> **[QUOTE 1]** Section 4.7 of the AMQP 0-9-1 core specification explains the

> conditions under which ordering is guaranteed: messages published in

> one channel, passing through one exchange and one queue and one

> outgoing channel will be received in the same order that they were

> sent. RabbitMQ offers stronger guarantees since release 2.7.0.

Let's assume in the following that there are no consumers active, to simplify things. We are publishing over one single channel.

So far, so good.

RabbitMQ also provides possibility to inform the publisher that a certain publication has been completely and correctly processed **[*]**. This is explained on [2]. Basically, the broker will either send a `basic.ack` or `basic.nack` message. The documentation also says this:

> **[QUOTE 2]** `basic.ack` for a persistent message routed to a durable queue will be

> sent after persisting the message to disk.

> In most cases, RabbitMQ will acknowledge messages to publishers in the

> same order they were published (this applies for messages published on

> a single channel). However, publisher acknowledgements are emitted

> asynchronously and can confirm a single message or a group of

> messages. The exact moment when a confirm is emitted depends on the

> delivery mode of a message (**persistent vs. transient**) and the

> properties of the queue(s) the message was routed to (see above).

> Which is to say that different messages can be considered ready for

> acknowledgement at different times. This means that acknowledgements

> can arrive in a different order compared to their respective messages.

> Applications should not depend on the order of acknowledgements when

> possible.

At first glance, this makes sense: persisting a message takes much more time than just storing it in memory, so it's perfectly possibly that the acknowledgment of a later transient message will arrive before a the acknowledgement of an earlier persistent message.

But, if we re-read the first quote regarding message order **[QUOTE 1]**  here above, it gets confusing. I'll explain. Assume we are sending two messages to the same exchange: first a persistent and then a transient message. Since RabbitMQ claims to preserve message order, how can it send an acknowledgment of the second/transient message before it knows that the first/persistent message is indeed completely written to disk?

In other words, does the remark regarding acknowledgement order **[QUOTE 2]**  here above only apply in case the two messages are each routed to completely different target queue(s) (which might happen if they have different routing keys, for example)? In that case, we don't have to guarantee anything as done in **[QUOTE 1]**.

[*] In most cases, this means 'queued'. However, if there are no routing rules applicable, it cannot be enqueued in a target queue. However, this is still a positive outcome regarding publication confirmation.

[1]: https://www.rabbitmq.com/semantics.html#ordering

[2]: https://www.rabbitmq.com/confirms.html#publisher-confirms7

Can anyone shed some lights on this? That would be greatly appreciated.

Thank you.

[Michael Klishin]

RabbitMQ knows message position in a queue regardless of whether it is transient or not.

My guess (I did not write that part of the docs) the ack ordering section primarily tries to communicate that if two messages

are routed to two different queues, those queues will handle/replicate/persist them concurrently. Reasoning about

ordering in *more than one* queue is pretty hard. A message can go into more than one queue as well.

Nonetheless, RabbitMQ queues know what position a message has in what queues. Once all routing/delivery acknowledgements

are received by a channel that handled the publish, it is added to the list of acknowledgements to send out. Note that that

list may or may not be ordered the same way as the original publishes and worrying about that is not practical for many reasons,

most importantly: the user typically primarily cares about the ordering in the queues.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Peter Billen]

Hi Michael,

Thank you for your prompt feedback.

You basically confirmed my thought process. Message publication order is guaranteed for one single queue, but not across multiple queues, which are processed in parallel.

I was confused when reading the docs about publisher confirmations, as the quoted section made me believe that publication order could not be guaranteed for one single queue in vacuum as well. Though, it does not state that explicitly, but I was not sure how to interpret that particular section.

Thank you for clarifying.

Best regards.

[Michael Klishin]

If you have specific ideas as to what paragraphs on the site could use some editing, let us know or submit a PR [1] and we will consider it :)

1. https://github.com/rabbitmq/rabbitmq-website

[Peter Billen]

Will do this later on. (For my other earlier question as well - https://groups.google.com/forum/#!topic/rabbitmq-users/NCgkXviONuQ)

[jiangnan xiang]

I have another question, if a order (such as buy a Apple's  stock) conntains some message reports, these message reports must be ordered, now I use orderId to hash (orderId % 4)make sure the same orderId's message reports put into the same queue.  I use CachingConnectionFactoryone CHANNEL mode, RabbitTemplate send message reports to the queue, can i make sure reports are ordered?

thank you.