Guaranteed ordered delivery of messages in case of unexpected events (RabbitMQ 3.6.x)

[Behrang Saeedzadeh]

Hi,

In the RabbitMQ docs it is mentioned that:

Section 4.7 of the AMQP 0-9-1 core specification explains the conditions under which ordering is guaranteed: messages published in one channel, passing through one exchange and one queue and one outgoing channel will be received in the same order that they were sent. RabbitMQ offers stronger guarantees since release 2.7.0.

 

Messages can be returned to the queue using AMQP methods that feature a requeue parameter (basic.recover, basic.reject and basic.nack), or due to a channel closing while holding unacknowledged messages. Any of these scenarios caused messages to be requeued at the back of the queue for RabbitMQ releases earlier than 2.7.0. From RabbitMQ release 2.7.0, messages are always held in the queue in publication order, even in the presence of requeueing or channel closure.

 

With release 2.7.0 and later it is still possible for individual consumers to observe messages out of order if the queue has multiple subscribers. This is due to the actions of other subscribers who may requeue messages. From the perspective of the queue the messages are always held in the publication order.

In our case, we have a queue (durable) that only has one single-threaded consumer. We need guaranteed ordered delivery of messages, even when one of the following scenarios happen:

• The RabbitMQ cluster or a subset of its nodes crashes (I think we have 6 nodes in the cluster)
• The consumer application crashes
• The channel is closed due to other events (e.g. networking issues)

We are using Spring AMQP. Is it safe to use a prefetch count of greater than 1 (e.g. 50) in our scenario?

There's a bit of uncertainty in this regard in our team.

P.S: is there a way to configure the consumer client to throw an exception (or notify us in any other way) that a message is delivered out of order? For example some RDBMS systems assign a monotonic increasing implicit ROW_ID to rows. Does RabbitMQ/AMQP have a similar concept?

Thanks in advance,

Behrang

[Michael Klishin]

The scenarios you've described should not affect ordering with a single consumer.

What can affect ordering is:

 * Concurrent consumers or having more than 1 consumer that may be re-queueing messages

 * Per-message TTL

 * Priority queues

So I think you are all set. I don't know what your consumers may be doing

but using a prefetch value > 1 sounds OK, at least as far as delivery ordering goes.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ