RabbitMQ 3.12 is coming!

[Michal Kuratczyk]

Hi!

On behalf of the RabbitMQ team, I'm happy to share that RabbitMQ 3.12 will be released soon. 

We've published a blog post that shows significant performance improvements over 3.11:

https://blog.rabbitmq.com/posts/2023/05/rabbitmq-3.12-performance-improvements/

The most important change, which affects a lot of users, is that there is no longer a distinction between lazy and non-lazy (default) queue mode for classic queues.

All queues will now behave similarly to, but not exactly like, lazy queues -- they will keep a small part of the messages in memory to serve them quickly,

but will write messages to disk as the queue gets longer. This should significantly reduce memory usage for many users.

We also recommend switching to classic queues v2. This provides additional performance benefits.

We expect v2 to become the default in 3.13 and the only option in the version following 3.13.

Release candidate version is available for testing:

Github: https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.12.0-rc.2

Docker image: rabbitmq:3.12.0-rc.2-management

As always, we'd appreciate your feedback. Let us know how 3.12 and classic queues v2 work for you!

Best,

--

Michał

RabbitMQ team

[Vilius Šumskas]

Hello and congratulations to the team!

 

Improvements in performance are always appreciated. I have a question regarding lazy queues though.

 

Our application is HTTP service backend which uses RabbitMQ mirrored classic queues as an intermediate between 3^rd parties querying that HTTP server and RabbitMQ consumers connecting to thousands of SQL databases. In a sense, this is public API service allowing our clients to query their on-premise SQL databases through one centralized endpoint. Every client have their own queue and these queues are very short, i.e. we don’t keep or try to redeliver RabbitMQ messages if something gets wrong in the pipeline. Even if we fully restart RabbitMQ cluster, external clients get HTTP 5XX response and are usually instructed to resend their HTTP query.

 

Since lazy queues will try to write everything to disk, and disks usually are much slower than memory, does this make lazy queues not suitable for our case?

 

And another question, does mentioned lazy behaviour applies to CQv1, or to CQv2 also?

 

Currently we are preparing to migrate to quorum queues, because HA is important for us, but it will take time. For the moment I’m more concerned about what will happen during intermediate timeframe, when we upgrade from 3.11 to 3.12. Hence, my questions.

 

--

    Best Regards,

    Vilius

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAA81d0vPVfAfiXjZteP-%2BpGRocfvhMHYOrXtSseo-F142GoW9Q%40mail.gmail.com.

[Michal Kuratczyk]

Hi,

As I mentioned in the blog post - based on our testing, the new default/only mode is more efficient than either old default or the lazy queues, especially if you switch to v2

(the change affects both v1 and v2). We obviously highly recommend testing/verifying this since there are lots of ways people use RabbitMQ and we can't test everything.

Just because the new mode is more similar to the lazy mode than to the old default mode, do not assume it is simply the lazy mode.

In 3.12, some messages may never hit the disk and they don't requeue to disk (though they may in the future once v1 is gone).

For your use case, we may have an even better option in the future: highly-available non-replicated queues:

https://github.com/rabbitmq/rabbitmq-server/discussions/6376

Best,

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/AM8PR01MB7699E83EFEBD0747A8A07888927F9%40AM8PR01MB7699.eurprd01.prod.exchangelabs.com.

--

Michał

RabbitMQ team

[Vilius Šumskas]

Thank you for your response.

 

Subscribed to the discussion. That option definitely looks interesting for us.

To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/CAA81d0sd8paYhHZcHkzhz-rYw7QrxhCY0Ck6nX7kSiq86_BgnA%40mail.gmail.com.