Support for priority in quorum queue of RabbitMQ.

[Ishan Ghelani]

Hello,

I am currently overseeing a migration process within our system, specifically transitioning from classic priority queue mirroring to quorum queue within an existing RabbitMQ cluster. Our environment is utilizing Celery version 5.3.1 alongside Python 3.9.

Currently, our setup utilizes a priority queue with queue mirroring. However, due to the absence of message priority support in the quorum queue, I had to create multiple queues, each containing distinct priority levels.

Is there any native support for the quorum queue in Celery?
If not, are there any plans underway to incorporate such support into Celery's functionality? Could you kindly provide insights into the expected timeline for the integration of this feature?

Furthermore, as part of this migration process, I am trying to implement custom priority handling. Specifically, I have configured the publisher to route messages across different queues based on their respective priority levels. Subsequently, I aim to ensure that consumers selectively retrieve messages based on their priority levels.

i.e., let us consider a scenario with three queues: q1, q2, and q3, with q3 representing the highest priority. Ideally, I seek to configure the worker to exclusively process tasks from q3 until it is fully drained, subsequently transitioning to q2, and so forth.

I have provided "task_queues" option in the python code while creating celery object. Worker should listen to these queue list. The workers are picking the tasks from random queue listed. It is not honoring the order given in the task_queues list. How can I configure this to instruct worker for priority levels?

I would greatly appreciate any insights, guidance, or recommendations you might offer to streamline this migration process and address the aforementioned queries effectively.