Skip to first unread message
Bernd Wechner
Apr 21, 2020, 9:17:43 AM4/21/20
to celery-users
I can see a lovely list of queues in RabbitMQ using a URL like:
I'd like to get such a list programatically using Kombu, and ideally AMQP broker agnostic (i.e. work for RabbitMQ and Redis and any other broker Celery supports).
I'd be content with a list of queues that are bound to a given exchange.
But alas I can find no means of getting or building such a list in Kombu. I've looked long and hard.
I am loather to use a RabbitMQ specific solution, though I'm not beneath implementing a Kombu extension if there is an AMQP generic means of getting such information and Kombu simply hasn't implemented it.
Regards,
Bernd.
Jason Held
May 23, 2020, 3:17:06 PM5/23/20
to celery-users
For redis, the queue names are top level keys in redis' keyspace.
Flower gets the info by broadcasting for this info against the workers. But agreed that's not ideal because it requires the workers to be up/responsive (which with a healthy cluster they are).
And again for redis, totally agreed, it would be nice if there was as an actual registry within a celery/kombu `queues` or similar key. It would probably be namespaced so something `celery.queues`.
That said, I'm not aware if celery/kombu already does something like this (at least for redis) but I'd be a little surprised if it did.
I may re-review the queue creation code path (again more interested in redis but the investigation will likely illuminate for the spread of brokers) and post back.
Flower gets the info by broadcasting for this info against the workers. But agreed that's not ideal because it requires the workers to be up/responsive (which with a healthy cluster they are).
And again for redis, totally agreed, it would be nice if there was as an actual registry within a celery/kombu `queues` or similar key. It would probably be namespaced so something `celery.queues`.
That said, I'm not aware if celery/kombu already does something like this (at least for redis) but I'd be a little surprised if it did.
I may re-review the queue creation code path (again more interested in redis but the investigation will likely illuminate for the spread of brokers) and post back.
Bernd Wechner
May 23, 2020, 8:29:00 PM5/23/20
to celery...@googlegroups.com, Jason Held
Yes, it sure would be nice of Kombu kept a record of at least all the
Kombu created and managed queues available for querying. Give a queue
name kombu has facility for testing if it exists, but not a way of
requesting the list of queues. I personally am using RabbitMQ not Redis
but it's clear that RabbitMQ also has some facility for listing its
queues as its web management interface has a list. I'm not sure if
there's a AMQP general protocol for querying the list of queues in a
given context (connection, node, cluster whatever) and if not the code
to provide one if it were implemented would end up being implementation
specific (in the redis and rabbitmq support).
Kombu created and managed queues available for querying. Give a queue
name kombu has facility for testing if it exists, but not a way of
requesting the list of queues. I personally am using RabbitMQ not Redis
but it's clear that RabbitMQ also has some facility for listing its
queues as its web management interface has a list. I'm not sure if
there's a AMQP general protocol for querying the list of queues in a
given context (connection, node, cluster whatever) and if not the code
to provide one if it were implemented would end up being implementation
specific (in the redis and rabbitmq support).
Jason Held
May 23, 2020, 8:49:52 PM5/23/20
to celery-users
Right. AMQP (0.9.1 or 0.10) would have had an opportunity to do this (seems like it might not have), but 1.0 definitely won't -- it's a much simpler protocol (doesn't allow or disallow broker, queue, exchange, etc, forces the application to build it on top). Yeah, it'll definitely have to be on the framework (or app) level.
Reply all
Reply to author
Forward
0 new messages