[rabbitmq-discuss] Exchange and queue naming conventions

[Bill Moseley]

Are there any good conventions to follow when naming queues, exchanges, and bindings?  As I'm adding more queues I'm starting to think some type of naming convention would be very smart.  Just curious what other's do.

For now I'm only using a direct exchange type.  When should I be declaring additional exchanges vs. using the same (or even the default) exchange?

A pattern I'm seeing is I have an app-specific producer than sends a message to a generic consumer (used by multiple apps) but then that consumer sends a message to another app-specific consumer.  What I'm doing is passing the return queue name in the reply_to attribute.  I'm not passing in the exchange name -- so I'm just using the same exchange for every binding.

I also declare the exchange, queue, and binding in every component (the producer and both consumers).  I assume that's typical as it allows starting the components in any order.  Is that appropriate?

Finally, I have been naming queue and bindings with the same name.  Is there a reason that should not be done?

Thanks,

--
Bill Moseley
mos...@hank.org

[David Wragg]

Hi Bill,

Bill Moseley <mos...@hank.org> writes:
> Are there any good conventions to follow when naming queues, exchanges, and
> bindings? As I'm adding more queues I'm starting to think some type of
> naming convention would be very smart. Just curious what other's do.

I don't think there is any one right way. When choosing names for
queues, exchanges, and keys, the use of dotted paths is widespread. But
your questions are more about usage than naming.

> For now I'm only using a direct exchange type. When should I be declaring
> additional exchanges vs. using the same (or even the default)
> exchange?

It depends on your message exchange patterns.

Generally, you should arrange your application to take full advantage of
the facilities AMQP presents. If your application ends up dispatching
or filtering messages in a way that could be delegated to RabbitMQ, then
you should do.

Also, consider where you might later want to add new producers and
consumers - even if just for debugging purposes. The ability to easily
add additional parties into message exchanges is one of the major
advantages of a messaging-based approach as opposed to point-to-point
mechanisms such as web services.

> A pattern I'm seeing is I have an app-specific producer than sends a message
> to a generic consumer (used by multiple apps) but then that consumer sends a
> message to another app-specific consumer. What I'm doing is passing the
> return queue name in the reply_to attribute. I'm not passing in the
> exchange name -- so I'm just using the same exchange for every
> binding.

For a request-response pattern, this seems fine.

One variation is to make the exchange a topic exchange rather than a
direct exchange. This allows you to intercept all replies with a
wildcard binding (e.g. for debugging).

> I also declare the exchange, queue, and binding in every component (the
> producer and both consumers). I assume that's typical as it allows starting
> the components in any order. Is that appropriate?

Yes, this kind of thing is perfectly normal with AMQP.

Typically, both the producers and consumers will declare the exchanges
of interest. But only consumers will declare and bind the queues,
because they know which messages they wish to receive.

> Finally, I have been naming queue and bindings with the same name. Is there
> a reason that should not be done?

By naming bindings, I assume you are referring to the binding key.
There's nothing wrong with what you describe, except that this sounds
very close to what the AMQP nameless exchange does implicitly. So
perhaps you could take advantage of that, and so avoid the need to
declare an exchange and explicitly bind your queues.

David

--
David Wragg
Staff Engineer, RabbitMQ
SpringSource, a division of VMware
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq...@lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

[Bill Moseley]

On Mon, Jan 24, 2011 at 10:14 AM, David Wragg <da...@rabbitmq.com> wrote:

Hi Bill,

Bill Moseley <mos...@hank.org> writes:
> Are there any good conventions to follow when naming queues, exchanges, and
> bindings?  As I'm adding more queues I'm starting to think some type of
> naming convention would be very smart.  Just curious what other's do.

I don't think there is any one right way.  When choosing names for
queues, exchanges, and keys, the use of dotted paths is widespread.  But
your questions are more about usage than naming.

Hi David,

I do think it makes sense to use the dotted notation, and then use topic queues as that makes for very flexible routing options.  I was wondering if anyone had worked out some "best practices" over time that would be useful as we start to add more tasks.

I'm trying to break up some of our existing processes that are very tightly coupled - the workers know quite a bit about the producers.  For example, a worker might be configured to handle just one application in "QA" mode, and another in "production" mode.  And likewise, the producer (in our current system) must know where to send the message ("I'm in QA mode so I need to send to the QA worker").

So, as an example I'm wondering about schemes where an app might use a routing key of "app1.qa.rotate_image" which would allow us to configure consumers that handles all image rotation for all apps and all modes, or separate consumers based on the mode.

BTW -- in that case can I configure binding for three queues:  *.qa.rotate_image. *.production.rotate_image, and then a third for all except "qa" a and "production"?

Finally, I'm still a bit confused when to create new exchanges (assuming all the same type).  Just trying to get some basic guidelines.  Is there any overhead with using more exchanges vs. just more bindings on the same exchange?  Is is common to use separate exchanges for grouping consumers (i.e. an exchange for report queues and another exchange for image manipulation queues)?

Thanks very much for your help,

--
Bill Moseley
mos...@hank.org

[Michael Klishin]

2011/1/25 Bill Moseley <mos...@hank.org>

I do think it makes sense to use the dotted notation, and then use topic queues as that makes for very flexible routing options.  I was wondering if anyone had worked out some "best practices" over time that would be useful as we start to add more tasks.

1+ year and several apps later I see the following pattern developing on my team:

1. Exchanges are declared by producers.

2. Queues are declared and bound by consumers.

3. In vast majority of cases, consumers are "smart" and producers are very dumb: they just put messages in well-known places. Consumers

    are "smart" in a sense that they almost always keep some persistent state (in Redis, for example), while producers are just monitored in case they crash.

4. Exchanges are named using the following pattern $prefix.$applications_group.$app.$environment. For example, megacorp.mail_delivery.shaper.qa or megacorp.search.online_indexer.production. It works well with multiple installations and if you want to route some data to your customers/partners/archive.

Also, it becomes increasingly obvious that many exchanges and bindings would be unnecessary if we begin to use default exchange more.

 

BTW -- in that case can I configure binding for three queues:  *.qa.rotate_image. *.production.rotate_image, and then a third for all except "qa" a and "production"?

"Except" case is not handled by AMQP very well (not in 0.9.1 I am most familiar with, anyway). Maybe *.other.rotate_image would be good enough?

 

Finally, I'm still a bit confused when to create new exchanges (assuming all the same type).  Just trying to get some basic guidelines.  Is there any overhead with using more exchanges vs. just more bindings on the same exchange?  Is is common to use separate exchanges for grouping consumers (i.e. an exchange for report queues and another exchange for image manipulation queues)?

Unless you have thousands of them on a small box, it probably does not make much difference. For support cases, dealing with apps that use 150 exchanges may be significantly more difficult than if they were using just 3, though.

Hope this helps.
--
MK

[Jeff Hinrichs]

That was very helpful, thank you.  I am very interested if others concur or have differing wisdom?

Best,

Jeff