How to publish to multiple queues with work queue behavior?

[Wickman]

Hi

(I posted this question on Stackoverflow, but I'think maybe this is a better forum for it)

I have two types of consumers: FileConsumer writes messages to file and MailConsumer mails messages. There may be multiple consumers of each type, say three running MailConsumers and one FileConsumer instance.

How can I do this:

• Each published message should be handled by exactly one FileConsumer instance and one MailConsumer instance
• Publishing a message should be done once, not one time for each queue (if possible)
• If there are no consumers connected, messages should be queued until consumed, not dropped

What type of exchange etc should I use to get this behavior? I'd really like to see some example/pseudo-code to make this clear.

This should be easy to do, but I couldn't figure it out from the docs. It seems the fanout example should work, but I'm confused with these "anonymous queues" which seems like it will lead to sending same message into each consumer.

Essentially, I'm looking for the same behavior as NSQ is using, as described here.

Image of how I think it should work:

[Michael Klishin]

 On 30 October 2014 at 15:57:35, Wickman (martin....@gmail.com) wrote:
> How can I do this:
>
> Each published message should be handled by exactly one FileConsumer
> instance and one MailConsumer instance
> Publishing a message should be done once, not one time for each
> queue (if possible)
> If there are no consumers connected, messages should be queued
> until consumed, not dropped
> What type of exchange etc should I use to get this behavior?

If I understand the intent on the picture, you need a fanout exchange. It will put
a copy of every message to all queues bound to it. Messages in every queue will be
distributed on the round robin basis between consumers (we'll ignore consumer
priorities for now). With basic.qos prefetch = 1 you can make every consumer to
handle "one message at a time".

Delivery distribution between consumers already happens the way you expect (the same
message cannot be delivered to more than 1 consumer at a time), and does not
depend on exchange type.

See tutorials 2 through 5 [1] and [2] if something isn't clear.

1. http://www.rabbitmq.com/getstarted.html
2. http://www.rabbitmq.com/tutorials/amqp-concepts.html
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Simon MacMullen]

On 30/10/14 12:57, Wickman wrote:
> This should be easy to do, but I couldn't figure it out from the docs.
> It seems the fanout example

> <https://www.rabbitmq.com/tutorials/tutorial-three-java.html> should

> work, but I'm confused with these "anonymous queues" which seems like it
> will lead to sending same message into each consumer.
>

Just to be a bit more explicit than Michael was: the example you linked
shows multiple anonymous queues bound to an exchange, but there's no
reason you can't bind multiple shared, named queues to an exchange too.

Cheers, simon