Combining publish/subscribe with RPC message queues with RabbitMQ

[John]

Put this on stackoverflow but apparently it was too board. Blah. 

http://stackoverflow.com/questions/27493246/combining-publish-subscribe-with-rpc-message-queues-with-rabbitmq

----

I am trying to put together a message queue which allows clients to communicate to each other individually and send messages directly. Basically like tweeting on twitter (everyone can read it) while also being able to being able to send someone a direct (private) message. For example.

I am using RabbitMQ, and python and the pika3 library for this.

I can do each separately fine, using a "fanout" exchange for the first and a "direct" exchange for the second. I thought I'd be able to use a "topic" exchange to do both simultaneously. I'd bind one queue to "public.*" and the other to "private.{clientId}".

However, this does not seem to work, and I'm not really quite sure why. I've been looking around but I haven't seen anyone with a similar problem, which makes me think I'm doing things the completely wrong way around.

Another way of doing it would be to have two exchanges, a "fanout" exchange for publish/subscribe messages and a "direct" exchange for RPC messages. Listening on two exchanges seems less efficient to listening on one though.

Long story short...

How do I should up my queues and exchanges in RabbitMQ to get a publish/subscribe and RPC message queue working simultaneously?

[Michael Klishin]

 On 16 December 2014 at 01:38:48, John (jwnor...@gmail.com) wrote:
> I am trying to put together a message queue which allows clients
> to communicate to each other individually and send messages
> directly. Basically like tweeting on twitter (everyone can
> read it) while also being able to being able to send someone a direct
> (private) message. For example.
>
>
> I am using RabbitMQ, and python and the pika3 library for this.
>
>
> I can do each separately fine, using a "fanout" exchange for the
> first and a "direct" exchange for the second. I thought I'd be
> able to use a "topic" exchange to do both simultaneously. I'd
> bind one queue to "public.*" and the other to "private.{clientId}".

Don't do that. Use 2 exchanges: 1 fanout and the default one
(should be sufficient for "direct messages", definitely so in the beginning).

> However, this does not seem to work, and I'm not really quite sure
> why. I've been looking around but I haven't seen anyone with a
> similar problem, which makes me think I'm doing things the completely
> wrong way around.

We can't suggest much without seeing your code.

See the first 2 examples on http://rubybunny.info/articles/getting_started.html,
they are not in Python but should be easy to follow nonetheless.

> Another way of doing it would be to have two exchanges, a "fanout"
> exchange for publish/subscribe messages and a "direct" exchange
> for RPC messages. Listening on two exchanges seems less efficient
> to listening on one though.

You consume from queues, not exchanges. And no, it is not less efficient, in fact,
you will likely see better CPU core utilisation with more queues involved. But the key
reason for using 2 separate exchange types is conceptual simplicity, of course.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[John]

Oh great, thanks.

What's the difference between using the "default exchange" and just using a named exchange set the "direct"?

[Michael Klishin]

On 16 December 2014 at 01:49:04, John (jwnor...@gmail.com) wrote:
> What's the difference between using the "default exchange"
> and just using a named exchange set the "direct"?

See Nameless exchange in [1]. You can get the same effect with a "named" exchange
of direct type but it would be more work, and "direct messages" is exactly why the default exchange
exists in the spec .

1. https://www.rabbitmq.com/tutorials/tutorial-three-python.html

[John]

Thank you very much. This is all exactly what I needed to know.