Horizontal scalability of fanout exchanges with many different routing keys?

[shal...@gmail.com]

Suppose I a have a fanout exchange replicated across hundreds of nodes. On each node, there are queues bound to the fanout exchange with different routing keys.

If I send a message with routing key 'foo' to the exchange, will this:

1. Transfer the message (over the network) to all nodes and then essentially discard it on any nodes that don't have a bound queue with routing key 'foo'; or
2. Selectively transfer only to nodes that have bound queues listening for routing key 'foo'?
   

I'm just learning about RabbitMQ so if I'm completely confused please let me know =)

Thanks!

[Michael Klishin]

 On 3 December 2014 at 02:17:38, shal...@gmail.com (shal...@gmail.com) wrote:
> Suppose I a have a fanout exchange replicated across hundreds
> of nodes. On each node, there are queues bound to the fanout exchange
> with different routing keys.

Fanout exchanges do not use routing keys. A copy is unconditionally placed into every
queue bound. Are you sure you are not talking about topic exchange?

> If I send a message with routing key 'foo' to the exchange, will
> this:

> 1. Transfer the message (over the network) to all nodes and then

> essentially discard it on any nodes that don't have a bound queue
> with routing key 'foo'; or

> 2. Selectively transfer only to nodes that have bound queues listening

> for routing key 'foo'?

2.

Exchanges are just routing table names. They are shared in the cluster, so every
node has an up-to-date list of bindings most of the time (for most systems, saying all the time
is fair). From there it selects a subset of queues that match and a copy is delivered
to master nodes of those queues.

Note that there are no master nodes in general, every queue has a master node and 0 or more mirrors.
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[shal...@gmail.com]

On Wednesday, December 3, 2014 10:21:45 AM UTC+11, Michael Klishin wrote:

 On 3 December 2014 at 02:17:38, shal...@gmail.com (shal...@gmail.com) wrote:
> Suppose I a have a fanout exchange replicated across hundreds  
> of nodes. On each node, there are queues bound to the fanout exchange  
> with different routing keys.

Fanout exchanges do not use routing keys. A copy is unconditionally placed into every
queue bound. Are you sure you are not talking about topic exchange?

You're right, I didn't realise fanout exchanges ignore the key. After reading so more I think topic exchanges or even direct exchanges would work for me.
 

> If I send a message with routing key 'foo' to the exchange, will  
> this:
> 1. Transfer the message (over the network) to all nodes and then  
> essentially discard it on any nodes that don't have a bound queue  
> with routing key 'foo'; or
> 2. Selectively transfer only to nodes that have bound queues listening  
> for routing key 'foo'?

Exchanges are just routing table names. They are shared in the cluster, so every
node has an up-to-date list of bindings most of the time (for most systems, saying all the time
is fair). From there it selects a subset of queues that match and a copy is delivered
to master nodes of those queues.

 
Great, that's exactly what I was hoping for.

Thanks for your help Michael.

[Simon MacMullen]

On 02/12/14 23:21, Michael Klishin wrote:
> Exchanges are just routing table names. They are shared in the cluster, so every
> node has an up-to-date list of bindings most of the time (for most systems, saying all the time
> is fair). From there it selects a subset of queues that match and a copy is delivered
> to master nodes of those queues.

A copy is delivered to the master and all slaves, since we need to not
lose messages if the master dies while the message is being delivered to
queues.

Also (in case this wasn't clear) if a message gets routed to 1,000
queues on one node, only one copy of the message is sent across the
cluster link.

Cheers, Simon