RabbitMQ sharding

156 views
Skip to first unread message

John Lao

unread,
Feb 21, 2017, 6:14:54 AM2/21/17
to rabbitmq-users

RabbitMQ Dev - 

I've implemented a Sharding into our RabbitMQ cluster (see definition below). For my tests, I'd sent 1000 messages into the exchange, I am expecting that all 1,000 messages will be distributed into the 6 queues, however in my tests it was sent to all queues (each queues having 1,000 messages). 

Can you help suggests a way I can achive proper distribution of messages into the shard queues? 

Thanks,
John


Shard Policy


Exchange


 


Queue Message Distribution





Michael Klishin

unread,
Feb 21, 2017, 7:04:19 AM2/21/17
to rabbitm...@googlegroups.com
Your exchange is of type direct and not the custom 
one introduces by rabbitmq-sharding. See plugin README.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Staff Software Engineer, Pivotal/RabbitMQ

John Lao

unread,
Feb 21, 2017, 9:21:04 PM2/21/17
to rabbitmq-users

Hi Michael,

Thanks for your help.

I also managed to use the x-modulus-hash type for my exchange, however when I send 1,000 messages with a blank routing key, all of the messages goes to the same queue. Then I sent another 1,000 messages using a routing key value of "1" and all 1,000 messages goes to another queue. 

It seems the partitioning is based on the routing key provided by the producer. Is an alternative way that messages get equally partitioned without relying on the routing key value? 



Karl Nilsson

unread,
Feb 22, 2017, 4:21:51 AM2/22/17
to rabbitm...@googlegroups.com
The easiest solution would be to provide a randomly generated routing key.

Cheers
Karl

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Karl Nilsson

Staff Software Engineer, Pivotal/RabbitMQ
Reply all
Reply to author
Forward
0 new messages