What is max number of subscriptions per channel? and how does it affect delivery time?

[Chris Brown]

We have a redis cluster (sentinel) with three masters and three slaves on (m3.large in AWS).

For an examples: If we had one channel, and 1,000,000 subscribed to this channel.....how long would a small msg take to get to all of them?

Thank you very much,

[hva...@gmail.com]

Very quickly, though it depends entirely on the network latency between the master and slaves, and between the slaves and the subscribed clients.

But m3.large? "You're gonna need a bigger boat."

Pub/sub is a message multiplier, and this can clog your server's network bandwidth.  One message coming in is re-transmitted to all those subscribed clients.

Let's say you divide your million connections between the 4 servers, so there are 250,000 subscriptions per server.  We'll ignore for the moment how that means 250 thousand individual TCP connections.  Per the protocol description for pub/sub (https://redis.io/topics/pubsub), the channel name appears in outgoing messages as well as the message contents.  So let's assume the channel name is a single character 'X' and the message is a single character 'A'.  The full message, including message formatting (which can be found in the pubsub page and the general protocol page at https://redis.io/topics/protocol) is 31 bytes:  '*3\r\n$7\r\nmessage\r\n$1\r\nX\r\n$1\r\nA\r\n'  ('\r' represents the single character Carriage Return, and '\n' represents the single character Line Feed, aka Newline).

And these 31 bytes are multiplied by 250,000, the number of connections subscribed per server.  So an extremely short message (1 character) published to the channel will make each server try to transmit 7.75 megabytes to the subscribers.  A 10-character message will trigger output of 77.5 megabytes.

Amazon's guidance on the network bandwidth for m3.large instances (from http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-ec2-config.html) is 500 megabits/second, or about 62.5 megabytes/second.  The 10-character message will saturate the machine's outbound network bandwidth for more than a second.  I.e., some messages will transmit to the waiting clients quickly, but others will take up to 1.25 seconds to push their way through the bottleneck.  Larger messages will make it worse.  Not only that, but during that time, replication of new messages from the master to the slave will also be delayed by the bottleneck.  Since you asked about the message transmission latency, these additional delays cause by clogging the network interface are not good news.

Once you understand the way pub/sub multiplies messages, you can plan your network bandwidth (i.e., your AWS instance size) to handle your traffic.  Don't forget to factor in natural growth in traffic (more messages per minute) and fault handling (one slave crashes and its clients re-connect to the other slaves, increasing their subscription counts).