Redis Pub-Sub performance issue

[bk.k...@jabong.com]

Hi,

I am testing Redis PubSub performance test. We use golang - gopkg.in/redis.v3. Got some strange results.

Test setup :

Publishing 12 KB document at the rate of 1000 requests/second for 60 seconds.  (Total data size - 12 * 60 * 1000 = 720 MB)

client-output-buffer-limit (both soft and hard limit) is set as 1gb. 

Publisher, subscriber and redis server are running in the same server.

When running the test, subscriber is failing to receive some of the messages. In redis.log, we get this error message - 8218:S 21 Dec 19:09:24.858 # Client id=2208 addr=172.16.84.112:36763 fd=75 name= age=85 idle=0 flags=N db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=37397 omem=1073742664 events=rw cmd=subscribe scheduled to be closed ASAP for overcoming of output buffer limits.

So, how pub-sub actually works in redis. As i mentioned above, we are publishing only 720MB of data but how the redis-client-output-buffer fills with 1GB of data. When running the test, we start both publisher and subscriber at the same time and subscriber also receives messages in parallel, so that means the received messages will be deleted from output buffer right, then again how the client-output-buffer has 1 GB.

Thanks,

Pradeep