Redis-cluster benchmarking

[RogerV]

I encountered this question today on my Stack Overflow feed- http://stackoverflow.com/questions/26857962/redis-cluster-benchmarking.

 On first thought, I felt that the question writer was wrong and the facts weren't true. However, to my surprise, when I tried a similar setup on my machine, I encountered the same thing.

Would anyone know what is happening here.

For those who do not want to follow links, here is the question verbatim

"

What is the right way to benchmark the redis-cluster(released recently in 3.0 RC). AFAIK, redis-benchmark utility hits only one node of the cluster.

EDIT:(Details) My single instance of redis without any clustering gives a throughput of ~90 k set/get operations but the cluster setup fails big time.

SETUP 1: 8 core machine running a cluster of 3 masters+ 3 slaves(all on the same machine) I also run 3 benchmarking utilities on the same machine. The throughput drops to 25 k on each master node.

This makes me think that I am, perhaps, running one too many processes for the number of cores on my machine.

Setup 2: I update the setup to have 3 Masters and 0 slaves. Interestingly, this doesn't help the case either and the throughput is still 25 k on each machine

The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get

Any help on this front would be appreciated.

"

[Josiah Carlson]

I replied on SO.

The short version:

1. he seems to be only benchmarking 1 master

2. even if he was benchmarking all 3 masters, 2/3 of the commands would fail due to sharding

3. there isn't a good Redis cluster benchmark yet

 - Josiah

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To post to this group, send email to redi...@googlegroups.com.
Visit this group at http://groups.google.com/group/redis-db.
For more options, visit https://groups.google.com/d/optout.

[Ashwani Gupta]

I have the same concern as Roger. I successfully bench-marked single standalone redis instance. Is there any way/tool to benchmark a redis cluster (My cluster is having 3 masters and 3 slave nodes as I am following this : https://redis.io/topics/cluster-tutorial

Just a weird thought - Can I have nginx infront of redis cluster providing all requests to redis cluster and perform the benchmark on this setup ??

- Ashwani

[Tuco]

I had tried to benchmark redis cluster in the following way.

• created a jar file in java, which would create random keys and perform a set of operations on redis cluster, like set, get.
• the jar when running would output the following data after performing each set of operation
• randomId  <TimeStampInYYMMDDHHMMSS>   <timetakenForSetOperationInMicroSecs> <timeTakenForGetOperationInMicrosecs>
• resulting in one entry like 123565656   20161222105020  123   345 for each record inserted/updated.
• 4 processes of jar were run on each server.
• the above was done on each of the 7 servers.

In total, there were 4*7=28 application being simulated to run by above, and the logs in the above format were captured in different logs files.

After that, a spark job was run on the log files which would output the following.

         <TimestampYYYYMMDDHHMMSS>,<noOfRequestsCompletedDuringThatSecond>,<averageTimeForSetOperation>,<AverageTimeForGetOperation>

The above was transferred to a CSV/output file, and the following graphs were created in excel

• No of Requests Served Vs Time
• Average Time for Set Operation Vs Time
• Average Time for Get operation Vs Time

The cluster had 3 masters + 3 slaves, and it could easily serve upto 30K operations per second. As i increased the servers, it reached upto 40K. I am sure it would have reached higher if we more servers.

In production, we have redis cluster with a lot more nodes server serving upto 1 million qps, but that's because we use pipelining etc.

Hope it helps.

[Adolfo Herrera]

Did you have any success benchmarking the cluster?