approximating the no of requests which a redis instance can handle for our use case

[Tuco]

Hi, 

Let's say we have a simple application in which we only make "get" calls in redis and we find out from redis-cli info commandstats that each get call takes 1 microsec.

Considering redis is single threaded, is it fair to assume that one redis instance will be able to serve 10^6 calls per second and not more(1 sec <=> 10^6 microsecs).

Now, lets say that the redis server is asked to serve 1.5* 10^6 requests per second. Will it be fair to say that the redis instance will not be able to serve, but clients will eventually start lagging and we need to switch to a cluster.

Similarly, in redis-cli info, we get the "used_cpu_sys" and "used_cpu_user". We hit redis-cli info all every 300 secs and push the output to grafana. Further, we can see that in some of the application where we are using a standalone redis, the sum of the above "used_cpu_sys" and "used_cpu_user" is close to 200 secs. Can we assume that once it reaches above 300 consistently, we will be in the soup. 

Also, in one of the case, we use lot of lua scripting, and some of the processing is done inside lua. Now, we can see that the "used_cpu_sys" and "used_cpu_user" of those instances is high. Also, the usec value in  cmdstat_evalsha is increasing and has reached 150+ seconds(for our 300 second window). Is it fair to say that we need to expand our cluster or reduce the processing in lua.

Basically we are trying to see if a redis instance will not be able to serve more than a particular number of request(for our usage pattern) and start lagging behind on increase in the value of "used_cpu_sys" and "used_cpu_user", and usec values in cmdstat_* entries. We can try to test the same using a test environment, but we felt that it will require lot of infra to have redis use all its cpu, and some expert advice would help.

Thanks

Tuco

[Tuco]

any comments on it, pls

[hva...@gmail.com]

I'm just going to respond to your first use case below.  For the LUA question, your logic seems fine, but you really need to test at scale to find out for sure.

(my reply after your question)

Tuco wrote:

Let's say we have a simple application in which we only make "get" calls in redis and we find out from redis-cli info commandstats that each get call takes 1 microsec.

Considering redis is single threaded, is it fair to assume that one redis instance will be able to serve 10^6 calls per second and not more(1 sec <=> 10^6 microsecs).

Now, lets say that the redis server is asked to serve 1.5* 10^6 requests per second. Will it be fair to say that the redis instance will not be able to serve, but clients will eventually start lagging and we need to switch to a cluster.

I think it's likely your Redis server will stall sooner than that because of network congestion.  You haven't said how many bytes each 'GET' call retrieves from the Redis server, and you haven't explained the capacity of the network between the client machines and the Redis server machine.  (you haven't said that the clients are on the same server either)

A reply to a GET has 3 bytes of overhead (the leading ':' or '$' and the trailing '\r\n').  10^6 GET calls per second of keys that contain only a single byte will make the Redis server send 4 x 10^6 bytes through the network in replies.  (for that matter, the 'GET' commands will push more than that into the Redis server's network interface, since there's a 4-byte 'GET ', plus the key name, and ending '\r\n')  A 10Gb network interface at 95% saturation can pass 9.5Gb/s, which is 1.19 Gigabytes/second.  A 100Gb interface will pass 10x more, but you need 3-4x more than that to deliver 10^6 answers to clients.  Obviously your key names and values will be much longer than a single byte, so the network saturation point will be far lower than my calculations.

Perhaps your clients are on the same server as the Redis server process, so there's no bottleneck through the network interface.  However, I wanted to point out this factor that didn't seem to be part of your evaluation.

[Tuco]

Hi, 

We have a 10Gb network interface and clients are on separate machines, 

In one of the stacks, lua is extensively used(which does not get the data, just sets in redis), and time is cpu time occasionally hovers between 100-200 secs(for our 300 secs cycle where we capture the stats). So may be we should avoid doing more processing in lua.

In another, we can see the time increasing to ~150+ where we have around 1K - 10K per sec, with hgetalls of values ~ 1mb. 

Especially in case we use lua, we should be more careful of not doing too much processing with too many calls to drive up the cpu time.

Thanks for replying.
Tuco