redis response time with sorted sets

[Isart Montane]

Hi,

I'm having some trouble with the redis response time, so I might need some help from you guys :)

we are using Redis intensively on our system, we are running a group of 6 redis servers sharded using twemproxy. All of the redis servers are configured to not persist data to disk (we can afford data loss).

We use it to make some calculations on frequency (something like a tag cloud) and get the most used terms from a text, so we use sorted sets for it, we are using ZADD and ZINCRBY to add or increment the score for a given term. The issue we are having is that if we mesure the latency for the ZINCRBY we sometimes get spikes of >1s. The server is not swapping and close to 20% of CPU usage.

My feeling is that some of this ZINCRBY commands get "queued" because of the single threaded nature of redis while slower commands are running (a big ZADD with 1M terms might take up to 5s), is that possible? how does redis handle fast commands (like ZINCRBY) while the slow ones are running?

I've also checked latency using redis-cli --latency and I get the same spikes, even when running it from localhost (so no network delays)

The sorted sets we work on are quite big, sometimes > 30M, and we have seen similar performance issues running without twemproxy, so I don't think the issue is there.

Any help is appreciated,

Isart Montane Mogas

[Jan-Erik Rediger]

You are absolutely right with your conclusion. Redis is single-threaded
and thus can only handle one request a time. Long-running calls like a
huge ZADD will block all other connections until finished.

You could split up the ZADD in smaller ones to circumvent this.

[Yiftach Shoolman]

Have u validated this by running slowlog on the relevant shards ?

Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at http://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/groups/opt_out.

[vishal sharma]

I am facing same issue while extract data with hgetall and hmget command and response time increased upto 7-8 seconds. I need your help to resolve this.

Please help me tune commands or any other alternate way.