Read Timeout Exception while deleting 1 million keys

[Raju Yadav]

favorite

I have a redis HA with one master and two slaves. And i have around 10 million keys in my redis. For a given flow i am deleting around 1 Million keys with a batch size of 1000 and simulatneoulsy in other flows keys are getting put into the redis. but every time while performing delete operation i encounter java.net.SocketTimeoutException: Read timed out. I have kept timeout at 8 seconds. Is there any issue with Redis Delete? I am using Jedis cleint 2.7. Stack Trace: "redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out":{ "RedisInputStream.java:201":"redis.clients.util.RedisInputStream.ensureFill", "RedisInputStream.java:40":"redis.clients.util.RedisInputStream.readByte", "Protocol.java:141":"redis.clients.jedis.Protocol.process", "Protocol.java:205":"redis.clients.jedis.Protocol.read", "Connection.java:297":"redis.clients.jedis.Connection.readProtocolWithCheckingBroken", "Connection.java:267":"redis.clients.jedis.Connection.getAll", "Connection.java:259":"redis.clients.jedis.Connection.getAll",

down vote

[hva...@gmail.com]

Have you tried the suggestions in the Redis latency troubleshooting page?  It's https://redis.io/topics/latency

To answer your question, here isn't an issue with delete.  There can be an issue with any command that takes longer to execute when the database grows large.
The section of the latency troubleshooting page titled "Latency generated by slow commands" explains this.

With the idea of slow commands in mind, what command do you send to the Redis server to get the list of keys you want to delete?

What tells you that Redis stops responding during the delete and not during the command that gets the list of keys?
The slow log suggested in the latency troubleshooting page is a good method to find out.

When you say you are deleting with "a batch size of 1000", does this mean:
* You send a single DEL command with 1000 key names as arguments? or
* You put multiple DEL commands into a MULTI/EXEC transaction? or
* You issue multiple DEL commands in a pipeline?

What happens when you reduce your batch size to something small, like 30?

The answers to these questions will point toward what's causing the server to be slow in responding, and from there the solution.