Too many open connections in redis server in ESTABLISHED state

[sushil kumar]

When we close the Jedis connection, it closes on client side but on server connection is still in ESTABLISHED mode. connections are never getting closed on server for the same client and it keeps growing. And we are getting redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException:

when we run netstat on client it shows only 4 connections alive (ESTABLISHED ) but when we do netstat on redis server we see 900 connection in ESTABLISHED state with the same client.
CLIENT LIST on redis server also shows 900 connections ilde for days.

Redis / Jedis Configuration

redis.with.sentinel = true
redis.pool.connection.timeout.millis=10000
redis.pool.max.total=100
redis.pool.max.idle=25
redis.pool.min.idle=10
redis.pool.max.wait.seconds=5000

GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
poolConfig.setMaxTotal(maxTotal);
poolConfig.setMaxIdle(maxIdle);
poolConfig.setMinIdle(minIdle);
poolConfig.setMaxWaitMillis(maxWaitInSeconds*1000);
pool = new JedisSentinelPool(clusterName, sentinels,poolConfig,timeoutInMillis);

Client code to access resource from redis:

try (Jedis jedis = pool.getResource()) {
     jedis.get(key);
}catch (Exception e) {
}

Jedis version: 2.8.0

Redis version: Redis server v=3.0.3 malloc=jemalloc-3.6.0 bits=64 build=a6bf2be256a3f774

Why is redis server still keeping all the connections ESTABLISHED even though client has closed it.
Client host:

Even after setting timeout value to 7200 in redis.conf, I am still seeing all those connections open in sentinel 6379. Timeout should have closed all idle connections only pub/sub are not closed but we are not using pub/sub.

Is there any other setting for JedisSentinel to close idle connections ?

I have observed that most of the open connections are for sentinel which is running on port 6379 on slave host in cluster

Redis host:

Redis Server sentinel client list:

why I have all client connection CMD=subscribe ?

From client code i am not using pub/sub model for getting connection.

Please suggest why so many connections are in open state. We are not using any load balancers or proxy.

ticket open at : https://github.com/xetorthio/jedis/issues/2038

[Allan Wax]

A couple of suggestions

a) set blockWhenExhausted in the pool

b) switch to using a jedis cluster instead of sentinels.  A whole lot of logic involving resources and connection is taken care of for you and it's simpler to use as you only need to create one jedis cluster in your code and you can use it anywhere in multiple threads.  The only drawback is if you want to reference multiple keys in a pipeline and the keys have been spread across multiple servers.  For the most part there is an easy solution to this as well.

[sushil kumar]

I have tried setting timeout value in conf file to 7200 seconds but still I could see connections are increasing on sentinel port on slave machines.

As per below document it says whenExhaustedAction is by default set to  WHEN_EXHAUSTED_BLOCK . And we are already using maxWait property. 

https://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/pool/impl/GenericObjectPool.html