Redis with large number of TIME_WAIT socket connections

[kaleeswaran s]

We are using the below python code to connect with the Redis server. I am seeing huge connections are on the TIME_WAIT state. 

I thought of closing the connection once the operation is done with Redis server. It didn't work.

Is there anyway to close the established redis connection ?

Any thoughts on this ?

root@ubuntu:~$ netstat | grep :6479 | grep TIME_WAIT |wc -l
9061
root@ubuntu:~$ netstat | grep :6479 | grep ESTABLISHED |wc -l
7

 When I use 

disconnect() , 

I am getting the error with this.

@staticmethod
def disconnectRedisConnection(r_server):
    if r_server is not None and r_server:
        r_server.connection.disconnect()

I am getting the below error,

r_server.connection.disconnect()
AttributeError: 'Redis' object has no attribute 'connection'

Any thoughts on the huge TIME_WAIT connections / Closing the connection once the operation is done with Redis ? 

Code:

trackerDetails = CacheUtil.getTrackerDetailsByID('35855370608')

CacheUtil code as follows:

import threading
from time import sleep
import time, datetime
import traceback
import CACHE_CONST
import json
import os

import MySQLdb
import redis

# Static methods to interact with the Redis cache server
class CacheUtil(object):

    # Log Errors
    @staticmethod
    def log_message(msg):
        log_file = None
        log_file = open (os.path.abspath(CACHE_CONST.REDIS_LOG_FILE), "a")
        print(msg)
        if (log_file):
            message = time.strftime("%d-%m-%Y %H:%M:%S")    
            message += " :: " + str(msg)
            log_file.write(message + "\n")

    @staticmethod
    def saveToCache(hashName, hashValue):
        r_server = CacheUtil.getRedisConnection()
        r_server.hmset(hashName, hashValue)
        CacheUtil.disconnectRedisConnection(r_server)

    @staticmethod
    def getTrackerDetailsByID(trackerId):
        trackerDetail = None
        r_server = None
        hashName = "tDetails:" + str(trackerId)
        try:
            if trackerId is not None:
                print("getTrackerDetailsByID ")
                r_server = CacheUtil.getRedisConnection()
                trackerDetail = r_server.hgetall(hashName)
            else:
                CacheUtil.log_message("getDetailsByID failed with empty trackerId ")
        except:
            CacheUtil.log_message("getDetailsByID failed, ll fetch from DB " + str(traceback.format_exc()))
        finally:
            CacheUtil.disconnectRedisConnection(r_server)
        return trackerDetail

    @staticmethod
    def getRedisConnection():
        print("Get Redis Connection on Util ")
        r_server = redis.Redis(host=CACHE_CONST.REDIS_SERVER_URL, port=CACHE_CONST.REDIS_SERVER_PORT, db=0, password=CACHE_CONST.REDIS_PASS_PHRASE, socket_connect_timeout=2, socket_timeout=2)
        return r_server;

    @staticmethod
    def disconnectRedisConnection(r_server):
        if r_server is not None and r_server:
            r_server.connection.disconnect()

[hva...@gmail.com]

I thought of closing the connection once the operation is done with Redis server. It didn't work.

In fact, this is probably the cause of the TIME_WAITs, rather than the solution for it.  A large number of brief connections (I.e., connect, send a command, get the reply, disconnect) will produce a large number of former-sessions in TIME_WAIT state.  The solution is to not have the client behave that way.  Open a connection (or a few connections) and exchange commands/replies down the same connection(s), keeping the connection(s) open.

Some good real-world discussion of how one-command-per-connection is bad can be found in the blog post that was posted to the Reddit discussion forum a few months ago, and the thread of follow-up comments:  https://www.reddit.com/r/redis/comments/5q5ddr/learn_redis_the_hard_way_in_production/

There are also some performance tuning tips you can do on your server to shorted the time your server's connections sit in TIME_WAIT.  They are typically posted on the Internet as advice on tuning web servers for lots of connections.  The last answer at this StackOverflow question has a couple of good pointers:  https://stackoverflow.com/questions/5062839/mysql-time-wait-too-many-connections-problem