http connections stuck in close_wait state

[Dustin]

  I'm doing some http requests periodically from a process and finding that one of them is holding file descriptors in close_wait for a long time.  I do close the response body, but is there perhaps something else I should be doing to clean up?  I'm using the shared transport and not specifically asking it to clean up, but I'd expect it to be friendly and happy here.

[Andrew Gerrand]

To avoid exhausting source TCP ports (they tend to get stuck in
TIME_WAIT), you'll need to set these sysctl values:

Linux:
sudo sysctl net.ipv4.tcp_tw_recycle=1
sudo sysctl net.ipv4.tcp_tw_reuse=1 (may not be necessary)
OS X:
sudo sysctl -w net.inet.tcp.msl=1000

Andrew

[Rémy Oudompheng]

Hello,

Did you actually check there were open file descriptors, or kill the
process to see if they were still there? You should only be afraid of
file descriptors, lingering connections are usual.

Rémy.

[Dustin]

On Monday, January 16, 2012 2:52:12 PM UTC-8, Rémy wrote:

 

Did you actually check there were open file descriptors, or kill the
process to see if they were still there? You should only be afraid of
file descriptors, lingering connections are usual.

  I found them using lsof -p [my pid].  I expect them to be closed, but they were still there and all to the same destination (this thing is making a few regular http requests to different locations).

  There's a bit of magic in the http.Transport stuff.  I'm just afraid I may have overlooked something that should clean things up.  For this app, I'm probably best off disabling keepalives in the transport.

[Josef Johansson]

Hi,

First, CLOSE_WAIT is not the same thing as TIME_WAIT, and its values
can't be changed through sysctl.
CLOSE_WAIT is due to the face that the client has sent a FIN, and the
system has initiated closing of the connection. The system is now
waiting for the server to close the connection and thus being able to
send a ACK to the client. Which then closes the connection.

If I read you correctly, you're using http.Client to send a request. I
think the problem in that case is that the server is either not
closing you're connection or not listening to you when you're closing
your connection. Have you set the request.Close = true?

Server side I see issues with this.
I think this is caused by the design in net/http/server.go where we
don't fire up the ServeHTTP as a separate Go-process and start listen
to the client connection again.
It would be possible to incorporate the same bodyEOF pattern as in
readLoop (net/http/transport.go), to make sure that we're not reading
the body of the request.

/Josef

[vincent]

there's a related issue opened: http://code.google.com/p/go/issues/detail?id=1967