DisableKeepAlive not being honored in http.transport

[Ryan Rank]

I wrote a small program that runs a repeated GET against a given URL. However, the DisableKeepAlives boolean does not seem to be honored. Code is similar to this:

transport := &http.Transport{

    DisableKeepAlives: false,

}

client := &http.Client{

    Transport: transport,

}

for i := 0; i < 10; i++ {

    _, err := client.Get("https://foo.com")

}

Based on the code, I would expect the HTTP session to stay alive. I ran the preceding code with keepalive enabled and disabled, performance was not impacted. I then did a TCPDump and saw that the TCP session was being built and torn down with every HTTP Request. I have a feeling that I'm missing something. Does anyone have any ideas?

Go 1.11.1 on CentOS 7

[Amnon Baron Cohen]

Connections are indeed reused where possible by default in the Go standard library.
But as http (before http2) does not support multiplexing, it is not possible to reuse
a connection which is still in use (i.e. which is still in the process of reading the response).
These means that the body of the http response does need to be read till completion
and then closed by the client.

So you will want to do something like

for i := 0; i < 10; i++ {

    r, err := client.Get("https://foo.com")

    if err != nil {

       panic(err)
    }
    io.Copy(os.Stdout, r)
    r.Close()

}

Hope this helps....
- Amnon

[Ryan Rank]

This is what I did, and see no change in behavior.

for i := 0; i < 10; i++ {

    response, err :=client.Get("https://foo.com")
    if err!= nil{

        panic(err)
    }

    ioutil.ReadAll(response.Body)
    response.Body.Close()
}

The network trace shows a SYN, SYN-ACK, ACK at the beginning of each HTTP GET, and a FIN-ACK followed by some RST packets (those RSTs are common for the foo.com in this case).

I do know that foo.com honors keepalives because it's readily apparent when I use ab to test.

Also, I did upgrade to go 1.14.2 and nothing changed.

[Amnon Baron Cohen]

In these situations I would normally use wireshark to look at the life-cycle of a single session.
Which direction are the FIN and RST packets going?
Are they sent from the client, or the server?
If the RST packets are sent by the server - i.e. the server is slamming shut the connection,
then the client can not reuse them and has to create new ones.

If foo is https, then this does make it harder to use tcpdump to look at the headers.
But you can print out the response headers sent bt foo.com using https://golang.org/pkg/net/http/httputil/#DumpResponse
and this should give you a bit more information.

Hope this helps...
- Amnon

[Ryan Rank]

.98 is the client, .36 is the server.

This is the end of one transaction and the beginning of another. The client sends a FIN-ACK, then immediately starts the next transaction with the SYN packet. After that, the connection is ended; it looks like the client sends the RST packet. I'll take a look at DumpResponse. Hopefully I see something useful from that.

[Ryan Rank]

Interesting. No headers coming back.

[Ryan Rank]

When I change to a different URI (https://foo.com/bar) it works as expected. So there's something with the original response that causes this to not work. I'm curious as to what, but this is working as designed and expected.

Thank you for the help!