Draining http response bodies

[Amnon]

Is it necessary for a http client to fully read the http response body?

Opinion on the Go forums seems divided https://forum.golangbridge.org/t/do-i-need-to-read-the-body-before-close-it/5594

But a simple benchmark https://play.golang.org/p/5JDWYbRe0lD
suggests that leaving unread data in the response body will prevent 
the connection being reused, and much slower performance.

The documentation https://pkg.go.dev/net/http states:

"The client must close the response body when finished with it:"

The following example does include a call to io.ReadAll(resp.Body)
but it does not spell out whether there would be a performance penalty
for failing to read the entire body.

It would be good if the standard library documentation was a bit more explicit here.

[Amnon]

Added issue https://github.com/golang/go/issues/48860

[David Riley]

I suspect a lot of this actually comes down to operating system level handling of TCP sockets, which is something that it would be hard for the docs to address other than maybe offering a warning that your OS *might* not release the port for reuse if the body isn't fully drained.

I could be wrong, though, there's half a chance it's really a GC thing instead. I'd have to look at the source to be sure.


- Dave

[Amnon]

Whatever the OS, the crucial question is whether the TCP connections get reused, or whether a new TCP (and potentially TLS) connection has to be negotiated, involving multiple round trips to the server.

Someone has kindly pointed out that the docs do spell out the behaviour - I just missed the comment:

// It is the caller's responsibility to
// close Body. The default HTTP client's Transport may not
// reuse HTTP/1.x "keep-alive" TCP connections if the Body is
// not read to completion and closed.