Does the HTTP Client support Keepalive or Pipelining?

[Travis Reeder]

Is the default to open/close the connection each request?  Or is it working some magic behind the scenes?  Or should I use ClientConn to get this type of functionality?

[Evan Shaw]

On Fri, Sep 9, 2011 at 7:29 PM, Travis Reeder <tre...@gmail.com> wrote:
> Is the default to open/close the connection each request?  Or is it working
> some magic behind the scenes?

If I'm reading the code correctly:

When the http.Request.Close field is false (which is the "default"),
the connection stays open. Connections are reused between requests
when possible, so there shouldn't be any extra work involved.

Multiple requests can be active with the same server on different
goroutines. This is all handled by http.DefaultTransport, an instance
of http.Transport.

- Evan

[Brad Fitzpatrick]

It supports keep-alive.  Just be sure to fully read (and/or close) the Response.Body, otherwise that TCP connection won't be re-used.

 

Pipelining could be supported but it's a crap spec and so broken in the real world that it's not worth it.

[Tarmigan]

On Fri, Sep 9, 2011 at 11:43 AM, Brad Fitzpatrick <brad...@google.com> wrote:
> It supports keep-alive.  Just be sure to fully read (and/or close) the
> Response.Body, otherwise that TCP connection won't be re-used.

Regarding the "or close", is closing the response body enough? When I
have a lot of connections open and I only read part of the Body before
closing, the Read()s in other goroutines die earlier too. Disabling
pipelining or reading the whole body seemed to fix it. I had assumed
this was a design decision and worked around it, but if Close() after
a partial Read is supposed to work, I could try to make up a test case
if that would be helpful.

Thanks,
Tarmigan

[Brad Fitzpatrick]

Yes, Close after a "partial read" (all reads are generally partial reads) should work, assuming the server sent you a keep-alive response in the first place.

I love bug reports, though, especially with repro cases. 

[Tarmigan]

Good news is I can't reproduce it and I think it's fixed (in the last
2 months?). Less thrilling is that my binaries are getting bigger.
My test one is fairly simple and is now 3.9M, while older ones I have
lying around were 2.7M for the same code.

Thanks,
Tarmigan

[Brad Fitzpatrick]

The exact same code?  That's a little surprising.

[Tarmigan]

It is surprising.
$ ls -lh tracker
-rwxr-xr-x 1 tarm tarm 3.9M Sep 9 19:09 tracker
-rwxr-xr-x 1 tarm tarm 2.8M Aug 20 13:57 tracker.good
-rwxr-xr-x 1 tarm tarm 2.7M Jun 23 09:53 tracker.ok

I don't 100% trust the timestamps, but they probably aren't far off.

git log shows my last change as June 22.

-Tarmigan

[Brad Fitzpatrick]

On Sat, Sep 10, 2011 at 9:30 AM, Tarmigan <tarmiga...@gmail.com> wrote:

On Fri, Sep 9, 2011 at 10:53 PM, Brad Fitzpatrick <brad...@golang.org> wrote:
> On Fri, Sep 9, 2011 at 7:18 PM, Tarmigan <tarmiga...@gmail.com> wrote:

>> Less thrilling is that my binaries are getting bigger.
>> My test one is fairly simple and is now 3.9M, while older ones I have
>> lying around were 2.7M for the same code.
>
> The exact same code?  That's a little surprising.

It is surprising.

Is it the exact same code, though?

[Travis Reeder]

Thanks for the info guys! 

[Tarmigan]

Exactly the same code. Just rebuilt to be sure.

~/devel/tracker (master)$ ./tracker_test_01 -version
weekly.2011-06-23 8965
~/devel/tracker (master)$ ./tracker_test_00 -version
weekly.2011-09-07 9720
~/devel/tracker (master)$ ls -lh tracker_test*
-rwxr-xr-x 1 tarm tarm 3.9M Sep 12 17:06 tracker_test_00
-rwxr-xr-x 1 tarm tarm 2.7M Sep 12 17:11 tracker_test_01

-Tarmigan

[Tarmigan]

bisected to hg rev ce2e5f44b310
https://code.google.com/p/go/source/detail?r=ce2e5f44b310a2e4a7df2b030183eab62f5b10ad

Not sure whether you all care, but ccing Russ anyway.

~/devel/tracker (master)$ ls -lh tracker_test_09 tracker_test_10
-rwxr-xr-x 1 tarm tarm 2.7M Sep 12 17:44 tracker_test_09
-rwxr-xr-x 1 tarm tarm 3.9M Sep 12 17:47 tracker_test_10
~/devel/tracker (master)$ ./tracker_test_10 -version
weekly.2011-09-01 9660
~/devel/tracker (master)$ ./tracker_test_09 -version
weekly.2011-09-01 9659

Thanks,
Tarmigan