Both Keep-Alives and HTTP2 not working. High latency for every connection.

Peter Waller

unread,

Mar 21, 2016, 12:31:47 PM3/21/16

to golang-nuts

Hi All,

I'm using ListenAndServeTLS with a self-signed certificate and running go1.6.

These things are confusing me a little and I have a demo program below which shows the problem:

1) HTTP2 doesn't seem to work out of the box as I had expected it to, go1.6 talking to go1.6.

2) Keep-Alives don't seem to be working, and therefore connection warm-up is always quite costly, even with every request going to the same server. (I always see the ConnState go to 'Closed' after each request).

3) Connection cost seems to depend on the frequency at which I make connections. It's generally quite slow (~30ms for 1 req/s), unless I make connections at a very high rate. Then the per-connection cost comes down ( ~10ms for ~100req/s). (CPU cache?).

===

The following program runs both the client and server in the same binary. Please take a quick look at this playground. You'll want to run it locally, owing to the playground's fun time and network connection physics:

http://play.golang.org/p/WFTakG8jkY

- It generates a self signed ECDSA certificate (which is also a CA) using P224.

- It starts a TLS server.

- It makes a http client with a certificate pool which just contains that one certificate.

- It makes requests every ~1ms, times them, and shows the resp.Proto. (Please play with this duration).

Observations:

- No matter what I have tried to do with the Server/Client/Transport settings, it always seems to close after every connection (according to ConnState). Does this mean Keep-Alive is not having a chance to work?

- The proto (resp.Proto) always says HTTP/1.1 (measured from the client in the above program). Shouldn't it be HTTP2? or did I forget to do something to enable it?

- The cost of a connection seems to be around 10ms on my computer.

- But only if I make connections at a rate of ~1ms.

- (If you tweak the timer on line 50 to 1 second instead, the cost of a connection can be much higher, 30-60ms.).

- With the race detector on, the per connection cost goes to 150-250ms. Expected? Any way to ameliorate? This is what made me notice that things weren't behaving as well as I had hoped.

- GODEBUG=http2debug=2 has no visible effect on the program.

In my current use case, I'm trying to keep the latency down, since this is on the interactive path for my users. I'm interested in any techniques which do that. I'm OK with paying the odd slow connection but an extra 30ms most times in production seems high, when presumably it should be able to do session resumption or re-use an existing connection almost "for free", no? (250ms in development with -race is also proving to be a little painful).

If this is all to be expected, that would be good to know too.

Thanks in advance,

- Peter

Tamás Gulácsi

unread,

Mar 21, 2016, 3:03:14 PM3/21/16

to golang-nuts

Race detector always slows things down - if it weren't, I could be default on...

C Banning

unread,

Mar 21, 2016, 4:50:41 PM3/21/16

to golang-nuts

Try this -

...

http.HandleFunc("/", serverHandler)

log.Fatal(server.ListenAndServeTLS("", ""))

}

func serverHandler(w http.ResponseWriter, r *http.Request ) {

io.Copy(w, r.Body)

r.Body.Close()

Donovan Hide

unread,

Mar 21, 2016, 4:51:46 PM3/21/16

to Peter Waller, golang-nuts

Hi Peter!

This comment is probably of interest:

https://github.com/golang/go/blob/637e1aad5cecbdd12fb5f957e2b28ced5c3261fc/src/net/http/transport.go#L170-L178

Cheers,

Donovan.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Donovan Hide

unread,

Mar 21, 2016, 5:06:00 PM3/21/16

to Peter Waller, golang-nuts

This should give you the expected performance and protocol:

https://gist.github.com/donovanhide/f3e33714ca54a3f6b6f5

:-)

Peter Waller

unread,

Mar 21, 2016, 5:16:36 PM3/21/16

to Donovan Hide, golang-nuts

I think the source of my confusion might be this line?

https://github.com/golang/go/blob/637e1aad5cecbdd12fb5f957e2b28ced5c3261fc/src/net/http/transport.go#L146

Peter Waller

unread,

Mar 21, 2016, 5:22:18 PM3/21/16

to Donovan Hide, golang-nuts

Another question, why is Keep-Alive not working with HTTP2 disabled? Is there a bug, or did I break something?

Brad Fitzpatrick

unread,

Mar 21, 2016, 5:22:45 PM3/21/16

to Peter Waller, Donovan Hide, golang-nuts

I screwed up the Go 1.6 release, and DefaultTransport/DefaultClient don't have HTTP/2 enabled by default. But if you make your own Transport, then it does do HTTP/2. The fix should be in a future Go 1.6.x release.

Donovan Hide

unread,

Mar 21, 2016, 5:35:32 PM3/21/16

to Peter Waller, golang-nuts

Another question, why is Keep-Alive not working with HTTP2 disabled? Is there a bug, or did I break something?

Well, you aren't actually reading the response body. Do a ioutil.ReadAll(resp.Body) after defer resp.Body.Close() and keep-alive works as expected.

Peter Waller

unread,

Mar 21, 2016, 5:44:50 PM3/21/16

to Donovan Hide, golang-nuts

Thanks for the link Donovan, very much appreciated and clears up my misunderstanding.

Argh. I'm sure I have a lot of incorrect code in the wild! I totally missed this:

https://go-review.googlesource.com/#/c/1531/

I was under the incorrect impression closing the body without reading it was fine, possibly from how it used to behave in the past?

Thanks everyone for chipping in.

C Banning

unread,

Mar 21, 2016, 5:50:24 PM3/21/16

to golang-nuts, donov...@gmail.com

Seems to work for me - the patch I suggested took your example from ~8ms to ~150µs with HTTP/1.1 on my MacBook Pro.

Reply all

Reply to author

Forward