Server response time hike.

Manideep Attanti

unread,

Jul 29, 2016, 10:22:51 AM7/29/16

to golang-nuts

The pprof stats show around 900 go-routines and most of them are waiting on read request. This is causing server response time to shoot up. Any insights on how to debug this.

goroutine 1842715 [IO wait, 1 minutes]:
net.runtime_pollWait(0x7f753ba71ef0, 0x72, 0xc8270e9000)
	/root/.gvm/gos/go1.6.2/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc841f5a7d0, 0x72, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc841f5a7d0, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc841f5a770, 0xc8270e9000, 0x1000, 0x1000, 0x0, 0x7f753da8a050, 0xc82000a1c0)
	/root/.gvm/gos/go1.6.2/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc826364ec8, 0xc8270e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/net/net.go:172 +0xe4
net/http.(*connReader).Read(0xc82420db40, 0xc8270e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/net/http/server.go:526 +0x196
bufio.(*Reader).fill(0xc84b94a8a0)
	/root/.gvm/gos/go1.6.2/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc84b94a8a0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/bufio/bufio.go:132 +0xcc
net/http.(*conn).readRequest(0xc842ca8b80, 0x0, 0x0, 0x0)
	/root/.gvm/gos/go1.6.2/src/net/http/server.go:702 +0x2e6
net/http.(*conn).serve(0xc842ca8b80)
	/root/.gvm/gos/go1.6.2/src/net/http/server.go:1425 +0x947
created by net/http.(*Server).Serve
	/root/.gvm/gos/go1.6.2/src/net/http/server.go:2137 +0x44e

All other goroutines are waiting on same read request.

Thanks.

Manideep

James Bardin

unread,

Jul 29, 2016, 10:38:49 AM7/29/16

to golang-nuts

The open connections are either totally irrelevant, or are the _result_ of whatever is causing the response time to increase.

Are you setting a reasonable ReadTimeout and WriteTimeout on your server?

Maybe slow/stalled clients are holding server resources?

Sokolov Yura

unread,

Jul 30, 2016, 9:13:20 AM7/30/16

to golang-nuts

Couldn't it be Keep-Alive connections?
What if you add "Connection: close" header to every server response?
Or call SetKeepAlivesEnabled on server https://golang.org/pkg/net/http/#Server.SetKeepAlivesEnabled

пятница, 29 июля 2016 г., 17:22:51 UTC+3 пользователь Manideep Attanti написал:

Reply all

Reply to author

Forward