Performance of httputil.ReverseProxy

[Mehran Saliminia]

 I have a performance problem with reverseproxy in golang.

I wrote a very simple reverseproxy in Go and then started it on a single machine with 2 CPU Cores and 4G of RAM. The maximum throughput that I could reach was about 6k reqs/sec, whereas my backend app was able to handle over 18k reqs/sec (it's a very simple nodejs app). 

The proxy server gets saturated (100% CPU util) just with 6k requs/seconds. When I don't set MaxIdleConnsPerHost, the throughput even worse, about 4k reqs/sec. Has anybody already profiled the golang's builtin reverseproxy?

Is it really its maximum capacity?

this is my code:

func main() {

backend, err := url.Parse("http://XXX.XX.XX.XX:3000")

if err != nil {

panic(err)

}

        

  go func() {

log.Println(http.ListenAndServe("127.0.0.1:6060", nil))

}()

 

proxy := httputil.NewSingleHostReverseProxy(backend)

proxy.Transport = &http.Transport{

Dial: func(network, addr string) (net.Conn, error) {

conn, err := net.DialTimeout(network, addr, 5*time.Second)

if err != nil {

return conn, err

}

return conn, err

},

DisableKeepAlives: false,

MaxIdleConnsPerHost: 200,

}

http.HandleFunc("/", handler(proxy))

err = http.ListenAndServe(":3003", nil)

if err != nil {

panic(err)

}

}

 

func handler(p *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) {

return func(w http.ResponseWriter, r *http.Request) {

w.Header().Set("X-VCAP", "TEST")

p.ServeHTTP(w, r)

}

} 

[James Bardin]

The http Transport as used in the reverse proxy has trouble with the type of traffic generated by synthetic benchmarking tools. You're probably seeing something similar to this: https://github.com/golang/go/issues/6785

[Mehran Saliminia]

Yes, it seems to be similar. Do you mean that it doesn't happen in production?

[James Bardin]

On Sat, Dec 13, 2014 at 12:08 AM, Mehran Saliminia <msali...@gmail.com> wrote:

Yes, it seems to be similar. Do you mean that it doesn't happen in production?

In general, yes it doesn't happen in production. Generating load using common http load testing tools will create a steady stream of new requests, never giving the transport's idle connection handling time to catch up once it falls behind. In production you usually aren't limited to a few clients that continuously make new requests as fast as possible. 

Check your GOMAXPROCS setting too. Some concurrency may help handle the load, but too much just increases contention on the transport. 

Another thing I've done is to use multiple transports to handle the load. There's a few ways you could wire that up, but it's easy to start with reverseproxy.go and modify it to suit your needs.

[Mehran Saliminia]

Thank you James for your tips!