Awful performance and failure with httputil.ReverseProxy

[Kamil Kisiel]

I'm trying to proxy a server using httputil.ReverseProxy but am seeing abysmal performance (about 1/10th the speed) and even worse, transfers seem to abort before they complete.

For example, via the proxy:

sambuca:~ kamil$ curl http://proxy.example.com/foo > /dev/null

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

 99 12.7M   99 12.7M    0     0   353k      0  0:00:37  0:00:36  0:00:01     0

curl: (18) transfer closed with 14407 bytes remaining to read

And via a direct connection:

sambuca:~ kamil$ curl https://real.example.com/foo > /dev/null

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100 12.7M  100 12.7M    0     0  4079k      0  0:00:03  0:00:03 --:--:-- 4559k

The code I'm using to build the ReverseProxy looks like this:

// reverseProxy adds a reverse proxy handler rooted at path which proxies all requests through target.

func reverseProxy(path string, target *url.URL) {

    targetQuery := target.RawQuery

    director := func(req *http.Request) {

        req.URL.Scheme = target.Scheme

        req.URL.Host = target.Host

        req.Host = target.Host // workaround golang.org/issue/5692

        req.URL.Path = singleJoiningSlash(target.Path, req.URL.Path)

        if targetQuery == "" || req.URL.RawQuery == "" {

            req.URL.RawQuery = targetQuery + req.URL.RawQuery

        } else {

            req.URL.RawQuery = targetQuery + "&" + req.URL.RawQuery

        }

        fmt.Println(req.URL)

    }

    http.Handle(path, http.StripPrefix(path, &httputil.ReverseProxy{Director: director, FlushInterval: 100 * time.Millisecond}))

}

func singleJoiningSlash(a, b string) string {

    aslash := strings.HasSuffix(a, "/")

    bslash := strings.HasPrefix(b, "/")

    switch {

    case aslash && bslash:

        return a + b[1:]

    case !aslash && !bslash:

        return a + "/" + b

    }

    return a + b

}

Most of it is copied from httputil.NewSingleHostReverseProxy. I've tried various values of FlushInterval but they don't seem to improve the situation.

Any ideas as to what the problem might be?

[Brad Fitzpatrick]

I've never profiled it, but I don't expect it to be that bad.

I use httputil.ReverseProxy in front of a number of my personal sites and have never noticed any performance problems or aborted transfers.

Unfortunately I don't have time to debug this.  If you have a complete program which demonstrates the issue in isolation, please file a bug.

[Kamil Kisiel]

Thanks, I was kind of hoping there was something I was obviously missing but I guess not.

I have done some profiling, as the program pegs the CPU at 100% and it seems it spends about 80% of the time in crypto/des. Not sure if that's expected with just 1 request or not. I'll spend some time later this week to whittle it down to a simple testcase and post a bug.

[Brad Fitzpatrick]

I imagine this is likely https://code.google.com/p/go/issues/detail?id=4299

Maybe crypto/des needs love.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Kamil Kisiel]

The DES stack trace in comment #5 is just like what I am seeing, so that's likely it. Maybe I will change the order of the cipher suites to see if AES is any faster.

That still doesn't answer why the request doesn't even complete, and I haven't configured any timeouts on the server being proxied (also written in Go) so presumably that shouldn't be the problem.

[Brad Fitzpatrick]

I'm introduced in repro cases for requests not completing if you've got one.