How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

357 views
Skip to first unread message

Hugo Bollon

unread,
Mar 15, 2021, 6:37:51 PM3/15/21
to golang-nuts
Hi!
I'm actually building an automation tool based on Selenium with Go called IGopher and I have had a few requests to implement native proxy support.  
However, I am facing an issue with those with authentication...  
I can't send the proxy credentials to Chrome and without them it asks through an alert box for authentication that I can hardly interact with through Selenium (I'm not even sure it's possible in headless mode) .

So I thought of an intermediary proxy system hosted locally by my program which will add the Proxy-Authorization header and transfer the request to the remote proxy:

IGopher_proxies.jpg

Something like this:  proxy-login-automator

I'm not very familiar with proxies to be honest, but I tried this approach using NewSingleHostReverseProxy
```go
var (
localServerHost  string
remoteServerHost string
remoteServerAuth string
)

// ProxyConfig store all remote proxy configuration
type ProxyConfig struct {
IP       string `yaml:"ip"`
Port     int    `yaml:"port"`
Username string `yaml:"username"`
Password string `yaml:"password"`
Enabled  bool   `yaml:"activated"`
}

func PrintResponse(r *http.Response) error {
logrus.Infof("Response: %+v\n", r)
return nil
}

// LaunchForwardingProxy launch forward server used to inject proxy authentication header
// into outgoing requests
func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) error {
localServerHost = fmt.Sprintf("localhost:%d", localPort)
remoteServerHost = fmt.Sprintf(
"http://%s:%d",
remoteProxy.IP,
remoteProxy.Port,
)
remoteServerAuth = fmt.Sprintf(
"%s:%s",
remoteProxy.Username,
remoteProxy.Password,
)

remote, err := url.Parse(remoteServerHost)
if err != nil {
panic(err)
}

proxy := httputil.NewSingleHostReverseProxy(remote)
d := func(req *http.Request) {
logrus.Infof("Pre-Edited request: %+v\n", req)
// Inject proxy authentication headers to outgoing request into new Header
basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
req.Header.Set("Proxy-Authorization", basicAuth)
logrus.Infof("Edited Request: %+v\n", req)
logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, req.URL.Host, req.URL.Port())
}
proxy.Director = d
proxy.ModifyResponse = PrintResponse
http.ListenAndServe(localServerHost, proxy)

return nil
}
```

With this code snippet, I'm able to intercept the request and update the header. 
However, resending the CONNECT request fails with the following output:

```
INFO[0028] Pre-Edited request: &{Method:CONNECT URL://google.com:443 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:35610 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000164300}  function=func1 line=59

INFO[0028] Edited Request: &{Method:CONNECT URL://google.com:443 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Authorization:[Basic <auth>] Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:35610 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000164300}  function=func1 line=63

INFO[0028] Scheme: , Host: google.com:443, Port: 443     function=func1 line=64

2021/03/15 21:35:11 http: proxy error: unsupported protocol scheme ""
```

What am I doing wrong? Is there a way to send a CONNECT request without scheme in Go?
Maybe I'm doing something wrong or my approach to this problem is wrong. Are there better methods to achieve my goals?

If you have an idea to complete what I did or any other method, please let me know! :)

You can find all IGopher sources here: GitHub repository (Proxy stuff excluded)

Vladimir Varankin

unread,
Mar 16, 2021, 4:04:31 AM3/16/21
to golang-nuts
Hey there,

Seems the issue hides in the chunk, where you overwrite reverse proxy's "Director" method, which NewSingleHostReverseProxy creates internally. Since your own director doesn't set the client request's Schema and Host, you have to either do that manually or make sure you call the original director.

Try doing the following:

proxyDirector := proxy.Director // ← keep the original director
d := func(req *http.Request) {
        logrus.Infof("Pre-Edited request: %+v\n", req)

        proxyDirector(req) // ← call the original director to make sure the request will go through the proxy

        // Inject proxy authentication headers to outgoing request into new Header
        basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
        req.Header.Set("Proxy-Authorization", basicAuth)
        logrus.Infof("Edited Request: %+v\n", req)
        logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, req.URL.Host, req.URL.Port())
}
proxy.Director = d

Also, have a look at the implementation of NewSingleHostReverseProxy https://go.googlesource.com/go/+/go1.16/src/net/http/httputil/reverseproxy.go#142

Cheers,
V.

Hugo Bollon

unread,
Mar 16, 2021, 6:17:49 AM3/16/21
to golang-nuts
Thank you for your advice that I applied.
But now I have a 407 Proxy Authentication Required error, while the header is added to the request...
Here is the output:

INFO[0019] Pre-Edited request: &{Method:CONNECT URL://google.com:443 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000814240}  function=func1 line=60

INFO[0019] Edited Request: &{Method:CONNECT URL:http://51.178.xx.xx:3128/ Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Authorization:[Basic <auth>] Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:45382 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000814240}  function=func1 line=69

INFO[0019] Scheme: http, Host: 51.178.xx.xx:3128, Port: 3128  function=func1 line=70

INFO[0019] Response: &{Status:407 Proxy Authentication Required StatusCode:407 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Content-Language:[en] Content-Length:[3520] Content-Type:[text/html;charset=utf-8] Date:[Tue, 16 Mar 2021 10:03:44 GMT] Mime-Version:[1.0] Server:[squid/3.5.27] Vary:[Accept-Language] Via:[1.1 vps799016 (squid/3.5.27)] X-Cache:[MISS from vps799xxx] X-Cache-Lookup:[NONE from vps799xxx:3128] X-Squid-Error:[ERR_CACHE_ACCESS_DENIED 0]] Body:0xc0004de180 ContentLength:3520 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc0005ee100 TLS:<nil>}  function=PrintResponse line=33


PS: Is it possible on Google Groups to format code snippets? Markdown not seems to be supported

Vladimir Varankin

unread,
Mar 17, 2021, 4:30:22 AM3/17/21
to golang-nuts
I think I didn't get what you're building right. Now, it looks like, instead of implementing a custom RR's director, you need to configure its Transport [1], which will be aware of your auth proxy in the middle. Have a look at net/http.Transport.Proxy field [2] for that.

Hugo Bollon

unread,
Mar 17, 2021, 11:00:22 AM3/17/21
to golang-nuts
Yeah, I'm not very comfortable with networking so it's possible I'm not using the right terms...

I want to be able to allow my users to use, with Selenium (and therefore Chrome), a proxy with authentication. The problem is that I cannot send the proxy credentials to the Chrome instance or interact with the proxy connection alert box.

So basically, what I want to do, is a localhost proxy managed by my program which will receive all Chrome CONNECT requests, inject the Proxy-Authorization header to authenticate requests and send them back to the user remote proxy.

I managed to get something with a TCP tunnel, I relied on the source code of io.Copy source code.
I managed to intercept the request from the incoming net.Conn, decode the http request from the tcp packet, add the Proxy-Authorization at the end, re-encode the packet and finally write it to the outgoing net.Conn (to the remote proxy )
But I'm getting a 400 Server Error on the remote proxy side...

(It's WIP stuff, so it's not cleaned or optimized yet. I just pushed it so you could check it out)

Hope I am clear enough ...
Anyways, thanks for your help! :)

Hugo Bollon

unread,
Mar 17, 2021, 1:36:17 PM3/17/21
to golang-nuts
I fixed the 400 Server Error issue and add a request checking before inject Proxy-Authorization header in order to alter only GET or CONNECT requests: https://github.com/hbollon/IGopher/blob/proxy/internal/proxy/pipe.go
But now I'm getting this error after curl request: OpenSSL SSL_read: error:1409442E:SSL routines:ssl3_read_bytes:tlsv1 alert protocol version, errno 0

Maybe my approach is completely crazy or wrong but I feel can succeed lmao
Reply all
Reply to author
Forward
0 new messages