HTTPS and Auto Proxy auth?

1,050 views
Skip to first unread message

Vincent

unread,
Sep 20, 2015, 10:27:12 AM9/20/15
to Fiddler
Hi y'all - 

We're trying to use Fiddler to solve a problem: a program that doesn't understand proxies, but are in an environment where we must use a proxy to be allowed out to the internet.

So we're trying to get Fiddler to take all traffic on a given local address (on the machine running the program, not the target server) and redirect it, via an upstream proxy, to a remote server. In the process, Fiddler should handle the proxy auth.

What we have working so far:
  • We can get Fiddler to do the proxy auth handshake just fine as a reverse proxy, when using HTTP.
  • However, when we switch to using HTTPS, Fiddler no longer handles the proxy auth handshake, so things fail at the 407 from the upstream proxy.
This works, but only does HTTP:

oSession.host = "mytarget:80";
oSession.hostname = "mytarget";
oSession["X-AutoAuth"] = "(default)";

This doesn't work:

oSession.oRequest.headers.UriScheme = "https";
oSession.host = "mytarget:443";
oSession.hostname = "mytarget";
oSession["X-AutoAuth"] = "(default)";

Thoughts?

EricLaw

unread,
Sep 20, 2015, 10:48:30 AM9/20/15
to Fiddler
What exactly happens? Can you share a capture?

Keep in mind that a HTTP/407 isn't legal inside a HTTPS tunnel.

Stefan

unread,
Sep 21, 2015, 2:27:11 PM9/21/15
to Fiddler
Hi Eric,

I'm working with Vincent on the attempted reverse proxy setup. I have pasted some captures below for the following scenarios:
  1. HTTP tunnel as described in Vincent's first configuration snippet
  2. HTTPS tunnel as described in the second snippet
  3. HTTPS tunnel again, with fiddler.composer.HTTPSProxyBasicCreds set to the proxy creds
In the Composer Options window we have ticked the "Automatically Authenticate" box. This does an automatic NTLM negotiation for HTTP requests to the upstream proxy, but fails to do the same when CONNECT'ing for the HTTPS target. Instead, CONNECT fails with a 407 return code. Fiddler just gives up at this point, while e.g. Google Chrome authenticates via NTLM with the upstream proxy (when going direct, without Fiddler) and then successfully CONNECTs.

To add more colour to what we're trying to avoid: We want to use this on 'end user's' machines, so the big plus of Auto-Auth is the user not having to enter their creds in plaintext somewhere. 

If there's a way to get there with the below flow, that would be ideal.....

See below for the sanitised captures.

Thanks,
Stefan


=== Same Request that I always send to Fiddler ===

GET /dis/ HTTP/1.1                                                              
Host: localhost


=== (1) Reverse proxying a HTTP target ===

--- Request #1 ---


HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 PROXYHOST
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Proxy-Authenticate: Basic realm="PROXYHOST.corp-domain.com"
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 4119

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Error Message</TITLE>
... proxy error page ...
</HTML>

--- Request #2 ---

Proxy-Authorization: Negotiate BASE64FOO=====

HTTP/1.1 407 Proxy Authentication Required ( Access is denied.  )
Via: 1.1 PROXYHOST
Proxy-Authenticate: Negotiate MOREBASE64FOO=====
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 0

--- Request #3 ---

Proxy-Authorization: Negotiate EVENMOREBASE64FOO=====

HTTP/1.1 301 Moved Permanently
Via: 1.1 PROXYHOST
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 178
Date: Mon, 21 Sep 2015 16:58:55 GMT
Content-Type: text/html
Server: nginx

<html>
... expected content from http target...
</html>


=== (2) Reverse proxying a HTTPS target ===

--- Request #1 ---


HTTP/1.1 502 Fiddler - Gateway Connection Failed
Date: Mon, 21 Sep 2015 17:05:54 GMT
Content-Type: text/html; charset=UTF-8
Connection: close
Cache-Control: no-cache, must-revalidate
Timestamp: 18:05:54.871

[Fiddler] The connection to the upstream proxy/gateway failed. Closing Fiddler, changing your system proxy settings, and restarting Fiddler may help.  <br />System.Exception Upstream Gateway refused requested CONNECT.                                                                                                                                                                            

--- What Wireshark sees for Request #1 ---

CONNECT targethost.com:443 HTTP/1.1
Connection: close

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 PROXYHOST
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM
Proxy-Authenticate: Basic realm="PROXYHOST.corp-domain.com"
Connection: close
Proxy-Connection: close
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Content-Length: 727   

<HTML><HEAD><TITLE>Error Message</TITLE>
... proxy error page ...
</HTML>


=== (3) Reverse proxying a HTTPS target (again) ===

As above but after setting "prefs set fiddler.composer.HTTPSProxyBasicCreds user:pass" in QuickExec

--- Request #1 ---


HTTP/1.1 200 OK
Server: nginx
Date: Mon, 21 Sep 2015 17:18:15 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding

d
["v8","v8.0"]
0

--- What Wireshark sees for Request #1 ---

Proxy-Authorization: Basic ... is sent with the first CONNECT request, which means the CONNECT succeeds at the first attempt, no 407.

EricLaw

unread,
Sep 21, 2015, 4:40:47 PM9/21/15
to Fiddler
In the failing scenario(s), does Fiddler's Log tab contain the text:

fiddler.network.connect2> Upstream gateway demanded proxy authentication, which is not supported in this scenario. If your gateway supports BASIC authentication, in the QuickExec box, enter the following command prefs set fiddler.composer.HTTPSProxyBasicCreds yourusername:yourpassword

Stefan

unread,
Sep 22, 2015, 6:55:03 AM9/22/15
to Fiddler
Just checked. You're right, the log tab shows this:

11:46:52:9970 fiddler.network.connect2> Upstream gateway demanded proxy authentication, which is not supported in this scenario. If your gateway supports BASIC authentication, in the QuickExec box, enter the following command
            prefs set fiddler.composer.HTTPSProxyBasicCreds yourusername:yourpassword
HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Via: 1.1 PROXYHOST
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: 

We're trying this with Fiddler v4.6.0.2. Judging by the log message, looks like Fiddler doesn't do auto auth for CONNECT, or is there another way?

Thanks,
Stefan

EricLaw

unread,
Sep 22, 2015, 9:39:31 AM9/22/15
to httpf...@googlegroups.com
When Fiddler gets a HTTPS request, it must use a HTTPS connection to the server to send the request.

In most cases, Fiddler has an established HTTPS connection to the server already because the client previously sent a CONNECT and Fiddler in turn either directly connected to the HTTPS server or it took that CONNECT request and passed it along to the upstream gateway proxy.

In corner cases (e.g. the Request Composer or your scenario of a Reverse Proxy), Fiddler doesn't have an established connection to the target HTTPS server through the upstream proxy and it must itself establish one. This works fine if the proxy doesn't require authentication or if it accepts Basic auth and the fiddler.composer.HTTPSProxyBasicCreds preference is set. In 4.6.0.2 and earlier, Fiddler requires that the CONNECT request return a 200 and it will fail if a 407 is received.

Having said that... I do have code that should unblock your scenario; it hasn't shipped previously only because I didn't have any good way to test it. If you'd like to test a private build with this feature unlocked, please send me mail (Help menu > Send Feedback... in Fiddler) and I'll work with you to get this tested out.

thanks,
-Eric
Reply all
Reply to author
Forward
0 new messages