Re: Newbie Questions

[EricLaw]

1. When going through a proxy, HTTPS traffic flows through what are called "CONNECT Tunnels". The reason HTTPS traffic is special is that it encrypts all of the data so that an intermediary (like Fiddler) cannot normally see it. The problem with doing that is that if the intermediary proxy (like Fiddler) can't see the traffic, it doesn't know where to send it. So the client sends a HTTP CONNECT request to the proxy and specifies the target destination. E.g. CONNECT husband.umd.edu:443 HTTP/1.1. (Note that the request line doesn't include a URL, only the hostname). 

 

The proxy is expected to open a connection to the target and then just blindly shuffle bytes back and forth. Fiddler could show the hostname in the HOST column and in the URL column, but for readability in English it instead shows the HOST only in the URL column and shows "Tunnel to" in the host column.

 

2. Yes, the fact that the browser felt the need to establish a connection to ssl.google-analytics.com is a good sign that the page it loaded includes one or more tracking requests to the Google servers. If you enable HTTPS decryption, you can see the actual requests and not just the tunnels.

 

3. SPDY is a new protocol that sits between TCP/IP and HTTP; it reformats HTTP messages in a compressed binary way to improve performance. SPDY traffic is always over SSL and hence you see it in Tunnels. When a tunnel is established, Fiddler looks at flags in the client and server's handshakes to determine if SPDY is likely in use and if so, it shows this icon. Note that if you enable HTTPS decryption, SPDY will no longer be used because the flags sent in a SPDY handshake cannot be sent by Fiddler and thus the client and server speak plain HTTPS instead.

[Ashish Negi]

@EricLaw Are these HTTP Tunnel Messages generated by the Fiddler ?

[EricLaw]

I'm not sure I understand your question.

A client, when sending HTTPS traffic through a proxy (like Fiddler) instructs the proxy to CONNECT to the target host. So the client is generating the CONNECT message and Fiddler displays it to you. Fiddler displays additional information about the HTTPS traffic (e.g. what the cipher is and so forth) by analyzing the first few hundred bytes that flow through that tunnel.

[Peter Cawdron]

Eric,

Can you clarify... 

• Not all HTTPS traffic will show up as using a connect tunnel (is it only used for particular cyphers?)
  
• Fiddler notes, "HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list." Is this the very next line entry? As I don't always see the same host/result combination.

Thanks,

[EricLaw]

Not all HTTPS traffic will show up as using a connect tunnel (is it only used for particular cyphers?)

A CONNECT tunnel is used by any client that knows that it is going through a proxy (e.g. your browser, your applications, your mobile devices, etc). It is not used by requests that are targeted at Fiddler itself (e.g. if you have Fiddler running as a reverse proxy for HTTPS). It is not used when Fiddler itself issues a HTTPS request (e.g. from the Composer) unless you also have another proxy upstream (e.g. ISA/Squid/etc).

Fiddler notes, "HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list." Is this the very next line entry? As I don't always see the same host/result combination.

Not necessarily, no. After the CONNECT tunnel is established, the client can send 0-∞ requests through that tunnel, at whatever times it wants. As a consequence, it's entirely normal for there to be a small gap between when the CONNECT is made and when a request that uses it is captured, and if other requests come in between that time, they'll be shown between the tunnel and the requests using that tunnel.

It's also important to reiterated that a client may reuse an existing CONNECT tunnel for multiple requests; there's not a 1:1 correspondence between tunnels and HTTPS requests.

[Mabraygas]

Thank you EricLaw for your answer in http://stackoverflow.com/questions/16171277/fiddler-tunnelled-http-requests-to-port-443.

Here I have another question. Enabled HTTPS Decryption in fiddler, I found my mobile app client will send a tunnels(also find SSLv3 ClientHello Handshake in it) request before every actual https request. I wonder is this caused by "client didn't make a keep-alive https connection with server side" or is cause by "client know there is a proxy so it do this process mandatory for every request"?

Thanks.

[EricLaw]

This typically indicates that the client or server is not configured to use Keep-Alive; you should look at the HTTP response headers from the server and see whether there's a Connection:close response header. If not, then chances are it's the client that is failing to use keep-alive.

[Mabraygas]

Eric, some new findings and description update.

1. There is a tunnel request before every actual https request if I uncheck the fiddler's "Reuse client connections" box. However, if I check this box, a tunnel request appears only if I don't send request to server for a while, but not before every https request.

2. If I uncheck the "Reuse server connctions" box, I find that every https actual request's "TCP/IP Connect" and "HTTPS Handshake" timer is greater than 0. If I check this box, they seems all 0.

3. Both tunnel request and https request are all have "Connection: keep-alive" header. But the response of tunnel request has "Connection: close", response of https request has nothing. No matter the two boxes described above checked or unchecked.

For 1 and 2, I think they makes sense. But for 3, I don't quite understand the reason. And, the main reason I'm thinking these is that I want to verify whether my client connects a keep-alive https connection with my server. But seems as long as I use fiddler as a proxy(no matter decryption https or not), client side will use a tunnel to connect with server. Which may not be the original action in real scenario. Is that right?..

Thank you!

[Richard Mangum]

Make sure you restart the mobile device after installing the certificate. I had tunnel.... until i restarted using the ipad

[Eric Lawrence]

Which client OS and version are you asking about?

Clients may behave differently in terms of their Keep-Alive behavior in the presence of a proxy, but they generally do not.

iOS 10 Beta had a bug whereby it would not use a HTTPS Tunnel where the response had a Connection: close header (this, per spec, is supposed to apply to the tunnel, not the requests/responses within the tunnel). I suppose it's remotely possible that when they fixed that bug they left another one whereby a tunnel with that header would not be properly reused; try removing that header to see. Check out the script in this post: https://textslashplain.com/2016/07/27/using-fiddler-with-ios-10-and-android-7/ and see if it changes things.