Using CONNECT to connect to an FTP server

[term srv]

 I understand the FTP support in Fiddler may be limited however I wonder about connecting to an FTP server via CONNECT. I have an application that I set to use the Fiddler debugging proxy and when it tries to make an FTP connection it uses CONNECT but that doesn't work. I'm using Fiddler v4.4.8.4 on Windows 7 x64 SP1 and Capture FTP requests is enabled. The FTP server I'm connecting to is working properly. I can reproduce using babyftp which is very basic ftp server.

C:\>socat TCP-CONNECT:localhost:8888 -
CONNECT 127.0.0.1:21 HTTP/1.1
Host: 127.0.0.1:21
User-Agent: socat
Connection: Keep-Alive

HTTP/1.1 200 Connection Established
FiddlerGateway: Direct
StartTime: 01:11:48.233
Connection: close

And nothing happens. If I hit return after the 200 OK then I can see the ftp 220 line, however I may or may not be able to send USER. With babyftp I can't but with kernel.org I can, for example. You can also use curl as another example:
curl --trace - -p -x localhost:8888 ftp://ftp.kernel.org/pub/site/sha256sums.asc

The only other thread I found that may be related is from 2010:
Monitor FTP traffic

Maybe this has something to do with buffers, because the data is being sent. I looked at rfc2817 which touches on CONNECT but I don't see any guidance for this issue. I think but am not sure (I haven't considered the FTP RFCs) that this could be a bug in Fiddler although as I said I understand FTP support is limited so maybe this is known. Any information you can provide will be helpful so I can decide how to handle this on my end. Thank you

[EricLaw]

Fiddler's FTP support is based on the CERN-gateway standard (whereby the client sends a HTTP request containing a FTP URL) used by browsers. Fiddler cannot parse FTP traffic from inside a CONNECT tunnel. Fiddler should blindly pass the data through the connection in both directions (if not, it's a bug; please send me a NetMon CAP or a Wireshark PCAP) but it will not be able to display the traffic in the Web Sessions list.

[EricLaw]

Based on the PCAP you sent me, I think I see the problem here. The problem is that Fiddler is waiting for the client to send some data; if that data isn’t a TLS ClientHello then Fiddler converts this connection into a blind-tunnel and simply shuffles bytes back and forth between the client and the server. However, in this scenario, the client doesn’t send anything until the server sends back its FTP/220 welcome message, and because Fiddler is waiting on the client before anything is sent from the server, there’s a deadlock.

 

You can manually resolve this problem by doing something like: Rules > Customize Rules > OnBeforeRequest

 

     if ((oSession.HTTPMethodIs("CONNECT") && oSession.host.Contains(":21"))

             {

           oSession["x-no-decrypt"] = "FTP traffic smuggled via connect tunnel";

     }

[term srv]

I just tried this but it didn't work. The server is sending the reply but libcurl is just waiting. I tried in socat as well. capture confirms data is being sent.

File Transfer Protocol (FTP)
    220 Welcome to mirrors.rit.edu.\r\n
        Response code: Service ready for new user (220)
        Response arg: Welcome to mirrors.rit.edu.

To reproduce try:

CONNECT mirrors.rit.edu:21 HTTP/1.1
Host: mirrors.rit.edu:21
Connection: Keep-Alive

I must have forgot I started this thread two years ago since I didn't follow up with you then.

[EricLaw]

1. What version number of Fiddler, and 

2. Can you send a SAZ file?

[term srv]

Telerik Fiddler Web Debugger (v4.6.2.32002)
Built: Tuesday, July 26, 2016

The file is attached. I tried twice that's why there are two connections in it. In both cases I waited for the server to timeout.

[Eric Lawrence]

The Log file attached does not show the required x-no-decrypt flag on the CONNECT tunnels.

[term srv]

Sorry, what happened was the flag didn't work so I removed it and when I ended up reproducing the issue for you it was without the flag. Attached shows the same issue but this time with the flag.

== FLAGS ==================
BitFlags: [ResponseGeneratedByFiddler] 0x100
X-ABORTED-WHEN: SendingResponse
X-CLIENTIP: ::ffff:127.0.0.1
X-CLIENTPORT: 54959
X-EGRESSPORT: 54960
X-HOSTIP: 129.21.171.72
X-NO-DECRYPT: FTP traffic smuggled via connect tunnel
X-PROCESSINFO: proxytest:1344
X-RESPONSEBODYTRANSFERLENGTH: 0

You can reproduce using curl -v -p -x localhost:8888 ftp://mirrors.rit.edu

[Eric Lawrence]

Thanks for getting back to me!

The code sample was incomplete. It should look like:

     if ((oSession.HTTPMethodIs("CONNECT") &&

 // Can't filter just on port because FTP uses multiple ports.

 // Could filter based on client's process name or whatever.

          oSession.host.Contains("rit.edu")) 

     {

       oSession["x-no-decrypt"] = "FTP traffic smuggled via connect tunnel";

       oSession["x-no-parse"] = "FTP traffic smuggled via connect tunnel";

     }

Alternatively, you can call .Ignore() on the Session to make it disappear and that will set these flags automatically.

[term srv]

Thanks Eric, adding x-no-parse worked. I decided to filter based on process name. For anyone who is interested it looks like this:

if (oSession.HTTPMethodIs("CONNECT") && oSession.LocalProcess.StartsWith("curl:", StringComparison.OrdinalIgnoreCase)) {