Reading WebSocket responses from Firebase using FiddlerCore in a Forms WebBrowser

[Johnny Malloy]

I have a Windows Forms app that uses the WebBrowser control. The control loads a website that provides SSO login to Google using Firebase. I'd like for my app to be able to piggyback the sign-on data coming back from Firebase and store it locally for reference.

I am using the latest FiddlerCore4 from NuGet (4.4.8.4) which works just fine reading both decrypted HTTPS and the WebSocket data as well. However my app obviously posts a cert warning for the HTTPS communications and I'd like to not have those popups appear.

I have tried implementing two different cert adding techniques I have found in the Fiddler book. In both cases, when I add a trusted cert the WebSocket requests stop coming through. FiddlerApplication.Log.OnLogString event handler returns:

** LogString: Upgrading Session #4 to websocket

But then all other request capturing does not happen. This only is the case when I add the trusted cert. When I have the end user OK the cert manually, the data is captured by Fiddler.

I have tried both the CreateAndTrustRoot() method and the setMachineTrust(X509Certificate2 oRootCert) methods suggested in the Debugging with Fiddler manual.

I know there is some websocket functionality exposed in the later versions of FiddlerCore, but since I have been able to capture the requests using the BeforeResponse event handler without the trusted cert, I am confused about which direction to go.

Here's my code

        public frmForm()

        {

            InitializeComponent();

            SourceBrowser.DocumentCompleted += SourceBrowser_DocumentCompleted;

            FiddlerApplication.BeforeResponse += FiddlerApplication_BeforeResponse;

            if (CreateAndTrustRoot())

            {

                System.Threading.Thread.Sleep(200);

                FiddlerApplication.Startup(8888, FiddlerCoreStartupFlags.DecryptSSL);

            }

            URLMonInterop.SetProxyInProcess("127.0.0.1:8888", "<-loopback>");

        }

        private bool CreateAndTrustRoot()

        {

            bool bCreatedRootCertificate, bTrustedRootCert;

            // Ensure root exists 

            if (!CertMaker.rootCertExists())

            {

                bCreatedRootCertificate = CertMaker.createRootCert();

                if (!bCreatedRootCertificate)

                    return false;

            }

            setMachineTrust(CertMaker.GetRootCertificate());

            // Ensure root is trusted 

            if (!CertMaker.rootCertIsTrusted())

            {

                bTrustedRootCert = CertMaker.trustRootCert();

                if (!bTrustedRootCert) return false;

            }

            return true;

        }

        private bool setMachineTrust(X509Certificate2 oRootCert)

        {

            try

            {

                X509Store certStore = new X509Store(StoreName.Root, StoreLocation.LocalMachine);

                certStore.Open(OpenFlags.ReadWrite);

                try

                {

                    certStore.Add(oRootCert);

                }

                finally

                {

                    certStore.Close();

                }

                return true;

            }

            catch (Exception eX)

            {

                localLog.LogInfo(new LogInfo { Message = String.Format("Exception setting machine trust") });

                return false;

            }

        }

        void FiddlerApplication_BeforeResponse(Session oSession)

        {

            localLog.LogInfo(new LogInfo { Message = String.Format("** Session Request URL: {0}, RESPONSE Body: {1}", oSession.url, Encoding.ASCII.GetString(oSession.ResponseBody)) });

            Uri sessionUri = new Uri(oSession.fullUrl);

            if (sessionUri.Host.Contains("firebaseio.com") && sessionUri.Query.Contains("&d0="))

            {

                    FireBase fb = new FireBase();

                    fb.LogAgent(response);

            }

        }

[EricLaw]

Thanks for your support of my book!

> I have been able to capture the requests using the BeforeResponse event handler 

WebSocket traffic never flows through a BeforeResponse handler; instead each message goes through the OnWebSocketMessage event handler.

The site confused you-- because the certificate wasn't trusted, the WebSocket connection failed entirely, so the site used a fallback to use a non-WebSocket transport instead. After you fixed the WebSocket traffic (by trusting the certificate), the WebSocket started getting used and the fallback transport that went through BeforeResponse was no longer used. 

Tip: Please don't manually try to convert response bytes to text; use GetResponseBodyAsString() which will work properly.

[Johnny Malloy]

Thanks for the response. I have noticed how quickly and well you handle random questions on all the different sites/forums. I just want to note how awesome that is and mention that Fiddler and FiddlerCore are fantastic and very useful tools. I also replaced the manual byte conversions heh.

Anyway your reply was clear and helped me isolate some of my confusion. I did and do notice Firebase replying with XMLHTTPRequest actions and should have realized that this was a fallback.

However it seems like the trusted cert aspect is still a minor problem. I added in the OnWebSocketMessage handler but sometimes I still see the websocket responses still coming through the BeforeResponse handler. I assume this is is due to my cert trusting actions not (always) working right and subsequently going to the fallback which Fiddler sees as "regular" http requests. It is strange because sometimes in these scenarios the OnWebSocketMessage will actually get handled and have a payload with key parameters but no values in them.

But sometimes the WebSocket response does contain the data. I am going to work on seeing if my cert trusting code is working properly but any insight you might have would be appreciated. I might have to just handle either case - I am just moving the data into a singleton or some local store so it shouldn't be too bad.

I am using Windows 7 and also setting IE compatibility mode to version 9 in the registry if that adds any insight on the certs.

[EricLaw]

Internet Explorer 9 doesn't support WebSockets, so I'm not sure how that is working at all. :-)

If you save a SAZ file containing the traffic, I'm happy to have a look at it. This should pretty quickly reveal what's going on.

[Johnny Malloy]

I'm assuming this is because its only compatibility mode IE9, when the actual browser version is IE11. If I can't figure it out myself I will post a followup.

[Johnny Malloy]

I seem to be having some confusing behavior with CertMaker that may or may not be related to the previous problem with WebSockets. The relevant code I am using is below.

When I run the app the first time, with no cert, everything works fine (WebSocket reuqests come through as expected in the event handler). The app asks to create the cert. If I shut down the app and restart, CertMaker.rootCertExists() returns false but CertMaker.rootCertIsTrusted() returns true. The code proceeds to create a new Root Cert but then bypasses the CertMaker.rootCertIsTrusted() statement. When this happens I get the ** LogString: [Fiddler] No HTTPS request was received message - which I take it means the cert is no good any more.

I have SAZ dumps if you want them, but I am assuming the problem here is with my cert code or more likely my understanding of certs.

        private bool CreateAndTrustRoot()

        {

            //return true;

            //CertMaker.removeFiddlerGeneratedCerts(true);

            bool bCreatedRootCertificate, bTrustedRootCert;

            // Ensure root exists 

            if (!CertMaker.rootCertExists())

            {

                bCreatedRootCertificate = CertMaker.createRootCert();

                if (!bCreatedRootCertificate)

                    return false;

            }

            // Ensure root is trusted 

            if (!CertMaker.rootCertIsTrusted())

            {

                //bTrustedRootCert = SetMachineTrust(CertMaker.GetRootCertificate());

[EricLaw]

Is your FiddlerCore application using CertMaker.dll+BCMakeCert.dll or makecert.exe?

[Johnny Malloy]

CertMaker.dll+BCMakeCert.dll

[Johnny Malloy]

FYI based on your inquiry I switched over to makecert.exe and things seem to be working at the moment.