Fiddler Monitor Programmatic HTTPS session

[karloz...@gmail.com]

I am programmatically making HTTPS calls using Microsoft COM and doing HttpSendRequest api.   Is there a way or a switch setting in Fiddler to see and capture those calls and returns.  I do see calls and returns status in Fiddler when I create an object reference to IE and drive the browsers.

But I would like to see actual calls and returns from the api traffic.

I am new to Fiddler and hoping that I am just missing something simple.

[Eric Lawrence]

Please elaborate on how specifically you're "making HTTPS calls using Microsoft COM." In general, WinINET applications adopt the system proxy settings (which point to Fiddler when it's "attached"). But it's possible to write a WinINET-based application that ignores the system proxy settings.

(It's also possible to configure Fiddler to ignore non-browser traffic. Use Help > Troubleshoot to eliminate that as a possibility.

[karloz...@gmail.com]

Thanks Eric for the reply.   We are using a product from West Wind.  Its basically a wrapper product around Microsoft HTTP api's..   and tone of the sample calls looks like

lnRetval=HttpSendRequest(hHTTPResult,tcHeaders,LEN(tcHeaders),this.cPostBuffer,LEN(this.cPostBuffer))

I did find in the help manual references to setting up a netsh setting for WinHTTP Apps. I tried that but it did not seem to make a difference.   I still see no traffic

[Eric Lawrence]

Rick (owner of West-Wind) is a big user of Fiddler and I'm confident that he can help with this. To be clear, Microsoft has *two* HTTP APIs, WinINET (which tends to adopt the system proxy settings automatically) and WinHTTP (which often does not).

The procedure for changing the WinHTTP proxy settings varies by OS-- which version of Windows are you using.

[karloz...@gmail.com]

Eric again thanks so much for your quick response and time.   Truly appreciate it.  

I will reach out to Rick too.   And maybe as a team effort someone can tell me the piece that I am missing

To answer your direct question on what O/S I am using Windows 7 64  but the app is 32 bit.  

I did come across your Blog posting on Using Fiddler with WinHTTP from 4/29/2013.  

I added (I believe I add correctly) the following to the fiddler custom script (in red)..  (as outlined in your posting)

    /*
    static function OnShutdown() {
            MessageBox.Show("Fiddler has shutdown");
    }
    */

   
    static function OnAttach() {
        UpdateWinHTTPSettings();
        MessageBox.Show("Fiddler is now the system proxy");
    }
   

   
    static function OnDetach() {
        UpdateWinHTTPSettings();
        MessageBox.Show("Fiddler is no longer the system proxy");
    }
   

    // The Main() function runs everytime yo

And also added (in red)... 

    function DoManualReload() {
        FiddlerObject.ReloadScript();
    }
   
    public static ToolsAction("Update WinHTTPSettings")
    function UpdateWinHTTPSettings() {
        var oPSI: System.Diagnostics.ProcessStartInfo
            = new System.Diagnostics.ProcessStartInfo();
        oPSI.UseShellExecute = true;
        oPSI.FileName = "netsh.exe";
        oPSI.Verb = "runas";
        oPSI.Arguments = "winhttp import proxy ie";
        System.Diagnostics.Process.Start(oPSI);

        // Re-run 32bit version
        oPSI.FileName = oPSI.FileName =
            Environment.SystemDirectory.Replace("system32", "syswow64")
            + "\\netsh.exe";   
        System.Diagnostics.Process.Start(oPSI);
    }

    public static ContextAction

[Eric Lawrence]

What WestWind classes are you using specifically?  did you set this to "preconfig"?  https://webconnection.west-wind.com/docs/_0jj1avjvc.htm

[karloz...@gmail.com]

*** Core classes

SET CLASSLIB TO wwIPStuff ADDITIVE

SET PROCEDURE TO wwHTTP ADDITIVE

SET PROCEDURE TO wwUtils ADDITIVE

SET PROCEDURE TO WWPOP3 ADDIT

NEWOBJECT("WWIPSTUFF","WWIPSTUFF.VCX")

CREATEOBJECT("wwHTTP")

Yes nHTTPConnectType = 0 (preconfig)

[ricks...@west-wind.com]

Karloz,

The components you are using are very old as you are using wwIPStuff - these components have been updated some time ago and replaced with the `wwhttp` class (which is the same class but with components broken out into separate PRG files). That class provides a cleaner interface and a number of changes in how it interacts with WinInet due to changes in the WinInet interfaces in the IE 9 timeframe. This includes different default proxy handling. 

First off I would recommend NOT using the low level components - there's no benefit to using HttpSendRequest. Instead just use `HttpGet`:

loHttp.HttpGet(url)

It provides all the features that HttpSendRequest() previously provided. 

The lower level components still work, it's just that there are a lot more things that can go wrong using them, where HttpGet() wraps the entire HTTP call process into a single operation that's properly configured based on teh settings you provide. It's possible even the older version you have already has the `wwhttp` component you're just using the older wwipstuff.vcx file instead.

More info on HTTP calls is here:

https://webconnection.west-wind.com/wwClient_docs/_s9001zxih.htm

All that said, wwHttp which is the newer class should automatically work with Fiddler - I use it that way all the time  as WinInet by default registers itself with the WinInet proxy pipeline. UNLESS you explicitly bypass the default proxy settings wwHttp traffic shows up in Fiddler without anything else configured. Again if you have a really old version, the default proxy settings were slightly different because WinInet changed default behavior in the IE 9 timeframe. The default proxy mode in recent versions is:

loHttp.nHttpConnectType = 0

which uses the default proxy settings that WinInet/IE provides (and which match your settings in the Internet Explorer Internet Settings configuration). I believe in older versions this value with 1 which is direct (no proxy config) which would bypass detection of a proxy like Fiddler.

I'm not really clear on what you're trying to accomplish. If it's simply monitor those HTTP calls you're making from FoxPro that should just work without having to configure anything other than configure Fiddler and install the Fiddler certificate to properly decode the HTTPS traffic.  Specifcally what exactly does this mean:

I do see calls and returns status in Fiddler when I create an object reference to IE and drive the browsers.

On the one hand you say you see requests and on the other you say you don't. Somehow I don't think you're telling us all we need to know here...

+++ Rick ---

[Eric Lawrence]

Thanks, Rick!

> On the one hand you say you see requests and on the other you say you don't.

I believe what Karloz is saying is that HTTP/HTTPS requests are seen when they originate from a WebBrowserControl or InternetExplorer.Application instance, but not when they originate from the WestWind controls. 

[karloz...@gmail.com]

Thanks Rick and Eric...  Sorry for the delay.   I was out of the office Thursday and Friday.

Again truly appreciate both of your time, energy and assistance in this issue.  I know both of your time is valuable. 

Rick I believe we did upgrade "recently", but have not yet installed or taken advantage of the new components.   We upgraded to look at the new sftp support

I will check to see and message you on your forum directly to make sure we have the new items and registration key etc etc.

And will give your ideas and suggestions a try.

Eric thanks again for you engagement too.

[karloz...@gmail.com]

correct Eric.   Sorry Rick if I was not clear in that regards.    But yes if I am doing calls similar to

r_oBrowser.NAVIGATE("HTTPS://" + .r_cCMDomainURL + "/cgi-bin/login.pl") after I have called up IE than Fiddler seems to track those calls.

I am hoping and I am sure that Ricks suggestions on using the new controls will go a long way in resolving this issue.

Or I am keeping my fingers crossed.