fiddler capture of azure mgmt service calls using client certificate security

865 views
Skip to first unread message

robertob

unread,
Feb 17, 2014, 2:26:56 PM2/17/14
to httpf...@googlegroups.com
if i issue an azure mgmt. service call using the required client certificate security it succeeds but when i try it with fiddler capture enabled i get a [System.Net.WebException] = {"The remote server returned an error: (403) Forbidden."}.  The code below shows how i'm manually generating call and i get the same result if i use the Microsoft.WindowsAzure.Management.Scheduler .net framework [ nfx ] abstraction.

Any insights on how i get fiddler capture to work under these circumstances as i'm wanting to use the nfx nuget package routines to issue calls and have fiddler capture inform me what the underlying rest calls were for cases where i need to construct them manually.

var subscriptionId = "9d2c200d-4b1f-4288-b0a8-182556895b58";
var cloudServiceName = "CS-NorthCentralUS-scheduler";           
var jobCollectionName = "MyJobCollection";
var requestUrl = string.Format("https://management.core.windows.net/{0}/cloudservices/{1}", subscriptionId, cloudServiceName);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(requestUrl);
request.Method = "GET";
request.Headers.Add("x-ms-version", "2013-03-01");
request.Accept = "application/xml";
request.ContentType = "application/xml; charset=utf-8";
var managementCertificate = Extensions.X509StoreFind(StoreLocation.CurrentUser, "c62c6a4c567913fdad8541952534cb783cdad4cc");
request.ClientCertificates.Add(managementCertificate);
HttpWebResponse response = null;
try { response = (HttpWebResponse)request.GetResponse(); }
catch (Exception ex) { var message = ex.Message; throw ex; }

fiddler capture enabled raw request
GET https://management.core.windows.net/9d2c200d-4b1f-4288-b0a8-182556895b58/cloudservices/CS-NorthCentralUS-scheduler HTTP/1.1
x-ms-version: 2013-03-01
Accept: application/xml
Content-Type: application/xml; charset=utf-8
Host: management.core.windows.net
Connection: Keep-Alive

fiddler capture enabled raw response
HTTP/1.1 403 Forbidden
Content-Length: 288
Content-Type: application/xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 17 Feb 2014 19:18:55 GMT
<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance"><Code>ForbiddenError</Code><Message>The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.</Message></Error>

EricLaw

unread,
Feb 17, 2014, 4:32:43 PM2/17/14
to httpf...@googlegroups.com

Excerpted from the Fiddler book:

HTTPS Client Certificates

...

One of the key design goals of Client Certificate authentication is to prevent network intermediaries (like Fiddler) from abusing the client’s credentials. Even if the client application sent its certificate to Fiddler, Fiddler cannot successfully reuse that certificate to respond to the server’s demand, because the client never provides Fiddler with its private key.

To resolve this limitation, you can supply any necessary client certificates and private keys directly for Fiddler to use when handshaking with the server. By default, if a server prompts the client for a certificate, Fiddler will look inside the %USERPROFILE%\Documents\Fiddler2\ folder for a file named ClientCertificate.cer and will use that certificate when responding to the server’s certificate demand.

In some cases, you may want to use a different client certificate for each secure connection. To do so, specify the location of the certificate using the https-Client-Certificate property on the CONNECT tunnel to the secure server. For instance, you can write code like this:

    static function OnBeforeRequest(oSession: Session)

    {

      if (oSession.HTTPMethodIs("CONNECT") {

        if (oSession.HostnameIs("exampleA")) {

          oSession["https-Client-Certificate"] = "C:\\certs\\CertA.cer";

        }

        else

        if (oSession.HostnameIs("exampleB")) {

          oSession["https-Client-Certificate"] = "C:\\test\\CertB.cer";

        }

      }

 

    //...

A .CER file does not contain the private key associated with the certificate’s public key. Instead, the .CER file merely acts as reference to Windows’ Personal certificates store (certmgr.msc). The Windows certificate store holds the private key associated with the certificate and releases it only as needed. Client certificates stored on a Smartcard will automatically appear within the Personal store when the Smartcard is inserted:

When the Smartcard is inserted, you may export a .CER file from certmgr.msc and use it just like any other client certificate. Note that your Smartcard must remain inserted for Fiddler to use its private key.

If the desired certificate isn't yet installed Windows’ Personal certificates store (e.g. you only have a .pfx file) you must first import it into the certificate store, then export a .CER file. After your certificate is installed, simply right-click the certificate and choose All Tasks > Export…. Save the .CER file to either the default ClientCertificate.cer location or the location you will specify via the https-Client-Certificate flag.

robertob

unread,
Feb 17, 2014, 7:21:03 PM2/17/14
to httpf...@googlegroups.com
thanks for pointer to specific details on how to address this

i tried placing the .cer export of my certificates - current user | personal | certificates store contained certificate with private key in %userprofile%\documents\fiddler\ClientCertificate.cer and got the same result.

i removed that and then also used fiddler | ctrl-r | ctrl-f "function OnBeforeRequest" and inserted the following block at start of function to enable applying that client certificate only when request pertained to azure mgmt. service host and got the same result.
if (oSession.HTTPMethodIs("CONNECT") /* ||  oSession.HTTPMethodIs("GET") */) {
    if (oSession.HostnameIs("management.core.windows.net")) oSession["https-Client-Certificate"] = "d:\\Users\\Shared\\Documents\\Settings\\Sbx\\myAzureSubMgmtCred050414.cer";
    //else if (oSession.HostnameIs("some.other.domain.com")) oSession["https-Client-Certificate"] = "%userprofile%\\documents\\fiddler2\ClientCertificate.cer";
}

any thoughts on what i did wrong applying the details from the fiddler book on how to get this working?

EricLaw

unread,
Feb 18, 2014, 5:33:37 PM2/18/14
to httpf...@googlegroups.com
Hrm... Are you on the current (2.4.5.9+) version of Fiddler?

If you right-click the CONNECT tunnel and choose Properties, do you see the HTTPS-Client-Certificate field as expected? Anything listed in the LOG tab?

robertob

unread,
Feb 18, 2014, 6:55:32 PM2/18/14
to httpf...@googlegroups.com
i was running 4.4.5.1 and i just upgraded and retried using 4.4.5.9 with same results.

context clicking on CONNECT tunnel session and choosing Properties shows i see the HTTPS-Client-Certificate field and nothing of interest in Log tab. 

attached is connect session properties, sessions archive and log tab contents.
connectSessionProperties.txt
log.txt
management.core.windows.net.saz

robertob

unread,
Feb 24, 2014, 6:11:09 PM2/24/14
to httpf...@googlegroups.com
i retried today using new 4.4.6.1 drop and am getting the same result.

what was possibly of interest is when i use the %userprofile%\documents\fiddler2\ClientCertificate.cer approach the http CONNECT session properties contained the following entry
    X-CLIENT-CERT: CN=Windows Azure Tools Serial#05FCC918CD5BBFAE4F8636F2FFD49D76

when i removed that %userprofile%\documents\fiddler2\ClientCertificate.cer and used ctrl-r | ctrl-f | search "onbeforerequest" to apply the following more specific client certificate behavior
  if (oSession.HostnameIs("management.core.windows.net") && oSession.HTTPMethodIs("CONNECT")) {
    oSession["https-Client-Certificate"] = "d:\\Users\\Shared\\Documents\\Settings\\Sbx\\myAzureSubMgmtCred050414.cer";
  }

the http CONNECT session properties contained this additional seemingly related entry
  HTTPS-CLIENT-CERTIFICATE: d:\Users\Shared\Documents\Settings\Sbx\myAzureSubMgmtCred050414.cer
  X-CLIENT-CERT: CN=Windows Azure Tools Serial#05FCC918CD5BBFAE4F8636F2FFD49D76

Is that expected?  do these http CONNECTION session property details, followed by the GET request generating a 403 forbidden suggesting that client certificate was involved but backend system didn't like it.  all i have to do to make code work in this case is simply f12 disable fiddler capture and i had a second pair of eyes confirm that the CONNECTION session properties displayed X-CLIENT-CERT had the CN and Serial# of the cert we are expecting.

robertob

unread,
Feb 27, 2014, 6:44:45 PM2/27/14
to httpf...@googlegroups.com
status update

A member of the azure scheduler mgmt. service test team used a clean vm install and followed the steps to enable fiddler capture of client certificate secured connections and on thier setup things worked fine.  This suggests that issue i'm hitting is a machine specific matter so when i have the time to do so i'll test a fiddler uninstall/reinstall to see if that resolves it and if not i'll test if it repro's for me on a 2ndary dev wks.

Thanks for the responses above with details on how to configure this setup, what to expect and how to diagnose connection has certificate being attached.

Jack Ma

unread,
Jul 11, 2014, 8:58:25 PM7/11/14
to httpf...@googlegroups.com
Facing exactly the same trouble, Robert, got any solutions for this one? Eric, any clue why this happens? Thanks, -jackma

EricLaw

unread,
Jul 13, 2014, 8:29:28 AM7/13/14
to httpf...@googlegroups.com
Jack, rather than saying you're facing "exactly the same trouble" can you please explain exactly the problem you're having, your Windows and Fiddler versions, and any other information that might be relevant?

Jack Ma

unread,
Jul 14, 2014, 5:27:17 PM7/14/14
to httpf...@googlegroups.com
Sure Eric. attached pls find the saved sessions about the issue that I was talking about.

basically, i'm facing that when using ctrl+r customruls.js approach, and the following logic, the subsequent requests after a successful CONNECT session will always be rejected by server returning 403 and client certificate missing error messages. I have noticed in the properties of the CONNECT session, things look good. I mean the cert I specified was picked up by Fiddler and no particular logs are presented in the Log tab.

The fiddler version I'm with is 4.4.9.0, the latest one.

Thanks,
-jackma

static function OnBeforeRequest(oSession: Session) {
  
        if (oSession.HTTPMethodIs("CONNECT")) {
            var securedZenithUrl = oSession.uriContains("command") || oSession.uriContains("querypre");
            if (!securedZenithUrl) {
                oSession["x-no-decrypt"] = "do not care.";
            } else {
                oSession["https-Client-Certificate"] = "C:\\Users\\jackma\\Desktop\\zenith_migration_work\\svc_client.cer";
                ////FiddlerObject.log(oSession["https-Client-Certificate"]);
            }
        }

...
https_client_cert_failure.saz

Devin Lusby

unread,
Sep 20, 2014, 3:09:32 PM9/20/14
to httpf...@googlegroups.com
Hello Gents,

I'm also attempting to pass Client Certificates to Azure (and Windows Azure Pack, incidentally).  I've attempted both the FiddlerScript approach and the default-path-to-cer-file approaches to add the certificate to the messages and it continually fails.  Same symptom, nothing new to add here: Works without fiddler, fails with it.  X-Client-Cert: is on the right requests and so is HTTPS-Client-Certificate, but no mojo.

You gents had any success/luck with this?

EricLaw

unread,
Sep 21, 2014, 1:32:04 PM9/21/14
to httpf...@googlegroups.com
Fails how, specifically?

What FiddlerScript did you use, specifically?

Do you have a SAZ or NetMon Capture I can look at?

robertob

unread,
Sep 21, 2014, 2:14:58 PM9/21/14
to httpf...@googlegroups.com
i heard back from Jack Ma a month ago that he got this working, in his words . . . "Just for your information, I discovered a workaround of this issue by simply re-importing (not even need to re-export) the client cert to the cert store. Everything just works again."
Reply all
Reply to author
Forward
0 new messages