Re: Receive WebSockets messages in Extension

[EricLaw]

It sounds like you're reporting two different problems:

 

1> You are finding that connections on "a proper broadband router" don't work?

2> You want to know how to see WebSocket messages in an Extension?

 

Is that correct?

 

For #1, does the traffic through the router work properly when Fiddler isn't running?

 

For #2, you'll need the upcoming v2.4.4 build of Fiddler, in which a FiddlerApplication.OnWebSocketMessage event has been introduced. In prior versions of Fiddler, only FiddlerScript could capture WS traffic (using a method of this name in CustomRules.js).

 

thanks,

Eric

[Nigel Dando]

 

#1 The application which I am monitoring works fine in all cases, (Dongle or Router, with or without Fiddler).

     It just appears to utilise WebSockets when proper broadband used.

 

#2 The upcoming change sounds just what I am looking for.

     When available I will add logic to monitor the "OnWebSocketMessage" in addition to the existing "AutoTamperResponseBefore",

  Then it should work in either case.

 

Thank You for your help

Nigel

 

 

 

[EricLaw]

Hi, Nigel-- You can install the v2.4.4 beta with this new event from here: http://getfiddler.com/dl/Fiddler2BetaSetup.exe

[Nigel Dando]

Thank you for your prompt reply.

 

I have installed the Beta version ok using the link.

When I try to run fiddler, my .dll will not load,

with message "This assembly is built by a runtime newer than the currently loaded runtime"

 

At bottom of message:- Fiddler v2.4.4.0 (x64 AMD64) [.NET 2.0.50727.5466 on Microsoft Windows NT 6.1.7601 Service Pack 1]

 

I am fairly new to this, could you point me to what I should check or change.

 

I have always built project with Target .Net 4.5 (tried  4, project will not build in 2)

 

Cheers

Nigel

 

 

 

[Nigel Dando]

Sorry I think my last post was a bit stupid..

I guess it is called v2.4.4.0 because it only runs with .NET 2.0

Assuming so, I have created a test cut down extension which does build and execute as target .NET 2

 

When I have previously received session response data ok, my extension contained:-

 

public void AutoTamperResponseBefore(Session oSession)

 {           

 String oBodyString = System.Text.Encoding.UTF8.GetString(oSession.responseBodyBytes);

  // process oBodyString here

  }

 

To try and receive the WebSocket message data I added this to the extension code:-

       

public void OnWebSocketMessage(WebSocketMessage oMsg)        

{

 String myString = oMsg.PayloadAsString();

 // process myString here

 }

This is obviously wrong as I can see the socket data scrolling up Fiddlers Log but my code never gets called.

 

Have I completely misunderstood the concept?

 

Any guidance would be much appreciated

 

Regards

Nigel  

 

 

 

 

 

 

 

 

[EricLaw]

You can install the v4 version of Fiddler (which works with v4-targeted extensions) from here: http://getfiddler.com/dl/Fiddler2BetaSetup.exe

Unlike the AutoTamper* methods, notification of WebSocket messages is performed as an event rather than an IAutoTamper interface method. Therefore, your extension needs to explicitly subscribe to the event. In the OnLoad method for your extension, add the following line:

    FiddlerApplication.OnWebSocketMessage += HandleWebSocketMessage;

Then implement the handler thusly:

    void HandleWebSocketMessage(object sender, WebSocketMessageEventArgs e)
    {
       // Your code here.
    }

[Nigel Dando]

Thank You Eric,

 

I followed your suggestions and it works wonderfully.

 

If connected to landline broadband the traffic is transported via WebSockets and received into my extension by event handler.

 

Then if I have to drop back to mobile broadband dongle, it switches to the old protocol and received into my extension using the AutoTamperResponse method.

 

Great Solution

Best Regards

 

Nigel

 

 

 

[EricLaw]

Thanks for confirming that it worked!

 

Your finding about the network connection is super-interesting. In the Mobile Broadband case, do you by any chance see any "CONNECT" tunnels that have a non-200 response code? Sites don't have any easy way to test whether the user is on a mobile network connection or not, so it would be hard to build a site that explicitly checked for this. However, some networks (particularly mobile networks) are configured to use transparent/interception proxies, and those proxies more often fail to properly establish WebSocket connections. So it's possible that the site's script detects the failing websocket connection and falls back to a non-WebSocket protocol.

 

(Another site to try this on would be http://websocketstest.com. That site uses both regular WebSockets and WebSockets-over-HTTPS; the latter are more likely to work through transparent proxies).

[Nigel Dando]

Hi Eric,

 

I have briefly viewed the session history for the mobile environment and have not spotted any Tunnel entries that do not have a result code of 200.

 

As I am not expert on these things I could do the following:-

I believe that Fiddler can save session history to a file, so I could run a short period  in each environments and email you the files.

And offline give you a bit of information about the application.

 

If that's ok I could do this after our UK bank holiday next week.

 

By the way.. Running today the first time using events, after an hour I did manage to freeze fiddler,

the web application continued to run, but fiddler stopped talking to me and the extension, I ended up terminating the fiddler task.

But that may be my lack of event handling knowledge so I will look closer at this and if it happens again gather more info.

The application I am monitoring is running in about 60 separate Windows each receiving traffic, this did not cause a problem

with AutoTamper and I don't know enough about events yet so I must read up on this to get a better understanding.

 

Regards

Nigel

 

 

 

 

[EricLaw]

Nigel-- I'm happy to review your code. Multi-threaded event handling is indeed quite difficult and it's easy to cause a deadlock if you're not careful.

 

If you send me a SAZ file (File > Save Session Archive) I'm happy to take a look at the behavior but it's not terribly important, I was just idly curious about what was causing the behavioral difference.

[Nigel Dando]

Hi

When I process the websocket events I am still causing the fiddler GUI to eventually stop responding.

 

My thought was that maybe I should perform my processing in a separate thread, passing it the event data in a queue so as to free up the fiddler thread as fast as possible.

 

Before doing  this I cut down my event handling method (in the extension) to a minimum, not calling my processing of the data,

So just counting the events and extracting the string of data from the event

 

void HandleWebSocketMessage(object sender, WebSocketMessageEventArgs e)

        {

 Variables.countSockets = Variables.countSockets + 1;

 String oBodyString = e.oWSM.ToString();

 // Response.ProcessResponse(oBodyString);

        }

 

This did not solve the problem.

Am I missing something?

 

PS.

If I perform the same processing thru AutoTamper it happily processes traffic all day.

 

        public void AutoTamperResponseBefore(Session oSession)
        {
                Variables.countHTTP = Variables.countHTTP + 1;
                String oBodyString = System.Text.Encoding.UTF8.GetString(oSession.responseBodyBytes);
                Response.ProcessResponse(oBodyString);
        }

 

Cheers

Nigel

 

[EricLaw]

WebSocket processing already occurs on a background thread (one thread per WebSocket) so that's not what's causing the problem here. What other code do you have in your project?

 

Note that this call:

 

    Variables.countSockets = Variables.countSockets + 1;

 

...is not thread-safe. If you want to update a shared variable like this, you either need to use Interlocked.Increment or an appropriate synchronization primitive (e.g. mutex).

[Nigel Dando]

Hi Eric,

 

I have created the smallest extension possible, with none of my code other than the extraction of the string from the event.

This runs fine for a while then starts pausing and eventually locks the fiddler GUI.

 

I have never found a problem with fiddler, and as I am opening 60 application windows each with its own websocket,  I think it fair to assume  I am pushing my quad core processor to its limits.

 

So for now I will revert back to using mobile dongle (forcing HTTP) which works wonderfully.

 

Fiddler is a great product and in a short time allowed me to build a  solution, even though I had never looked at a C program 2 months ago, or new anything about coms.

 

Fiddler allows me to monitors the behaviour of many "Stock Market Shares" one displayed in each minimised  window.

My extension then  applies my algorithm to calculates an indicator of the expected direction of the overall market Index.

I then have a  client program that calls once a second (using WCF) a thread in the fiddler extension to get a snap shot. 

It displays market activity and finally the point of all this;-  it  draws in a real time graph the indicator resulting from my algorithm.

 

This is for my use to prove the effectiveness of the algorithm in real time not as a product.

 

Hope I haven't bored you,

Thank you for all your help.

 

Nigel

 

 

 

 

 

 

 

 

[EricLaw]

Fascinating. At the point of the failure, can you use a tool like SysInternals' Process Monitor to look at the CPU usage and thread count of the Fiddler process? I'm curious to hear what they are.  Thanks!

[Nigel Dando]

Ok

 

I have searched for "Process Monitor" and found a download on what looks like a Microsoft site.

I'll download this and have a play tomorrow.

 

[Nigel Dando]

Good Evening Eric,

 

I downloaded and ran the Process Monitor but the Help tutorial not working, and as I am not very technical not sure how to get at what we need. But never mind I'll leave that for now.

 

So I reinstalled and ran my full extension using WebSockets with the following observations:- 

 

- Ran Ok for 15 mins with the fiddler GUI displaying the scrolling log of socket messages.

- After 15 mins the GUI became unresponsive but the Log continued to scroll socket messages.

- The extension processing continued and remained up to date, as my client retrieving data via a  thread in the extension reflected accurately the live traffic. 

- After a few more moments processing ok, I closed the application being monitored.

- My client immediately showed the traffic had stopped.

- The fiddler log continued to scroll socket messages for at least a further 3 mins.

- When the log stopped (after 3 mins), showing the final socket messages the Fiddler GUI then became Responsive again OK.

 

Tomorrow I will let the app run longer and see if at some point it begins to impact on the overall  performance.

  

At this stage of my development I don't really need to see the log so is there any way I can switch it off?

 

Best Regards

Nigel

 

 

 

 

[EricLaw]

Ah! The Log wasn't supposed to be showing WebSocket traffic anymore; I fixed that bug in 2.4.4.2.

 

Please upgrade to the latest bits (2.4.4.3 Beta at http://fiddler2.com/get-fiddler) and let me know if that resolves the issue.

 

thanks!

[Nigel Dando]

Thank You,

 

I needed .NET 4. So downloaded Fiddler (v4.4.4.3 beta)

 

Looks good, no socket messages in Log,

I'll test it with lots of traffic today.

 

and the sun is shining today in the south west of England what more could we want.

 

Cheers

Nigel