When use fiddlercore with selenium the performance is slow

[Ted Yang]

I'm using the fiddlerCore in Selenium web automation test to capture the web traffic. It works, but the performance is bad, nearly double the test execute time. And sometimes it will cause timeout of the test cases.

here is the article which I reference: http://jimevansmusic.blogspot.jp/2013/08/implementing-webdriver-http-status.html

I did some filter in the event handler method:

private static void FiddlerApplication_AfterSessionComplete(Session sess)
        {
            _hitCounter++;
 
            // Ignore HTTPS connect requests
            if (sess.RequestMethod == "CONNECT" || sess.RequestMethod == "OPTIONS")
                return;
 
 
            if (!string.IsNullOrEmpty(CaptureConfiguration.CaptureDomain))
            {
                if (sess.hostname.ToLower() != CaptureConfiguration.CaptureDomain.Trim().ToLower())
                    return;
            }
 
            if (CaptureConfiguration.IgnoreResources)
            {
                var url = sess.fullUrl.ToLower();
 
                List<String> extensions = CaptureConfiguration.ExtensionFilterExclusions;
                foreach (var ext in extensions)
                {
                    if (url.Contains(ext))
                    {
                        //ignore the specified extension
                        return;
                    }
                }

	     }

             ...

            }

does anyone meet the same problem, or there is a way to make the performance better?

[EricLaw]

1. You haven't explained what you mean by "slow." Are we talking about things taking "1 second longer" or "1 minute longer"?
2. You haven't said what version of FiddlerCore you're using.
3. You haven't said whether you see a performance issue when Fiddler itself is running.
4. You haven't shown the rest of your code.
5. You haven't said whether you're decrypting HTTPS traffic, and if so, which certificate generator you're using.

 

[Ted Yang]

Hi Eric,

Thanks for your reply, sorry that I didn't provide enough information, here is more:

1. It almost 80% slow than without fiddler enabled, the data is collected in the same environment and with 5 runs of our automation test cases.

2. We use Visual Stuido 2013, Selenium C# with FiddlerCore, we got the package from the NuGet Server, here is the version info:

  <package id="FiddlerCore" version="4.4.8.4" targetFramework="net45" xmlns="" />
  <package id="Selenium.Support" version="2.43.1" targetFramework="net45" xmlns="" />
  <package id="Selenium.WebDriver" version="2.43.1" targetFramework="net45" xmlns="" />
  <package id="Selenium.WebDriver.ChromeDriver" version="2.10.0.0" targetFramework="net45" xmlns="" />
  <package id="Selenium.WebDriver.IEDriver" version="2.43.0.0" targetFramework="net45" xmlns="" />

3. As we observed that there is no performance issue when Fiddler itself is running, we use Fiddler a lot in our work, and our product is an one page web application. We thought that it is because Fiddler running in a different process.

4. The main code for fiddlerHelper is attached. And here is the code to make Fiddler work with Chrome Driver:

            ChromeOptions options = new ChromeOptions();
            options.AddArgument("--start-maximized");
            options.AddArgument("--test-type");
 
            if (Properties.Settings.Default.EnableFiddlerTrace)
            {
                // Note that we're using a desired port of 0, which tells
                // Fiddler to select a random available port to listen on.
                var proxyPort = FiddlerHelper.StartFiddlerProxy(0);
 
                OpenQA.Selenium.Proxy proxy = new OpenQA.Selenium.Proxy();
                proxy.HttpProxy = String.Format("127.0.0.1:{0}", proxyPort);
                proxy.SslProxy = String.Format("127.0.0.1:{0}", proxyPort);
                options.Proxy = proxy;
            }
 
            if (null == this.driver)
            {
                this.driver = new ChromeDriver(browserDriverFolder, options);
            }

5. We want to decrpt HTTPS traffic, current I just install Fiddler, and install the cert use the option inside Fiddler, I believe it should be "makecert.exex" way. I also look for ways to automatically the cert install. There is code to do so, but one dialogue will popup.

Hope you could provide us more insights.

[EricLaw]

Ah, this may be an issue addressed by a later update to FiddlerCore.

In your FiddlerCore application's startup code, add the following:

                // Ensure proxy Thread Pool is ready for a traffic burst
                // The ThreadPool's default growth algorithm is too slow
                int iProcCount = Environment.ProcessorCount;
                int iMinWorkerThreads = Math.Max(16, 6 * iProcCount);
                int iMinIOThreads = iProcCount;
                if ((iMinWorkerThreads > 0) && (iMinIOThreads > 0))
                {
                    System.Threading.ThreadPool.SetMinThreads(iMinWorkerThreads, iMinIOThreads);
                }

This code should only be used with FiddlerCore v2or4.4.8.4; it is not needed for the next release of FiddlerCore.

[Ted Yang]

Hi Eric,

Much appreciate for your help, after adding your magic code in our framework, the performance improved a lot in my local test. I will run the whole tests in our environment and share more data if everything goes well.