Making Our Selenium Tests Worth More...Automate HAR Generation

[Stephen Feldman]

I've got this huge investment I've made in Selenium over the years with my product. We have hundreds and thousands of Selenium scripts that we execute every day as part of our continuous integration process. Well, we don't execute all of them, but we could if we wanted to. Our Selenium tests run cross-browser. They serve their purpose of both functional verification from the browser, as well as browser behavior verification. 

I want to make my investment in Selenium go further. Basically, I want to generate HAR files as part of every Selenium test. I want to then take the HAR and run automated analysis looking for regression issues, browser comparison problems, violations of other web performance principals, etc...

I am trying to do this with little disruption to my CI infrastructure. My ideal scenario would be to prototype a small code change to Selenium. I'm really not seeing that as an option. I guess Selenium does not have a way to intercept the HTTP request. Rather, I have to use a proxy like BrowserMob. 

Has anyone automated the generation of a HAR in a similar manner? I don't want this to be a manual process, nor do I want to limit this to just Firefox/Firebug. I need to run IE, FF and Chrome.

Any comments would be appreciated...

Regards,

Steve

[Emmanuel Cecchet]

Hi,

I did a BrowserMob proxy integration with Selenium for desktop browsers
and Android for the BenchLab project (http://benchlab.cs.umass.edu/).
Follow the link to the SourceForge page to get the source code.
SSL is tricky to get working but otherwise setting up the proxy works
decently. I chose to run the proxy in my app and the code looks like this:
DesiredCapabilities capabilities = null;
if (bcrProp.getBrowsermobProxy())
{
// start the browsermob-proxy
proxyServer = new ProxyServer(bcrProp.getBrowsermobProxyPort());
proxyServer.start();

// get the Selenium proxy object
Proxy proxy = proxyServer.seleniumProxy();

// configure it as a desired capability
capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
}
Use the capabilities to create your WebDriver instance.

Note that for IE, that will change your Internet connection settings to
always go through the proxy, you will need to revert that at the end of
your test if you don't want to lose your Internet connection when the
proxy goes down.

Hope this helps
Emmanuel

> --
> --
> You received this message because you are subscribed to the Google
> Groups "HTTP Archive Specification" group.
> To post to this group, send email to
> http-archive-...@googlegroups.com
> To unsubscribe from this group, send email to
> http-archive-specif...@googlegroups.com
> For more options, visit this group at
> https://groups.google.com/forum/#!forum/http-archive-specification
> <https://groups.google.com/forum/#%21forum/http-archive-specification>
>
> ---
> You received this message because you are subscribed to the Google
> Groups "HTTP Archive Specification" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to http-archive-specif...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Stephen Feldman]

Thank you so much for the prompt reply. I took a look at your project. So as you said above, you really had to leverage BrowserMob to obtain your HAR for more accurate timings. Did you experience any kind of latency inflicted by the proxy? Did you have any other problems leveraging BrowserMob?

I am very curious about your project. What were the motivations behind your project?

Regards,

Steve

[Emmanuel Cecchet]

Hi Steve,

I would recommend you to look at these 2 research papers to better understand our motivations:

• BenchLab: An Open Testbed for Realistic Benchmarking of Web Applications

  Emmanuel Cecchet, Veena Udayabhanu, Timothy Wood and Prashant Shenoy.
  Proceedings of 2nd USENIX Conference on Web Application Development (WebApps '11), June 15-16, 2011, Portland, OR
  pdf

• mBenchLab: Measuring QoE of Web Applications using mobile devices (Best Paper Award)

  Emmanuel Cecchet, Robert Sims, Xin He and Prashant Shenoy
  Proceedings of ACM/IEE International Symposium on Quality of Service (IWQoS 2013), Montreal, Canada, June 3-4 2013
  Paper (pdf) Slides (.pptx)

If the email does not preserve the links, you can find the publication on our lab page at http://lass.cs.umass.edu/projects/benchlab/

Yes, there are some issues with BrowserMob especially on Android. The URL parsing and caching is bogus and will lead to NPEs and other fun runtime exceptions. There is a measurable overhead on the initial connection establishment time. It's not as bad on the desktop but still your first page access will see very variable latency depending on how much time it takes to initiate the whole thing and depending on the browser (usually they open 6 connections in parallel).
Using SSL requires installing specific certificates in the browser so that the browser does SSL with BrowserMob and BM does SSL with the targeted web site.
On complex Web pages, you will need memory and storage (especially if you run BM in your app) and that can be a limiting factor on mobile devices. The threading model also needs to be reworked on Android.
We originally tried to use Firebug for Firefox and try to control the developer tools in Chrome but it ended up being too much work to keep up with the various versions of the browsers. So we decided to go the universal route with BrowserMob and we are pretty satisfied with the results. The overhead of the proxy does not seem to significantly affect latency on most networks (unless you are a very limited mobile device, check our mBenchLab paper to see our overhead measurements). The HAR generated by BM is usually correct, you can have some issues when you start to encounter network errors.

That's all I can think of for now on top of my head. I'll let you know if I can think of anything else.

Best,
Emmanuel

--

--
You received this message because you are subscribed to the Google
Groups "HTTP Archive Specification" group.
To post to this group, send email to
http-archive-...@googlegroups.com
To unsubscribe from this group, send email to
http-archive-specif...@googlegroups.com
For more options, visit this group at
https://groups.google.com/forum/#!forum/http-archive-specification
 

[Tim Hawkins]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

We had a simular issue, however browsermob does not allow you to get the domloaded and page ready events, as only the browser rendering the page knows that info. That information is key to most performance optimisation strategies.

>--
>--
>You received this message because you are subscribed to the Google
>Groups "HTTP Archive Specification" group.
>To post to this group, send email to
>http-archive-...@googlegroups.com
>To unsubscribe from this group, send email to
>http-archive-specif...@googlegroups.com
>For more options, visit this group at
>https://groups.google.com/forum/#!forum/http-archive-specification
>
>---
>You received this message because you are subscribed to the Google
>Groups "HTTP Archive Specification" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to http-archive-specif...@googlegroups.com.
>For more options, visit https://groups.google.com/groups/opt_out.

- --
Sent from my Android phone with K-@ Mail. PGP public key available.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.0.8

iNIEAQEIADwFAlHp6xw1HFRpbSBIYXdraW5zIChtYWluIGlkZW50aXR5KSA8dGlt
LnRoYXdraW5zQGdtYWlsLmNvbT4ACgkQV4SVfWuVeB9g0wQA0rPJphCBKN10hUNC
6+lYbPp2PObCysWyATChJQRAZEidGSzGaeRAKPBdzXeektSj01mRtxq7QQQekjC0
Y0rvgE8VLXRf+ChmVxEep7bURom+SSnGmWZ5IVFBRD1UbBMfE3EfGNQXwyReLBN1
jUxwC7iWvOPnSfQsKsZ/VOML+jo=
=rjSg
-----END PGP SIGNATURE-----

[Andrew]

If you can take a packet capture while the test is running, you can
use my tool, pcap2har, to generate a HAR from that. It's at
github.com/andrewf/pcap2har. It won't get the in-browser events, but
you can get pretty accurate timing info about network events. It also
can't parse SSL streams. That turns out to be pretty hard. :P If you
feel like hacking it, you can get access to the raw re-assembled TCP
streams and do your own stuff on that too.

Feel free to post to the support list
(https://groups.google.com/forum/#!forum/pcap2har) if you decide to
use it and have any trouble.

[Steve Feldman]

Thanks for the reply Tim. So what did you end up doing to obtain that information? Was there another approach you took?

Sent from my iPhone

[Stephen Feldman]

I will definitely take a look Emmanuel...

Regards,

Steve

[Stephen Feldman]

Tim,

Which path did you end up taking? Did you go the route of WebPageTest, PhantomJS or some other solution?

Regards,

Steve

[Markus Kohler]

Hi Steve,

Yes. I did this use different approaches. 

1. HTTPWatch, can be controlled using COM automation, using ruby from within a ruby Selenium test.  Unfortunately only available for Firefox and IE with different interfaces. The advantage is a simpler setup, 

2. Various proxies (apache, home grown): Works for all browsers and is probably the only feasible way to go at the moment if you need cross browser support.  

an integrated solution that would be easy to setup Is certainly something I would like to see as an open source project :-)

Regards,
Markus

--

[Vijay Kumar]

Hi,

I have done the setup with the proxy. I am able to generate the har file, but missing the https related info and calls. Can you highlight the workaround for ssl?

> http-archive-specification+unsub...@googlegroups.com

> For more options, visit this group at
> https://groups.google.com/forum/#!forum/http-archive-specification
> <https://groups.google.com/forum/#%21forum/http-archive-specification>
>  
> ---
> You received this message because you are subscribed to the Google
> Groups "HTTP Archive Specification" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to http-archive-specification+unsub...@googlegroups.com.

[Paresh Kalinani]

Use PhantomJS with BrowserMobProxy. PhantomJS helps us for JavaScript enables pages. The following code works for HTTPS web addresses, too.

Place 'phantomjs.exe' in C drive and you get the 'HAR-Information.har' file in C drive itself.

   

 package makemyhar;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.util.ArrayList;
    import net.lightbody.bmp.BrowserMobProxy;
    import net.lightbody.bmp.BrowserMobProxyServer;
    import net.lightbody.bmp.core.har.Har;
    import net.lightbody.bmp.proxy.CaptureType;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.phantomjs.PhantomJSDriver;
    import org.openqa.selenium.phantomjs.PhantomJSDriverService;
    import org.openqa.selenium.remote.CapabilityType;
    import org.openqa.selenium.remote.DesiredCapabilities;
   
    public class MakeMyHAR {
        public static void main(String[] args) throws IOException, InterruptedException {
           
            //BrowserMobProxy
            BrowserMobProxy server = new BrowserMobProxyServer();
            server.start(0);
            server.setHarCaptureTypes(CaptureType.getAllContentCaptureTypes());
            server.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);
            server.newHar("Google");
           
            //PHANTOMJS_CLI_ARGS
            ArrayList<String> cliArgsCap = new ArrayList<>();
            cliArgsCap.add("--proxy=localhost:"+server.getPort());
            cliArgsCap.add("--ignore-ssl-errors=yes");
           
            //DesiredCapabilities
            DesiredCapabilities capabilities = new DesiredCapabilities();
            capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
            capabilities.setCapability(CapabilityType.SUPPORTS_JAVASCRIPT, true);
            capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, cliArgsCap);
            capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY,"C:\\phantomjs.exe");
       
            //WebDriver
            WebDriver driver = new PhantomJSDriver(capabilities);
            driver.get("https://www.google.co.in/");
           
            //HAR
            Har har = server.getHar();
            FileOutputStream fos = new FileOutputStream("C:\\HAR-Information.har");
            har.writeTo(fos);
            server.stop();
        }
    }

[Jeff White]

I am late to this party but here is something I would like to contribute:

package com.browserproxy;


//CHECKSTYLE:OFF checkstyle:magicnumber

import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.core.har.Har;
import net.lightbody.bmp.proxy.CaptureType;
import org.junit.Assert;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.CapabilityType;
import java.io.File;
import java.io.IOException;
import java.net.*;
import java.util.Collection;
import java.util.Iterator;
import java.util.List;

/**
 *   Browser Mob based class
 *
 *
 */

@Slf4j
public class BProxy {

    @Getter
    private static BrowserMobProxy proxy = null;
    private int proxyPort = 9876;

    /***********************************************************
     * public BrowserMobProxy getProxyServer()
     * Sets up and starts the Browser Mob Proxy
     *
     *
     * @return Proxy instance
     ***********************************************************/
    public void getProxyServer() throws UnknownHostException {
        getProxyServer(CaptureType.REQUEST_CONTENT,
                CaptureType.REQUEST_HEADERS,
                CaptureType.RESPONSE_CONTENT,
                CaptureType.RESPONSE_HEADERS);
    }

    /**
     *
     * @param captureTypes Comma Separated list
     * @throws UnknownHostException
     *
     *  - Sets up and starts the Browser Mob Proxy
     *      * - Allows the ability to set capture types for the HAR capture
     *      *
     *      * EX:
     *      * getProxyServer(CaptureType.REQUEST_CONTENT,
     *      *                 CaptureType.REQUEST_HEADERS,
     *      *                 CaptureType.RESPONSE_CONTENT,
     *      *                 CaptureType.RESPONSE_HEADERS);
     *
     */
    public void getProxyServer(CaptureType... captureTypes) throws UnknownHostException {
        getProxyServer(proxyPort, captureTypes);
    }

    /**
     * public void getProxyServer(int port, CaptureType... captureTypes)
     * @param port  - Port to start Proxy server
     * @param captureTypes - Comma separated list of capture Types
     * @throws UnknownHostException
     *
     * - Ability to set the port
     * - Sets up and starts the Browser Mob Proxy
     * - Allows the ability to set capture types for the HAR capture
     *
     */

    public void getProxyServer(int port, CaptureType... captureTypes) throws UnknownHostException {
        if(proxy == null) {
            proxy = new BrowserMobProxyServer();
            proxy.setTrustAllServers(true);
            InetAddress connectableAddress = ClientUtil.getConnectableAddress(); // Default behaviour
            log.info("Connectable Address=" + connectableAddress);
            // above line is needed for application with invalid certificates
            log.info("Local Host Address=" + InetAddress.getLocalHost());
            proxy.start(port, InetAddress.getByName("localhost"));
            log.info("Local Host Address=" + proxy.getClientBindAddress());
            log.info("Starting Browser Mob Proxy");
            proxy.setHarCaptureTypes(CaptureType.getAllContentCaptureTypes());
            proxy.enableHarCaptureTypes(captureTypes);
        }
    }

    /***********************************************************
     * private InetSocketAddress getNetProxy()
     * @return address of proxy
     *
     * Returns the address of the network proxy being used
     ***********************************************************/
    protected InetSocketAddress getNetProxy() {
        System.setProperty("java.net.useSystemProxies", "true");
        List<java.net.Proxy> l = null;
        try {
            l = ProxySelector.getDefault().select(
                    new URI("http://www.google.com/"));
        } catch (URISyntaxException e1) {
            e1.printStackTrace();
        }

        InetSocketAddress addr = null;
        for (Iterator<java.net.Proxy> iter = l.iterator(); iter.hasNext(); ) {
            java.net.Proxy myProxy = iter.next();
            System.out.println("proxy hostname : " + myProxy.type());
            addr = (InetSocketAddress) myProxy.address();

            if (addr == null) {
                System.out.println("No Proxy");
            } else {
                System.out.println("proxy hostname : " + addr.getHostName());
                System.out.println("proxy port : " + addr.getPort());
            }
        }

        return addr;
    }

    /***********************************************************
     * public Proxy setSeleniumProxy(BrowserMobProxy proxyServer)
     * @param proxyServer
     * @return seleniumProxy
     *
     * Sets up the Selenium Proxy
     ***********************************************************/
    public Proxy getSeleniumProxy(BrowserMobProxy proxyServer) {
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxyServer);
        try {
            String hostIp = "localhost";
            seleniumProxy.setHttpProxy(hostIp + ":" + proxyServer.getPort());
            seleniumProxy.setSslProxy(hostIp + ":" + proxyServer.getPort());
        } catch (Exception e) {
            e.printStackTrace();
            Assert.fail("invalid Host Address");
        }
        return seleniumProxy;
    }

    /***********************************************************
     * private ChromeOptions getChromeOptions()
     * Sets the Chrome Options for Browser Mob Proxy
     * @return options
     *
     * This is for any apps using this outside of
     * the selenium framework.
     ***********************************************************/
    protected ChromeOptions getChromeOptions() {
        ChromeOptions options = new ChromeOptions();
        options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
        options.setCapability(CapabilityType.SUPPORTS_JAVASCRIPT, true);
        options.setCapability(CapabilityType.PROXY, getSeleniumProxy(proxy));

        return options;
    }

    /***********************************************************
     *  public void quit()
     *  Gives the ability to close (quit) the Browser Mob Proxy
     ***********************************************************/
    public void quit() {
        proxy.stop();
    }

    /***********************************************************
     * private void writeHar(Har har, String filePath)
     * @param har
     * @param filePath
     * Handles writing the Har file based upon path
     *
     ***********************************************************/
    protected void writeHar(Har har, String filePath) {
        try {
            proxy.getHar().writeTo(new File(filePath));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    /**
     * protected void writeHar()
     */
    protected void writeHar(){
        writeHar(getHar(), "target/test.har");
    }

    /**
     * public void blacklistRequests(String blackListCollection, int RespCode)
     * @param blackListCollection
     * @param RespCode
     *
     * Allows user to add items to blacklist from results
     * Ex:  blacklistRequests("https?://.*\\.google-analytics\\.com/.*", 410);
     * 410 is essentially "Gone"
     */
    public void blacklistRequests(String blackListCollection, int RespCode){
        proxy.blacklistRequests(blackListCollection, RespCode);
    }

    /**
     * public void whitelistRequests(Collection<String> collection, int StatusCode)
     * @param collection
     * @param StatusCode
     *
     * Ex: whitelistRequests("https?://*.*.yoursite.com/.*. https://*.*.someOtherYourSite.*".split(","), 200);
     *
     * Allows user to add URL patterns to allow
     */
    public void whitelistRequests(Collection<String> collection, int StatusCode){
        proxy.whitelistRequests(collection, StatusCode);
    }

    /***********************************************************
     * public void newHar(String name)
     * @param name
     *
     * Create a new HAR with the label specified by the parameter 'name'
     ***********************************************************/
    public void newHar(String name) {
        proxy.newHar(name);
    }

    /***********************************************************
     * public void newHar()
     *
     * Create new Har
     ***********************************************************/
    public void newHar() {
        proxy.newHar();
    }

    /***********************************************************
     * public void newPage()
     *
     * Creates a new page entry
     ***********************************************************/
    public void newPage() {
        proxy.newPage();
    }

    /***********************************************************
     * public void newPage(String pageName)
     * @param pageName
     * Designates a new page in the HAR
     ***********************************************************/
    public void newPage(String pageName) {
        proxy.newPage(pageName);
    }

    /***********************************************************
     * public Har getHar()
     * @return
     *
     * Collect the performance data from the BrowserMob proxy server.
     * Get the HAR data.
     ***********************************************************/
    public Har getHar() {
        return  proxy.getHar();
    }

    /***********************************************************
     *  public void endHar()
     *
     *  Ends the HAR file
     ***********************************************************/
    public void endHar() {
        proxy.endHar();
    }




}

############################################################################

package com.browserproxy;



import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Service;

@Service
public class BProxyConfig {

    @Bean
    public BProxy bProxy() {
        return new BProxy();
    }
}

############################################################################

[Jeff White]

The usage of those two classes:

//CHECKSTYLE:OFF checkstyle:magicnumber

import io.github.bonigarcia.wdm.ChromeDriverManager;
import lombok.extern.slf4j.Slf4j;
import net.lightbody.bmp.proxy.CaptureType;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import java.net.InetSocketAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.List;



@Slf4j
@RunWith(SpringRunner.class)
@SpringBootTest
public class BProxyIT {

    @Autowired
    BProxyConfig ProxyConfig;

    /***********************************************************
     * public void bproxyTest()
     *
     * Test of BrowserMob Proxy
     **********************************************************/



    @Test
    public void bproxyTest() {
        BProxy bProxy = ProxyConfig.bProxy();

       ChromeDriverManager.chromedriver().setup();
        try {
            bProxy.getProxyServer(CaptureType.REQUEST_HEADERS, CaptureType.RESPONSE_HEADERS);
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }

        // Check for Proxy (Currently turned off on the NextGear network
        InetSocketAddress addr = bProxy.getNetProxy();

        if (addr != null) {
            bProxy.getProxy().setChainedProxy(bProxy.getNetProxy());
        }

        ChromeOptions chromeOptions = bProxy.getChromeOptions();
        WebDriver driver = new ChromeDriver(chromeOptions);
        bProxy.newHar();
        bProxy.newPage("CNN Home Page");

        List<String> allowUrlPatterns = new ArrayList<String>();
        allowUrlPatterns.add("https?://.*(cnn.com)+.*");
        bProxy.whitelistRequests(allowUrlPatterns, 200);
        String blackListedItems = ".*(.cdn.*|.png|.jpg|.js)";
        bProxy.blacklistRequests(blackListedItems, 410);
        driver.get("http://www.cnn.com");
        //driver.findElement(By.name("q")).sendKeys("Browser Mob Proxy");
        bProxy.writeHar(bProxy.getHar(), "target/test.har");
        log.info("Shutting down Browser Mob Proxy");

        bProxy.quit();
        driver.quit();
    }
}

[Jeff White]

Correction of the AutoWired:

@AutoWired

BProxy bProxy;

Then you will not need "BProxy bProxy = ProxyConfig.bProxy;" in the test. 

[Jeff White]

Corrected class:

//CHECKSTYLE:OFF checkstyle:magicnumber

import io.github.bonigarcia.wdm.ChromeDriverManager;
import lombok.extern.slf4j.Slf4j;
import net.lightbody.bmp.proxy.CaptureType;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import java.net.InetSocketAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.List;



@Slf4j
@RunWith(SpringRunner.class)
@SpringBootTest
public class BProxyIT {

    @Autowired

    private  BProxy bProxy;

    /***********************************************************
     * public void bproxyTest()
     *
     * Test of BrowserMob Proxy
     **********************************************************/



    @Test
    public void bproxyTest() {

        ChromeDriverManager.chromedriver().setup();
        try {
            bProxy.getProxyServer(CaptureType.REQUEST_HEADERS, CaptureType.RESPONSE_HEADERS);
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }

        // Check for Proxy (Currently turned off on the NextGear network
        InetSocketAddress addr = bProxy.getNetProxy();

        if (addr != null) {
            bProxy.getProxy().setChainedProxy(bProxy.getNetProxy());
        }

        ChromeOptions chromeOptions = bProxy.getChromeOptions();
        WebDriver driver = new ChromeDriver(chromeOptions);
        bProxy.newHar();

        bProxy.newPage("Youtube Home Page");

        List<String> allowUrlPatterns = new ArrayList<String>();

        allowUrlPatterns.add("https?://.*(youtube.com)+.*");

        bProxy.whitelistRequests(allowUrlPatterns, 200);
        String blackListedItems = ".*(.cdn.*|.png|.jpg|.js)";
        bProxy.blacklistRequests(blackListedItems, 410);

        driver.get("http://www.youtube.com");