embedded browsermob proxy 2.1.1 with SSL and chained proxy

[karens...@yahoo.com]

I have been unsuccessfully trying to get the embedded browsermob proxy to work for SSL requests in a chained proxy environment. 

This site in this code works for Chrome and Firefox, but I only get a partial .har file with IE and errors like this:

34158 [LittleProxy-0-ClientToProxyWorker-4] INFO org.littleshoot.proxy.impl.ClientToProxyConnection - (AWAITING_INITIAL) [id: 0xa3546419, L:/fe80:0:0:0:914d:bb9c:f458:c88f%13:21471 - R:/fe80:0:0:0:914d:bb9c:f458:c88f%13:21503]: An IOException occurred on ClientToProxyConnection: An existing connection was forcibly closed by the remote host

In order to get this to work I did import into IE the ca-certificate.rsa.cer file that was posted here in the SSL support info: 
https://github.com/lightbody/browsermob-proxy

Here is my pom.xml dependency:

  <dependency>
   <groupId>net.lightbody.bmp</groupId>
   <artifactId>browsermob-core</artifactId>
   <version>2.1.1</version>
  </dependency>

And here is sample code:

 System.setProperty("bmp.allowNativeDnsFallback", "true");
 // start the proxy
 BrowserMobProxy proxy = new BrowserMobProxyServer();

 proxy.setChainedProxy(new InetSocketAddress("aaaa.bcde.com", 80));
 proxy.setTrustAllServers(true);

 // https://github.com/lightbody/browsermob-proxy/tree/master/mitm
 proxy.setMitmManager(ImpersonatingMitmManager.builder().trustAllServers(true).build());

 proxy.start();

 // get the Selenium proxy object
 Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

 // configure it as a desired capability
 DesiredCapabilities capabilities = new DesiredCapabilities();

 // IE ONLY
 capabilities.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
 capabilities.setCapability("ignoreZoomSetting", true);
 // Keep the proxy settings from being overwritten
 capabilities.setCapability(InternetExplorerDriver.IE_USE_PRE_PROCESS_PROXY, true);
 capabilities.setCapability(InternetExplorerDriver.UNEXPECTED_ALERT_BEHAVIOR, "accept");
 // END IE ONLY
  
 capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
 
 // start the browser up
 //WebDriver driver = new FirefoxDriver(capabilities);
 //WebDriver driver = new ChromeDriver(capabilities);
 WebDriver driver = new InternetExplorerDriver(capabilities);
 driver.manage().timeouts().pageLoadTimeout(45, TimeUnit.SECONDS);
 proxy.enableHarCaptureTypes(CaptureType.RESPONSE_CONTENT);

 proxy.newHar("boa");

 try {
    driver.get("https://www.bankofamerica.com/");
  } catch (TimeoutException ex) {
      // keep going
 }
// End of sample code

Can anyone tell me what I am missing?

Thank you in advance.

[je...@outlook.com]

Is the same exact code working with Firefox/Chrome but not IE? That would point to a problem with IE or the IE implementation of WebDriver.

Your code looks correct. One thing you might try is setting the java.net.preferIPv4Stack system property to true.

[Karen Devlin]

Thank you for your response.  After a lot of investigation, trial and error I finally got it working (at least at this moment) on Windows 7, 10 and MacOS for Chrome, Firefox, IE and Safari.  Unfortunately, I ran into multiple bumps in multiple areas and I can't remember which change ended up fixing this problem.  I have slept once or twice since making this post.  I'm including my setup code just fyi.  In the following method, the setChainedProxy flag is false only for one of our internal sites when running Chrome on Windows 10 due to a different proxy setup.  Note that I actually removed the MitmManager code.

Other than that, I did have to add this system property code as well:

// http://stackoverflow.com/questions/7615645/ssl-handshake-alert-unrecognized-name-error-since-upgrade-to-java-1-7-0

System.setProperty("jsse.enableSNIExtension", "false");

 private WebDriver bmpDriverSetup(String browserType, boolean setChainedProxy) {
      WebDriver driver = null;

      // http://stackoverflow.com/questions/26824979/how-to-set-browsermob-with-webdriver

      System.setProperty("bmp.allowNativeDnsFallback", "true");

      // https://github.com/lightbody/browsermob-proxy
      // https://bmp.lightbody.net/

      BrowserMobProxy bmProxy = new BrowserMobProxyServer();

      // https://github.com/lightbody/browsermob-proxy/issues/121
      // https://groups.google.com/forum/#!topic/browsermob-proxy/B8dig-njlz4
      if (setChainedProxy) {
       bmProxy.setChainedProxy(new InetSocketAddress("corporateproxy.com", 443));    
       bmProxy.chainedProxyAuthorization(CommonUtils.TEST_USER, CommonUtils.TEST_PASSWORD, AuthType.BASIC);
      }
      bmProxy.setTrustAllServers(true);

      // https://github.com/lightbody/browsermob-proxy/issues/358
      if (browserType.equals("Safari")) {
       bmProxy.start(11211);
      } else {
       bmProxy.start();
      }

      // get the Selenium proxy object

      DesiredCapabilities capabilities = new DesiredCapabilities();

      Proxy seleniumProxy = ClientUtil.createSeleniumProxy(bmProxy);

      if (browserType.equals("Safari")) {
       capabilities = DesiredCapabilities.safari();
      }

      capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
      // https://github.com/lightbody/browsermob-proxy/issues/108
      capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);

      String browser = browserType.toLowerCase();

      if (browser.contains("explorer")) {

       capabilities.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
       capabilities.setCapability("ignoreZoomSetting", true);

       capabilities.setCapability(InternetExplorerDriver.IE_ENSURE_CLEAN_SESSION, true);
       capabilities.setCapability(InternetExplorerDriver.ENABLE_ELEMENT_CACHE_CLEANUP, true);
       // Keep the system proxy settings from being overwritten
       capabilities.setCapability(InternetExplorerDriver.IE_USE_PRE_PROCESS_PROXY, true);
       driver = new InternetExplorerDriver(capabilities);
      } else if (browserType.equalsIgnoreCase("Firefox")) {
       driver = new FirefoxDriver(capabilities);
      } else if (browserType.equalsIgnoreCase("Chrome")) {
       driver = new ChromeDriver(capabilities);
      } else if (browserType.equalsIgnoreCase("Safari")) {
       driver = new SafariDriver(capabilities);
      } else {
       CommonUtils.error("Unable to launch browser type:  " + browserType, null);
      }

      bmProxy.enableHarCaptureTypes(CaptureType.RESPONSE_CONTENT);
      bmProxy.newHar("bmpMetrics");
      setBMProxy(bmProxy);
      return driver;
 }

---

From: "je...@outlook.com" <je...@outlook.com>
To: BrowserMob Proxy <browserm...@googlegroups.com>
Sent: Saturday, July 9, 2016 6:37 PM
Subject: Re: embedded browsermob proxy 2.1.1 with SSL and chained proxy

Is the same exact code working with Firefox/Chrome but not IE? That would point to a problem with IE or the IE implementation of WebDriver.

Your code looks correct. One thing you might try is setting the java.net.preferIPv4Stack system property to true.

On Monday, June 13, 2016 at 3:08:33 PM UTC-7, karens...@yahoo.com wrote:

I have been unsuccessfully trying to get the embedded browsermob proxy to work for SSL requests in a chained proxy environment. 

This site in this code works for Chrome and Firefox, but I only get a partial .har file with IE and errors like this:

34158 [LittleProxy-0- ClientToProxyWorker-4] INFO org.littleshoot.proxy.impl. ClientToProxyConnection - (AWAITING_INITIAL) [id: 0xa3546419, L:/fe80:0:0:0:914d:bb9c:f458: c88f%13:21471 - R:/fe80:0:0:0:914d:bb9c:f458: c88f%13:21503]: An IOException occurred on ClientToProxyConnection: An existing connection was forcibly closed by the remote host

In order to get this to work I did import into IE the ca-certificate.rsa.cer file that was posted here in the SSL support info: 
https://github.com/lightbody/browsermob-proxy

Here is my pom.xml dependency:

  <dependency>
   <groupId>net.lightbody.bmp< /groupId>
   <artifactId>browsermob- core</artifactId>
   <version>2.1.1</version>
  </dependency>

And here is sample code:

 System.setProperty("bmp. allowNativeDnsFallback", "true");

 // start the proxy
 BrowserMobProxy proxy = new BrowserMobProxyServer();

 proxy.setChainedProxy(new InetSocketAddress("aaaa.bcde. com", 80));
 proxy.setTrustAllServers( true);

 // https://github.com/lightbody/browsermob-proxy/tree/master/mitm
 proxy.setMitmManager( ImpersonatingMitmManager. builder().trustAllServers( true).build());

 proxy.start();

 // get the Selenium proxy object
 Proxy seleniumProxy = ClientUtil. createSeleniumProxy(proxy);

 // configure it as a desired capability
 DesiredCapabilities capabilities = new DesiredCapabilities();

 // IE ONLY
 capabilities.setCapability( InternetExplorerDriver. INTRODUCE_FLAKINESS_BY_ IGNORING_SECURITY_DOMAINS, true);

 capabilities.setCapability(" ignoreZoomSetting", true);
 // Keep the proxy settings from being overwritten
 capabilities.setCapability( InternetExplorerDriver.IE_USE_ PRE_PROCESS_PROXY, true);

 capabilities.setCapability( InternetExplorerDriver. UNEXPECTED_ALERT_BEHAVIOR, "accept");
 // END IE ONLY
  
 capabilities.setCapability( CapabilityType.PROXY, seleniumProxy);

 
 // start the browser up
 //WebDriver driver = new FirefoxDriver(capabilities);
 //WebDriver driver = new ChromeDriver(capabilities);
 WebDriver driver = new InternetExplorerDriver( capabilities);
 driver.manage().timeouts(). pageLoadTimeout(45, TimeUnit.SECONDS);
 proxy.enableHarCaptureTypes( CaptureType.RESPONSE_CONTENT);

 proxy.newHar("boa");

 try {
    driver.get("https://www.bankofamerica.com/");
  } catch (TimeoutException ex) {
      // keep going
 }
// End of sample code

Can anyone tell me what I am missing?

Thank you in advance.

--

---
You received this message because you are subscribed to a topic in the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/browsermob-proxy/yJ6rgltjpro/unsubscribe.
To unsubscribe from this group and all its topics, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.