Java - BMP with selenium grid

[Khan]

Hi all, 

Is there any working example of BMP with selenium grid? BMP is working fine locally, but not on selenium grid. Can someone please share your knowledge?

Regards,

Khan

[⇜Krishnan Mahadevan⇝]

Khan,

Can you please help elaborate what do you mean when you say "BMP is not working on Selenium Grid" ?

The Grid has got nothing to do with a proxy server [ BMP in this case ]. You only need to ensure that there is connectivity between the machine that hosts your nodes and the machine that hosts your BMP server.

Apart from that nothing else is required.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-proxy+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Khan]

Hi Krishnan,

Thanks for replying.

I'm saving the network requests/responses to a json HAR and then parsing the file for required requests/responses. I'm using the embedded way of starting the browsermob proxy on my local pc and everything is working as expected.

However, when I try to run the same test using jenkins(unix machine) running a selenium grid (windows machine - 1 node and 1 hub on the same machine) and using the remote-webdriver configuration then I'm not able to send the network traffic through browsermob proxy.

local webdriver looks like this:

public class DriverFactory {

    public static WebDriver driver = null;

    public static BrowserMobProxy proxy = new BrowserMobProxyServer();

    public static void setupDriver() {

        proxy.start(0);

        String browser = ConfigParser.getBrowser();

        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

        if (browser.equalsIgnoreCase("chrome")) {

            DesiredCapabilities chromeCapabilities = DesiredCapabilities.chrome();

            chromeCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy);

            String chromeBinary = System.getProperty(" ");

            if (chromeBinary == null || chromeBinary.equals("")) {

                String os = System.getProperty("os.name").toLowerCase().substring(0, 3);

                chromeBinary = "src/test/resources/libs/chromedriver-" + os + (os.equals("win") ? ".exe" : "");

                System.setProperty("webdriver.chrome.driver", chromeBinary);

                ChromeOptions options = new ChromeOptions();

                options.addArguments("--start-maximized");

            }

            proxy.enableHarCaptureTypes(CaptureType.getAllContentCaptureTypes());

            driver = new ChromeDriver(chromeCapabilities);

        } else {

            System.out.println("Using default browser as firefox");

            DesiredCapabilities capabilities = DesiredCapabilities.firefox();

            capabilities.setCapability("marionette", true);

            //System.setProperty("webdriver.firefox.bin", "/usr/lib/firefox/firefox");

            driver = new FirefoxDriver(capabilities);

        }

        setupBrowser(driver);

    }

    private static void setupBrowser(WebDriver driver) {

        driver.manage().window().setPosition(new Point(0, 0));

        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

    }

    public static WebDriver getWebDriver() {

        if (driver == null) {

            setupDriver();

        }

        return driver;

    }

}

Thanks in advance.

Khan

[⇜Krishnan Mahadevan⇝]

 Khan,

So I think you have two options :

Option #1

Stick to a constant port when spawning your BMP via the embedded mode, check and ensure that the port on which the BMP Server is listening on the UNIX Machine (which hosts your Jenkins instance), connectivity is open between the UNIX machine and then Windows machine.

Option #2

Pick a machine to act as your BMP server wherein you will spin off the BMP server in the standalone mode and then use the REST calls that BMP provides to do everything else. Here also the same constraint remains viz., to open up port connectivity between the machine where the BMP server runs and the windows node. The advantage with this approach is that it lets you test out port opened or not sort of manual testing that you can do, because all you would need to do is spawn a browser manually on the node and then try opening up a web page after you impregnated your proxy information into that browser.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

[Khan]

Krishnan,

Using the Option 1: my RemoteWebDriver looks like this:

public class RemoteWebDriverFactory {

    public static final String SELENIUM_HUB = "http://___.de:4444/wd/hub";
    public static RemoteWebDriver driver = null;

    public static BrowserMobProxy proxy = new BrowserMobProxyServer();

    public static void setupDriver() throws MalformedURLException {
        proxy.start(4455);

        String browser = ConfigParser.getBrowser();
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
        if (browser.equalsIgnoreCase("chrome")) {

            DesiredCapabilities capability = new DesiredCapabilities();
            capability.setCapability(CapabilityType.PROXY, seleniumProxy);
            capability.setBrowserName("chrome");
            capability.setPlatform(Platform.WIN10);
            System.setProperty("webdriver.chrome.driver", "/usr/bin/google-chrome");
            remoteWebDriver(capability, new URL(SELENIUM_HUB));

        } else {
            System.out.println("Using default browser as firefox");

            DesiredCapabilities capability = new DesiredCapabilities();
            capability.setBrowserName("firefox");
            capability.setPlatform(Platform.WIN10);
            remoteWebDriver(capability, new URL(SELENIUM_HUB));
        }
        setupBrowser(driver);
    }

However, when running test, remotedesktop to the windows machine shows:  "No internet connection. Check your proxy server."

The port 4455 between jenkins and unix machine is open.

What am I missing? kindly guide.

On Wednesday, August 24, 2016 at 12:49:03 PM UTC+2, Khan wrote:

[⇜Krishnan Mahadevan⇝]

Khan,

Can you please try changing your proxy instantiation logic to something like below and see if that helps ?

org.openqa.selenium.net.NetworkUtils utils = new org.openqa.selenium.net.NetworkUtils();
InetAddress nonLoopBackInet = utils.getIp4NonLoopbackAddressOfThisMachine();
proxy.start(4455, nonLoopBackInet);
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy, nonLoopBackInet);

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

[Khan]

Krishnan,

The proxy instantiation logic has some effect and now selenium logs shows correct jenkins host, however proxy is still not used and chrome shows:

Something wrong with the proxy server or address is incorrect.
ERR_PROXY_CONNECTION_FAILED.

Selenium log:

09:13:50.233 INFO - Executing: [new session: Capabilities [{proxy={autodetect=false, httpProxy=jenkins....:4455, proxyType=MANUAL, hCode=1835841364, class=org.openqa.selenium.Proxy, sslProxy=jenkins....:4455}, browserName=chrome, platform=WIN10}]])

hubCofig:

{"_comment" : "Configuration for Hub - hubConfig.json",
  "host": Jenkins IP,
  "maxSessions": 5,
  "port": 4444,
  "cleanupCycle": 5000,
  "timeout": 300000,
  "newSessionWaitTimeout": -1,
  "servlets": [],
  "prioritizer": null,
  "capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
  "throwOnCapabilityNotPresent": true,
  "nodePolling": 180000,
  "platform": "WINDOWS"}

DefaultNode:

{
  "capabilities":
      [
        {
          "browserName": "chrome",
          "maxInstances": 2,
          "seleniumProtocol": "WebDriver",
  "webdriver.chrome.driver": "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe"
        },
      ],
  "configuration":
  {
    "proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
    "maxSession": 5,
    "port": 5555,
    "host": Windows node IP,
    "register": true,
    "registerCycle": 5000,
    "hubPort": 4444,
    "hubHost": Jenkins IP
  }
}

WebDriver class:

public class RemoteWebDriverFactory {
    public static final String SELENIUM_HUB = "http://Jenkins_machine:4444/wd/hub";

    public static RemoteWebDriver driver = null;
    public static BrowserMobProxy proxy = new BrowserMobProxyServer();

    static org.openqa.selenium.net.NetworkUtils utils = new org.openqa.selenium.net.NetworkUtils();
    static InetAddress nonLoopBackInet = utils.getIp4NonLoopbackAddressOfThisMachine();

    public static void setupDriver() throws MalformedURLException {

        proxy.start(4455, nonLoopBackInet);

        String browser = ConfigParser.getBrowser();
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
        if (browser.equalsIgnoreCase("chrome")) {
            DesiredCapabilities capability = new DesiredCapabilities();
            capability.setCapability(CapabilityType.PROXY, seleniumProxy);
            capability.setBrowserName("chrome");
            capability.setPlatform(Platform.WIN10);
            System.setProperty("webdriver.chrome.driver", "/usr/bin/google-chrome");
            remoteWebDriver(capability, new URL(SELENIUM_HUB));
        } else {

            DesiredCapabilities capability = new DesiredCapabilities();
            capability.setBrowserName("firefox");
            capability.setPlatform(Platform.WIN10);
            remoteWebDriver(capability, new URL(SELENIUM_HUB));
        }
        setupBrowser(driver);
    }

    private static void remoteWebDriver(DesiredCapabilities capability, URL remoteAddress) {
        driver = new RemoteWebDriver(remoteAddress, capability);

    }

    private static void setupBrowser(WebDriver driver) {
        driver.manage().window().setPosition(new Point(0, 0));
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }

    public static WebDriver getWebDriver() throws MalformedURLException {

        if (driver == null) {
            setupDriver();
        }
        return driver;
    }
}

Now thats the complete configuration info. Maybe, I'm missing some settings?

Regards

To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.

[⇜Krishnan Mahadevan⇝]

Khan,

Without having access to the environment I don't think I can do much here. I am out of ideas.

You would need to spend time debugging this at your side. One thing for sure is that you have perhaps a network related issue

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribbings @ http://rationaleemotions.wordpress.com/

To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-proxy+unsubscribe@googlegroups.com.

[Khan]

Thanks Krishnan for your support.

I'll recheck the network.

Regards