Empty har file for Safari with Selenium and browsermobproxy

[coohe...@gmail.com]

Hello,

I am trying to get the har file for Safari using Selenium WebDriver and BMP.  However, I get an empty har file. 

I started the selenium server (java -jar ~/Downloads/selenium-server-standalone-2.41.0-2.jar )

This is my code snippet :
============
server = Server("~/browsermob-proxy-2.0-beta-6/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium.webdriver.common.desired_capabilities import
DesiredCapabilities
from selenium import webdriver
capabilities = DesiredCapabilities.SAFARI
capabilities['proxy'] = proxy.proxy
capabilities['platform'] = 'MAC'
capabilities['version'] = '7.0'

//Using Remote
proxy.new_har("SafariApple1")
browser1 = webdriver.Remote("http://localhost:4444/wd/hub", capabilities)
browser1.get('http://www.apple.com')
with open("/tmp/apple_s.json", 'w') as outfile1:
    json.dump(proxy.har, outfile1)
browser1.quit()

//Using Safari web driver
proxy.new_har("SafariApple2")
browser2 = webdriver.Safari(desired_capabilities=capabilities)
browser2.get('http://www.apple.com/iphone-5s')
with open("/tmp/iphone-5s_s.json", 'w') as outfile2:
    json.dump(proxy.har, outfile2)
browser2.quit()

I am attaching both the har files. Can you please give me pointers as to
what I can do to fix the problem.

thanks,
Madhur

[Scott]

I tried this without RC, and I do not believe that Safari respects the proxy you set this way. If I pre-setup the proxy in the System Settings, and then launched my test things worked okay.

Perhaps you can login to that mac machine, and setup the proxy prior to running your test to see if things work properly?

[coohe...@gmail.com]

Hi Scott, 

Thanks a ton for your response. I tried setting the proxy in System Preferences but I am hitting other problem. I get an unknown host exception when the Safari driver tries to connect to the selenium server. Can you please let me what I am missing. 

In System preferences Web Proxy(HTTP) -> 17.218.71.96 9101 (ip and port for server)

my code :

        ProxyServer server = new ProxyServer(9101) ;

        server.start();

        // get the Selenium proxy object

        Proxy proxy = server.seleniumProxy();

        proxy.setHttpProxy("17.218.71.96:9101");

        // configure it as a desired capability

        //DesiredCapabilities capabilities = new DesiredCapabilities();

        DesiredCapabilities capabilities = DesiredCapabilities.safari();

        

        // start the browser up

        WebDriver driver = new SafariDriver(capabilities);

In the last step, I get the following exception: 

Apr 11, 2014 5:17:19 PM org.openqa.selenium.safari.SafariDriverCommandExecutor start

INFO: Waiting for SafariDriver to connect

Apr 11, 2014 5:17:20 PM net.lightbody.bmp.proxy.util.Log info

INFO: java.net.UnknownHostException: localhost when requesting http://localhost:32476/

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Shutting down

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Stopping Safari

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Stopping server

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverServer stop

INFO: Stopping server

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Uninstalling extensions

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Shutdown complete

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Shutting down

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Stopping server

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Uninstalling extensions

Apr 11, 2014 5:18:04 PM org.openqa.selenium.safari.SafariDriverCommandExecutor stop

INFO: Shutdown complete

[Scott]

Afraid I'm not able to offer much help. All of my tests were done on a local machine.

You might need to start the proxy, and use the restful API when the test is (possibly) running on another machine.  I believe I saw on the bug reports or the summary message here the other day that the restful services were not working properly in embedded mode, so you might have to run this outside the scope of your test.

Best of luck.

[coohe...@gmail.com]

Hi Scott, 

My test is running on the local machine. I set the proxy in the system preferences (image attached) and then get the error that Safari driver is not able to connect. Were you able to get a har file for Safari? Did you have to start the selenium server in a special way? 

thanks!

[Scott]

Again, I did this outside of RC / RemoteWebDrivers.

Looking back at that code I may not have gotten things working with Safari at all, as my Groovy code looked like the following:

def static createBrowser(String selectedBrowser) {

  server = new ProxyServer(9090)

  server.start()

  WebDriver driver

  Proxy proxy = server.seleniumProxy()

  DesiredCapabilities dc = new DesiredCapabilities();

  switch (selectedBrowser) {

  case "ie":

    dc.setCapability(CapabilityType.PROXY, proxy)

    dc.setCapability("ie.setProxyByServer", true)

    driver = new InternetExplorerDriver(dc)

  break

  case "firefox":

    dc.setCapability(CapabilityType.PROXY, proxy)

    driver = new FirefoxDriver(dc)

  break

  case "chrome":

    ChromeOptions options = new ChromeOptions()

    options.addArguments("--proxy-server=localhost:9090")

    driver = new ChromeDriver(options)

  break

  case "opera":

    if ("Windows 8".equals(System.getProperty("os.name"))) {

      System.setProperty("os.name", "windows")

    }

    dc.setCapability(CapabilityType.PROXY, proxy)

    //driver = new OperaDriver(dc)

    driver = null //Opera has not updated to allow for testing with the latest version of their browser yet

  break

  case "safari":

    dc.setCapability(CapabilityType.PROXY, proxy)

    //driver = new SafariDriver(dc)

    driver = null //Safari Driver ignores the proxy settings...

  break

  default:

    println "This browser needs to be setup"

  break

  }

  driver

}

However I am confident that I got this to work with the REST API. I'll supply pseudo code here:

1) Start the browsermob proxy on the command line

2) Setup safari to use the proxy

3) Create a new har, curl should be fine, or the following -- there are a couple of things you'd need to setup to use this

def static createNewHar() {

  //Create HttpPut to your proxy server (note this will not work, it's not fluent), it will use default params

  new HttpPut("17.218.71.96/proxy/9091/har").setHeader("Content-Type", "application/x-www-form-urlencoded").execute() 

  new ObjectMapper().readValue(data, Har.class)

}

4. Create the driver

5. Drive to the page

6. Hit rest api and get har back

def static retrieveHar() {

  new ObjectMapper().readValue(new HttpGet("17.218.71.96/proxy/9091/har").execute(), Har.class))

}

Again, best of luck. It seems I eventually ignored safari even with local setup.

[Vijay Kalkundri]

Hi , 

 i do not get the full request and response as a part of the har file for the safari .I am able to get the network traffic for the firefox,Chrome. I am also attaching thr network traffic details that i get from both firefox and the safari. What is the modification that i need to do so that it captures the full network traffic :

server = Server("browsermob-proxy/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()
proxy.new_har(website)
if browser_type.upper()  == 'FIREFOX':
    profile = webdriver.FirefoxProfile()
    driver = webdriver.Firefox(firefox_profile=profile, proxy=proxy)
elif browser_type.upper() == 'CHROME':
    capabilities = DesiredCapabilities.CHROME
    proxy.add_to_capabilities(capabilities)
    driver = webdriver.Chrome(desired_capabilities=capabilities)
elif browser_type.upper() == 'SAFARI':
    profile_safari = webdriver.Safari
    dc = {'browserName' : 'safari','platform':'MAC'}
    proxy.add_to_capabilities(dc)
    driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',desired_capabilities=dc)

Regards,

Vijay Kalkundri