How does one set the network.proxy setting when calling BrowserMobProxy via the Selenium Webdriver or SST?

[Michael Hollinger]

So, I've written up a probelm that just plain has me stuck using BrowserMob proxy.  This question on StackOverflow:

details my setup and how I'm stuck.

The jist of it is that when I try to incorporate the proxy (so that I can generate a dump of all the calls my page under test is making out), I get the following error:

ERROR: test_simpleTest (sst.runtests.TestSimpletest)

----------------------------------------------------------------------

Traceback (most recent call last):

File "/usr/local/lib/python2.7/dist-packages/sst/runtests.py", line 270, in setUp webdriver_remote_url)

File "/usr/local/lib/python2.7/dist-packages/sst/actions.py", line 188, in start profile.set_proxy(selenium_proxy)

File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_profile.py", line 244, in set_proxy

self._set_manual_proxy_preference("ftp", proxy.ftp_proxy)

File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/firefox_profile.py", line 263, in _set_manual_proxy_preference

self.set_preference("network.proxy.%s" % key, host_details[1][2:])

IndexError: list index out of range

I can verify that the test runs *without* the proxy (so my setup isn't completely hosed), but I've yet to really get a good checklist for "this is how you set up the proxy so that your scripts can use it."

My Simple Test is straight forward - Using SST I literally just say

from sst.actions import *

go_to('http://usatoday.com')

and then it is run ussing sst-run simpleTest -x --browsermob=proxy-2.0.7/bin/browsermob-proxy

I've also tried running a standalone script like this:

from selenium import webdriver

from browsermobproxy import Server

import sys

server = Server("/Users/mhollinger/Google Drive/Code/SST/proxy-2.0.7/bin/browsermob-proxy")

server.start()

proxy = server.create_proxy()

proxy.new_har("usat")

#initialize webdriver

profile  = webdriver.FirefoxProfile()

profile.set_preference("network.proxy.http", "\"localhost\"")

profile.set_preference("network.proxy.http_port", 9090)

profile.set_preference("network.proxy.type", 1)

profile.update_preferences()

profile.set_proxy(proxy.selenium_proxy())

browser = webdriver.Firefox(firefox_profile=profile)

browser.get('http://www.usatoday.com')

print browser.title

print dir(proxy.har)

print proxy.har()

print browser.page_source

browser.quit() #important, or else loaded browser will remain running as a bg proc!

display.stop()

server.stop()

Could someone either tell me how to pass this setting in my script, or, barring that, point me to some good documentation?  (My code, setup, and the documentation I have are in the link above, and hey - you get points for answering!)  Thanks in advance.

[Patrick Lightbody]

Michael,

I'd love to help, but I have no idea what SST is and don't really have any experience with anything other than vanilla BMP + vanilla Selenium, which is covered in the readme. Good luck!

Patrick

--
 
---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Michael Hollinger]

Hi Patrick!

My time to apologize - I got pulled :)

Anyways, 

#1 - SST is this framework here http://testutils.org/sst/ It looks like a nice Python wrapper - but in the interests of simplicity, I've dropped it.

#2 - I was able to get this working *mostly* in native BMP.  This is how far I got:

from browsermobproxy import Server

from selenium import webdriver

from selenium.webdriver.common.proxy import *

import json

server = Server("/Users/mhollinger/Downloads/browsermob-proxy-2.0-beta-6/bin/browsermob-proxy")

server.start()

seleniumproxy = server.create_proxy()

try:

driver = webdriver.Firefox(proxy=seleniumproxy.selenium_proxy())

seleniumproxy.new_har("usat")

driver.get("http://www.usatoday.com/")

#---The first call doesn't seem to record anything, so I do a seccond

#---Note, I tried introducing a two second delay, but that didn't fix it on its own...

driver.get("http://www.usatoday.com/news/nation")

print "Actual HAR"

print seleniumproxy.har() # returns a HAR JSON blob

#---HAR is coming back blank---

print "Actual HAR"

json.dumps(seleniumproxy.har)

seleniumproxy.stop()  #And this doesn't work ('Attribute Error: 'Client' object has no attribute 'stop')

finally:

driver.quit()

Is it normal that I need to make two calls?

Thanks!

[Patrick Lightbody]

Unfortunately, I still can't really help since don't have any experience with the BMP wrappers that are out there. Which one are you using? Is that language Python? Sorry, I'm kind of a n00b when it comes to anything other than Java!