Firefox Proxy error (selenium 2.39.0, Firefox 22.0 on Ubuntu Linux)

[Mike A]

I'd like to proxy web traffic, so first I port-forward to a server on my local network:

  $  ssh -D 9999 te...@10.xxx.xxx.xxx

Then I try to run the script shown below, but it returns an error shown below.

The proxy works if I open firefox manually and set the proxy (Edit...Preferences...Advanced...Settings...Manual proxy config...localhost:9999).
But why doesn't it work in selenium?


test.py
--------------------
import time

from selenium import webdriver
from selenium.webdriver.common.proxy import *

myProxy = "host:9999"

proxy = Proxy({
    'proxyType': 'MANUAL',
    'httpProxy': myProxy,
    'ftpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': '' # set this value as desired
    })

driver = webdriver.Firefox(proxy=proxy)
# for remote
caps = webdriver.DesiredCapabilities.FIREFOX
proxy.add_to_capabilities(caps)
driver = webdriver.Remote(desired_capabilities=caps)

url = 'http://www.youtube.com/watch?v=3oem-M2tQU4'
driver.implicitly_wait(30)
driver.get(url)
time.sleep(30)
driver.quit()
--------------------


Traceback
--------------------
$ firefox -V
Mozilla Firefox 22.0
$ python -c "import selenium; print selenium.__version__"
2.39.0
$ python test.py
Traceback (most recent call last):
  File "test.py", line 20, in <module>
    driver = webdriver.Remote(desired_capabilities=caps)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 71, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 113, in start_session
    'desiredCapabilities': desired_capabilities,
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 162, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 349, in execute
    return self._request(url, method=command_info[0], data=data)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/remote_connection.py", line 410, in _request
    resp = opener.open(request)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 111] Connection refused>

[Mike A]

I have come to realize that the script in the original post sets a HTTP proxy, but what is needed is a SOCKS proxy.

Here is a way that works.

# Connect and forward a port

$ ssh -D 9999 te...@10.xxx.xxx.xxx

Then run this script:

import time

from selenium import webdriver

profile=webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', 'localhost')
profile.set_preference('network.proxy.socks_port', 9999)

driver = webdriver.Firefox(profile)