Empty response while using Selenium python chrome headless with proxy

V Reddy

unread,

Jan 8, 2020, 11:48:21 PM1/8/20

to Selenium Users

Hi,

I am running into an issue while trying to test the Selenium script with Chrome in headless mode and only during when I am setting the proxy settings.

The script works just fine when I am not setting any proxy, but returns an empty html body when I add the proxy:

<html><head></head><body></body></html>

Below is the log from my console:

2020-01-08 22:14:19,297 - selenium.webdriver.remote.remote_connection : 389 - DEBUG - POST http://127.0.0.1:44373/session {"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"binary": "/usr/bin/google-chrome", "args": ["headless", "no-sandbox", "ignore-certificate-errors", "disable-gpu", "disable-dev-shm-usage", "allow-running-insecure-content", "proxy-bypass-list=<-loopback>", "--proxy-server=http://127.0.0.1:8090", "--proxy-auto-detect"], "extensions": []}, "browserName": "chrome", "platformName": "any"}, "firstMatch": [{}]}, "desiredCapabilities": {"goog:chromeOptions": {"binary": "/usr/bin/google-chrome", "args": ["headless", "no-sandbox", "ignore-certificate-errors", "disable-gpu", "disable-dev-shm-usage", "allow-running-insecure-content", "proxy-bypass-list=<-loopback>", "--proxy-server=http://127.0.0.1:8090", "--proxy-auto-detect"], "extensions": []}, "platform": "ANY", "browserName": "chrome", "version": ""}}
2020-01-08 22:14:19,297 - urllib3.connectionpool : 205 - DEBUG - Starting new HTTP connection (1): 127.0.0.1:44373
2020-01-08 22:14:19,446 - urllib3.connectionpool : 393 - DEBUG - http://127.0.0.1:44373 "POST /session HTTP/1.1" 200 642
2020-01-08 22:14:19,447 - selenium.webdriver.remote.remote_connection : 440 - DEBUG - Finished Request
2020-01-08 22:14:19,447 - selenium.webdriver.remote.remote_connection : 389 - DEBUG - POST http://127.0.0.1:44373/session/c602cc3049abfb01e74d7d4d8a8b2f59/url {"url": "https://www.google.com"}
2020-01-08 22:14:19,504 - urllib3.connectionpool : 393 - DEBUG - http://127.0.0.1:44373 "POST /session/c602cc3049abfb01e74d7d4d8a8b2f59/url HTTP/1.1" 200 14
2020-01-08 22:14:19,504 - selenium.webdriver.remote.remote_connection : 440 - DEBUG - Finished Request
2020-01-08 22:14:19,504 - selenium.webdriver.remote.remote_connection : 389 - DEBUG - GET http://127.0.0.1:44373/session/c602cc3049abfb01e74d7d4d8a8b2f59/source {}
2020-01-08 22:14:19,514 - urllib3.connectionpool : 393 - DEBUG - http://127.0.0.1:44373 "GET /session/c602cc3049abfb01e74d7d4d8a8b2f59/source HTTP/1.1" 200 81
2020-01-08 22:14:19,515 - selenium.webdriver.remote.remote_connection : 440 - DEBUG - Finished Request
2020-01-08 22:14:19,515 - selenium-auth : 48 - INFO - Page-source -> <html><head></head><body></body></html>
2020-01-08 22:14:19,515 - selenium.webdriver.remote.remote_connection : 389 - DEBUG - DELETE http://127.0.0.1:44373/session/c602cc3049abfb01e74d7d4d8a8b2f59/window {}
2020-01-08 22:14:19,581 - urllib3.connectionpool : 393 - DEBUG - http://127.0.0.1:44373 "DELETE /session/c602cc3049abfb01e74d7d4d8a8b2f59/window HTTP/1.1" 200 12
2020-01-08 22:14:19,582 - selenium.webdriver.remote.remote_connection : 440 - DEBUG - Finished Request

Below are the system and software info:

OS: Ubuntu 18.04
Selenium: 3.14.0
google-chrome: Google Chrome 79.0.3945.117
chromedriver: ChromeDriver 79.0.3945.36

Python script that I am using:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options, DesiredCapabilities
from selenium.webdriver.common.proxy import Proxy, ProxyType

options = Options()

options.set_capability("acceptInsecureCerts", True)
options.set_capability("acceptSslCerts", True)
options.binary_location = "/usr/bin/google-chrome"
options.add_argument('headless')
options.add_argument('no-sandbox')
options.add_argument('ignore-certificate-errors')
options.add_argument('disable-gpu')
options.add_argument('disable-dev-shm-usage')
options.add_argument('allow-running-insecure-content')
options.add_argument('proxy-bypass-list=<-loopback>')
options.add_argument('--proxy-server=http://127.0.0.1:8090')
options.add_argument('--proxy-auto-detect')

driver = webdriver.Chrome(chrome_options=options)

driver.get("https://www.google.com")
page_source = driver.page_source.encode('utf-8')
logger.info("Page-source -> {}".format(page_source))

Any help or suggestions would be much appreciated!

Thanks,

V

Anisha Narang

unread,

Jul 16, 2020, 12:22:37 AM7/16/20

to Selenium Users

I am facing the same issue when trying to run the tests with proxy inside a docker container. any resolution to this problem yet? The tests run fine with proxy on my local machine but when I run them inside a docker container it shows a blank screen.

thanks.

bharath kumar

unread,

Jul 16, 2020, 9:07:20 PM7/16/20

to seleniu...@googlegroups.com

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/19ad43f6-8ab5-42b2-8ade-b02e5b16ecf7o%40googlegroups.com.

Reply all

Reply to author

Forward