Force Selenium Chrome Driver to use QUIC

[Siva Kesava]

Hi all, 

        I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.

Code Snippet: 

                chrome_options = webdriver.ChromeOptions()

        chrome_options.add_argument("--proxy-server={0}".format(url))

chrome_options.add_argument("--enable-quic")

        self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)

               self.proxy.new_har(args['url'], options={'captureHeaders': True})

       self.driver.get(args['url'])

               result = json.dumps(self.proxy.har, ensure_ascii=False)

I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only . Is there a way to force Chrome Driver to use QUIC?

[Siva Kesava]

Anyone having an idea please comment.

[David]

Interesting use case you have, and I learned about QUIC from your post.

Is there a ChromeDriver discussion group / mailing list? I'd try posting there. And/or filing an issue to ChromeDriver project. This might be too new for anyone in Selenium community to know about and use.

[Siva Kesava]

Yeah this is an interesting project we are currently working on. I am trying to analyse QUIC over DASH(youtube) and in order to do so I need HAR . To download HAR manually is a hectic task and I am trying to automate this with BrowserMobproxy(BMP) and Selenium but I could not understand why QUIC is not used and is it because of BMP or Selenium. I have posted it to chrome driver group also.

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/8CDPCBRCmDE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/1363f220-aeff-4f89-878d-3b03707cb7e2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[David]

well, if you're using wireshark to also do some quick validation, then you can facto the BMP part out. Simply use Selenium ChromeDriver w/o BMP, and sniff traffic with Wireshark to confirm you get QUIC. If you do, then try with BMP and likely BMP issue. If you don't, then it's already issue on ChromeDriver side.

On Thursday, October 27, 2016 at 11:31:03 AM UTC-7, Siva Kesava wrote:

Yeah this is an interesting project we are currently working on. I am trying to analyse QUIC over DASH(youtube) and in order to do so I need HAR . To download HAR manually is a hectic task and I am trying to automate this with BrowserMobproxy(BMP) and Selenium but I could not understand why QUIC is not used and is it because of BMP or Selenium. I have posted it to chrome driver group also.

On Thu, Oct 27, 2016 at 11:36 PM, David <mang...@gmail.com> wrote:

Interesting use case you have, and I learned about QUIC from your post.

Is there a ChromeDriver discussion group / mailing list? I'd try posting there. And/or filing an issue to ChromeDriver project. This might be too new for anyone in Selenium community to know about and use.

On Thursday, October 27, 2016 at 4:09:13 AM UTC-7, Siva Kesava wrote:

Anyone having an idea please comment.

On Wednesday, October 26, 2016 at 10:19:17 AM UTC+5:30, Siva Kesava wrote:

Hi all, 

        I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.

Code Snippet: 

                chrome_options = webdriver.ChromeOptions()

        chrome_options.add_argument("--proxy-server={0}".format(url))

chrome_options.add_argument("--enable-quic")

        self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)

               self.proxy.new_har(args['url'], options={'captureHeaders': True})

       self.driver.get(args['url'])

               result = json.dumps(self.proxy.har, ensure_ascii=False)

I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only . Is there a way to force Chrome Driver to use QUIC?

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/8CDPCBRCmDE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

[Siva Kesava]

How to work with Selenium ChromeDriver alone without BMP. I require BMP as a server to which Chrome Driver contacts and open the Url. Can you suggest how to open a browser with Selenium alone?

On Wednesday, October 26, 2016 at 10:19:17 AM UTC+5:30, Siva Kesava wrote:

[Siva Kesava]

What I mean is how can we download HAR without BMP. Can selenium alone do it?

 Sent with Mailtrack

ᐧ

--

You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/8CDPCBRCmDE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/a9674229-a4bf-471f-a01c-0c842101f361%40googlegroups.com.

[David]

Selenium or ChromeDriver itself can't deal with HAR files in terms of an API. But if you do some steps manually, you can however use browser extensions/addons for FF or Chrome that let you save/export/etc. HAR files, without a proxy like BMP.

Not sure if BMP is the only proxy that can deal with / capture HAR files. But it is pretty much the de facto one to use with Selenium in code. For outside of code, you have other options like Fiddler, Charles Proxy, and simply have the browser traffic routed through those proxies (via OS network config).

On Saturday, October 29, 2016 at 12:46:48 PM UTC-7, Siva Kesava wrote:

What I mean is how can we download HAR without BMP. Can selenium alone do it?

 Sent with Mailtrack

ᐧ

On Sun, Oct 30, 2016 at 12:32 AM, Siva Kesava <sivak...@gmail.com> wrote:

How to work with Selenium ChromeDriver alone without BMP. I require BMP as a server to which Chrome Driver contacts and open the Url. Can you suggest how to open a browser with Selenium alone?

On Wednesday, October 26, 2016 at 10:19:17 AM UTC+5:30, Siva Kesava wrote:

Hi all, 

        I am working on downloading HAR from Chrome for YouTube through Selenium Python Script.

Code Snippet: 

                chrome_options = webdriver.ChromeOptions()

        chrome_options.add_argument("--proxy-server={0}".format(url))

chrome_options.add_argument("--enable-quic")

        self.driver = webdriver.Chrome(chromedriver,chrome_options = chrome_options)

               self.proxy.new_har(args['url'], options={'captureHeaders': True})

       self.driver.get(args['url'])

               result = json.dumps(self.proxy.har, ensure_ascii=False)

I want QUIC to be used whenever I download HAR but when I look at the packets through Wireshark Selenium driver is using TCP only . Is there a way to force Chrome Driver to use QUIC?

--
You received this message because you are subscribed to a topic in the Google Groups "Selenium Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/selenium-users/8CDPCBRCmDE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to selenium-user...@googlegroups.com.
To post to this group, send email to seleniu...@googlegroups.com.

[Siva Kesava]

I have checked that Selenium Chrome Driver alone allows QUIC to be used but when BMP is ued it falls back to TCP.So I need a mechanism like that of Firefox  as mentioned in this post  http://stackoverflow.com/questions/10755480/how-to-capture-all-requests-made-by-page-in-webdriver-is-there-any-alternative/40322699#40322699 for chrome also. So if anyone has an idea on how to automate the HAR download for Chrome also as in firefox in the above post Please comment.

 Sent with Mailtrack

ᐧ

To unsubscribe from this group and all its topics, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/31ea07fb-f82a-47a3-9b75-5ba49ec6f472%40googlegroups.com.

[Siva Kesava]

Is there any alternative to BMP that can be used with Selenium so that HAR download can be done through code?

[David]

Sounds like I'm guessing that Chrome will either:

• fall back to TCP when running over a proxy for networking (you can verify this by using any other proxy - Fiddler, Charles proxy, etc.)
• fall back to TCP if the proxy used does not support QUIC (which we can assume BMP does not support, can confirm this if you find another proxy that supports QUIC)

Unfortunately though, I don't think there's any other proxy besides BMP that gives you Selenium integration. Maybe can try Fiddler, it has an API (.NET based?) but not sure if it supports HAR export.

And with browser plugins, you can't really invoke code/API to save/download a HAR, it's more of a manual step.