Issue with redirecting websites.

[jayant sable]

Hi all,

I wanted to headless chrome but facing following issue with redirecting websites. Like microsoft.com will redirect you to some other webpage in that case I'm getting following error message shown in screen shot. Also I'm using following command to start headless chrome.

chrome 

--headless 

--disable-gpu 

--remote-debugging-port=9222 

--remote-debugging-address=<someAddress> 

--proxy-server=<proxyServerIP&Port) 

--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.72 Safari/537.36" 

http://www.microsoft.com

Any help would be appreciated.

Regards,

Jay

[PhistucK]

First, make sure that the user agent string that you specified is indeed being sent to the server (using Fiddler, for example, or by initiating more requests somehow, say, using AJAX).

Second, their server might have some spam detector that does not detect a real human (for example, by not moving the mouse humanly) and so shows that screen. In the case, there is nothing the team can do about it, you will have to find a way to overcome your headless inhumanity (like any hacker has to ;)). ;)

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev+unsubscribe@chromium.org.
To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDo%3DTvOC3hc5XgqwOYaimRNy1VrSue0dj5T4nP7h1kEKg%40mail.gmail.com.

[jayant sable]

thanks for reply.

I tried to detect user agent using fiddler but somehow not getting the request for headless browser. All request seems to be coming from headfull chrome with it's own user agent. [I've read somewhere that user agent for headless browser will have "headless chrome" in user agent string instead of "chrome/version"] But all the request in fiddler are coming with headfull useragent.

Any other way we can detect what headless requested for with what user agent. Any other tool or way..?

Regards,

Jay

[PhistucK]

Try going to whatsmyuseragent.com and taking a screenshot, or evaluate some JavaScript that either calls console.log(navigator.userAgent) (and see it in the log) or initiates an XMLHttpRequest to your own server (and see it in its log) or something like that.

I do not know your setup or expertise, so it is not clear how to progress from here.

☆PhistucK

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDTY6fQHfd9Act-uraDqvxMtVP-EQFfeA%3D1%2Bs400kjEsg%40mail.gmail.com.

[Eric Seckler]

Headless should, by default, use a custom user agent string that contains "HeadlessChrome". The --user-agent flag should override it. If that's not the case, please file a bug with a repro-case. PhistucK is correct, though - websites may also use other means of detecting automated bots than the user agent string.

On Thu, Jul 27, 2017 at 10:55 AM jayant sable <jayan...@gmail.com> wrote:

thanks for reply.

I tried to detect user agent using fiddler but somehow not getting the request for headless browser. All request seems to be coming from headfull chrome with it's own user agent. [I've read somewhere that user agent for headless browser will have "headless chrome" in user agent string instead of "chrome/version"] But all the request in fiddler are coming with headfull useragent.

Any other way we can detect what headless requested for with what user agent. Any other tool or way..?

Regards,

Jay

On Thu, Jul 27, 2017 at 11:16 AM, PhistucK <phis...@gmail.com> wrote:

First, make sure that the user agent string that you specified is indeed being sent to the server (using Fiddler, for example, or by initiating more requests somehow, say, using AJAX).

Second, their server might have some spam detector that does not detect a real human (for example, by not moving the mouse humanly) and so shows that screen. In the case, there is nothing the team can do about it, you will have to find a way to overcome your headless inhumanity (like any hacker has to ;)). ;)

☆PhistucK

On Thu, Jul 27, 2017 at 8:26 AM, jayant sable <jayan...@gmail.com> wrote:

Hi all,

I wanted to headless chrome but facing following issue with redirecting websites. Like microsoft.com will redirect you to some other webpage in that case I'm getting following error message shown in screen shot. Also I'm using following command to start headless chrome.

chrome 

--headless 

--disable-gpu 

--remote-debugging-port=9222 

--remote-debugging-address=<someAddress> 

--proxy-server=<proxyServerIP&Port) 

--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.72 Safari/537.36" 

http://www.microsoft.com

Any help would be appreciated.

Regards,

Jay

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.

To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDo%3DTvOC3hc5XgqwOYaimRNy1VrSue0dj5T4nP7h1kEKg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.

To post to this group, send email to headle...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDTY6fQHfd9Act-uraDqvxMtVP-EQFfeA%3D1%2Bs400kjEsg%40mail.gmail.com.

[jayant sable]

Thanks for reply guys..!!

Facing one more issue with downloading files through headless chrome. Is it not supported by headless chrome?

I'm not able to download file. I've automated using selenium and once I click on download button, it is not downloading the file.

I'm using following preferences. [ tools = headless chrome,python & Selenium. ]

options.add_experimental_option('prefs', {

  'download.default_directory': 'C:\\pySe\\',

  'download.prompt_for_download': False,

  'download.directory_upgrade': True,

  'safebrowsing.enabled': True

})

options is instance of webdriver.ChromeOptions() which I'm passing to Chrome().

Regards,

Jayant

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev+unsubscribe@chromium.org.

To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDo%3DTvOC3hc5XgqwOYaimRNy1VrSue0dj5T4nP7h1kEKg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev+unsubscribe@chromium.org.

To post to this group, send email to headle...@chromium.org.

[Eric Seckler]

We're working on download support, see this bug :)

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.

To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDo%3DTvOC3hc5XgqwOYaimRNy1VrSue0dj5T4nP7h1kEKg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.

To post to this group, send email to headle...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDTY6fQHfd9Act-uraDqvxMtVP-EQFfeA%3D1%2Bs400kjEsg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.

To post to this group, send email to headle...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/CANeHHRDPQxyeVhHvi-2ghWf%2B60htJ1HM%2BmwrR6dcCn41nwbgvg%40mail.gmail.com.