Avoid getting CAPTCHA's on a bot-sensitive website

[Jack Zhang]

I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.

Is there any way for me to make selenium more human-like so I don't trigger CAPTCHA's?

I am using the ChromeDriver

[Joe Ward]

Well it's not wrong, is it? You are technically a bot in the situation you describe. Many websites introduce CAPTCHAs to avoid this type of scraping. If you are running some sort of automated test I would recommend that you take the advice in this link https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do. 

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f5ee84b4-a4a5-4204-b12b-0f781105f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Naruto Lee]

​i think u need ​be modify to ChromeDriver features. however  selenium have a features. u can enter 'navigator.webdriver' on Chrome Console , u will see that return 'true', in fact. chrome be without this field. u can use 'Object.defineProperty()' to be falsify that and it should be return 'undefined'.

By: Naruto Lee

2018-05-27 21:46 GMT+08:00 Joe Ward <ddl...@gmail.com>:

Well it's not wrong, is it? You are technically a bot in the situation you describe. Many websites introduce CAPTCHAs to avoid this type of scraping. If you are running some sort of automated test I would recommend that you take the advice in this link https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha-v2-what-should-i-do. 

On 25 May 2018 at 20:30, Jack Zhang <valac...@gmail.com> wrote:

I'm trying to scrape a login-only, bot-sensitive website. After logging in, when I perform a simple selenium function like driver.find_element_by_id('button').click(), the website displays a message along the lines of We think you are a bot. Please complete the CAPTCHA below to continue.

Is there any way for me to make selenium more human-like so I don't trigger CAPTCHA's?

I am using the ChromeDriver

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.

To post to this group, send email to seleniu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/f5ee84b4-a4a5-4204-b12b-0f781105f0ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-users+unsubscribe@googlegroups.com.
To post to this group, send email to selenium-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/CAJPyTs02AKk5xMZ-mif00VXiGD1uD-hYvqd-uv_P%2BDXjkzsFTQ%40mail.gmail.com.