Web scraping with Selenium on a 2FA log in site

[Tobias Andersson]

Is is possible to use Selenium to log in to a site that uses a 2 factor autentication method (e.g bankID for a bank)?

If not, is it possible to web scrape after you have manually logged in to the 2FA log in site?

[ddlionx]

depends on the t&c of what you're scraping. i sincerely doubt it's allowed

if it's a test version of some website you have permission to be using you can undoubtedly ask for some backdoor for testability

On Wed, 21 Oct 2020 at 16:50, Tobias Andersson <ta.tobias...@gmail.com> wrote:

Is is possible to use Selenium to log in to a site that uses a 2 factor autentication method (e.g bankID for a bank)?

If not, is it possible to web scrape after you have manually logged in to the 2FA log in site?

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/selenium-users/535cf7f8-e42f-49c7-af83-a5c420dff9a4n%40googlegroups.com.

[Joel]

>>is it possible to web scrape after you have manually logged in

You can easily find out. Authenticate manually. Copy a link that requires authentication to access.  Open up a new browser window and try to navigate to the saved link. If it goes to the link without forcing you to authenticate again, then you can do the same with Selenium.   

[Tobias Andersson]

 "Open up a new browser window and try to navigate to the saved link. If it goes to the link without forcing you to authenticate again, then you can do the same with Selenium." It works to open up a new browser and navigate to a saved link without logging in again. 

I want to scrape pages from a users overview page. That is every user has a unique id of 5 digits in the web address to navigate to that users overview. The problem I guess is that I need to put this address to Selenium to web scrape that page? It will be exhausting to put the unique id of every user into selenium. Do you know of any other more effective way?