Login and crawl a website

[Kyle Tice]

I've seen a few discussions on this topic but i am not clear if abot2 supports logging into a web application or not.  I do see items in the CrawlConfiguration for LoginUser, LoginPassword, IsAlwaysLogin but setting values here is not working.  Has anyone been successful with using this or is the course of action to Override the PageRequestor method?

[sjdi...@gmail.com]

Hi Kyle,

The settings you are referring to are related only to basic http authentication. AbotX offers no additional login features. You are correct to look toward building a custom PageRequestor implementation which I believe several people have done but don't think I've seen anyone share a common example. Logging in to a site is very difficult to abstract into a canned feature given the wide variability of each site's login process and I assume this is the reason no one has shared a sample.

I hope that helps 

On Wed, Nov 6, 2019 at 1:18 PM Kyle Tice <avz...@gmail.com> wrote:

I've seen a few discussions on this topic but i am not clear if abot2 supports logging into a web application or not.  I do see items in the CrawlConfiguration for LoginUser, LoginPassword, IsAlwaysLogin but setting values here is not working.  Has anyone been successful with using this or is the course of action to Override the PageRequestor method?

--
You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abot-web-crawl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abot-web-crawler/c65c0f3a-d358-4f08-9a7e-99cb26b0a6a5%40googlegroups.com.