Re: CrawlResult not returned if MaxPagesToCrawl is set ?

11 views
Skip to first unread message

sjdi...@gmail.com

unread,
Feb 13, 2021, 7:36:27 PM2/13/21
to John Ligtenberg, Abot Web Crawler
If you are asking where are the crawled pages then you need to subscribe to the PageCrawlCompleted event. If you get a Crawl result it means the crawl is done. 

Hope that helps

On Wed, Feb 10, 2021, 10:11 AM John Ligtenberg <john.li...@indivirtual.com> wrote:
Hello,

I've set the MaxPagesToCrawl to 10.

Crawling is working, I never get passed

var result = await crawler.CrawlAsync(new Uri(startPage));

Is there no CrawlResult returned if MaxPagesToCrawl is set?

Is there any other way to determine if the entire crawl process is finished?

Yours,

John Ligtenberg

--
You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abot-web-crawl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abot-web-crawler/62ccc887-187c-465c-91da-968da0e16f56n%40googlegroups.com.

John Ligtenberg

unread,
Feb 16, 2021, 3:20:56 AM2/16/21
to Abot Web Crawler

This problem has been solved, it was caused by a locked session.

I needed to add

[SessionState(SessionStateBehavior.ReadOnly)]

to my controller, to be able to crawl the pages. 

Reply all
Reply to author
Forward
0 new messages