Getting Page Data when Parallel Crawling

Skip to first unread message

Jun 10, 2021, 4:24:15 PM6/10/21
to Abot Web Crawler
I'm using the current version of Abot / AbotX (2.1.12) and using the demo code.  I see how to 'see' the individual page data when I run any of the demos other than 'DemoParallelCrawlerEngine()' by using the PageCrawlCompleted event handler.  But when doing DemoParallelCrawlerEngine(), I don't see how to get individual page data - the only events getting handled here are SiteCrawlStarting and SiteCrawlCompleted.  I would expect to find page data in the 'sender' of SiteCrawlCompleted, but I don't see any page data here.  Am I missing something obvious?


Rob Jones

Jun 23, 2021, 8:13:17 PM6/23/21
to, Abot Web Crawler

There is a clear example in the docs here under ParallelCrawlerEngine. See the code snippet with the comment "//Register for crawler level events. These are Abot's events!!!"

You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit
Reply all
Reply to author
0 new messages