Getting Page Data when Parallel Crawling

42 views
Skip to first unread message

rjone...@gmail.com

unread,
Jun 10, 2021, 4:24:15 PM6/10/21
to Abot Web Crawler
I'm using the current version of Abot / AbotX (2.1.12) and using the demo code.  I see how to 'see' the individual page data when I run any of the demos other than 'DemoParallelCrawlerEngine()' by using the PageCrawlCompleted event handler.  But when doing DemoParallelCrawlerEngine(), I don't see how to get individual page data - the only events getting handled here are SiteCrawlStarting and SiteCrawlCompleted.  I would expect to find page data in the 'sender' of SiteCrawlCompleted, but I don't see any page data here.  Am I missing something obvious?

Thanks,

Rob Jones

sjdi...@gmail.com

unread,
Jun 23, 2021, 8:13:17 PM6/23/21
to rjone...@gmail.com, Abot Web Crawler
Yes, 

There is a clear example in the docs here under ParallelCrawlerEngine. See the code snippet with the comment "//Register for crawler level events. These are Abot's events!!!"

--
You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abot-web-crawl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abot-web-crawler/6ead0649-95fd-4182-8125-3e2b5b2f2a9en%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages