I am working on project and using abot crawler the website. suppose if i have url like "www.abcd.com/xyz
" . This page contains thousand of links. I just to crawl the link which conatin "www.abcd.com/xyz
" as prefix and dont want to crawl the other link. so which function should crawling to determine whether to crawl this page or not.