Crawl Patterns

8 views
Skip to first unread message

Robin

unread,
May 23, 2012, 2:30:43 AM5/23/12
to Google Search Appliance/Google Mini - Google Search Appliance/Google Mini, robin_...@infosys.com
Hi,

I have got the following organization of URLs

1. There is a URL http://www.enterprise.com/ which has got 2 links
inside it http://www.enterprise.com/A and http://www.enterprise.com/B.
2. http://www.enterprise.com/A has got one link inside it
http://www.enterprise.com/A/1.
3. http://www.enterprise.com/B has got one link inside it
http://www.enterprise.com/A/2.

Initially, I had configured the Crawl URL pattern as
http://www.enterprise.com/
and hence the no. of Crawled URLs is 5
If I change the crawl pattern to http://www.enterprise.com/A, then
will the no .of Crawled URLs reduce to
3 - http://www.enterprise.com/A, http://www.enterprise.com/A/1,
http://www.enterprise.com/A/2
or 2 - http://www.enterprise.com/A, http://www.enterprise.com/A/1.

Thanks and Regards,
Robin

Dave Watts

unread,
May 23, 2012, 11:00:52 AM5/23/12
to google-search-...@googlegroups.com
The appliance just follows the links it's given, in the same way that
you could with a browser. So, with the constraints you've specified,
it would only find the two links /A and /A/1.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
http://training.figleaf.com/

Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on
GSA Schedule, and provides the highest caliber vendor-authorized
instruction at our training centers, online, or onsite.
Reply all
Reply to author
Forward
0 new messages