GSA crawls login page when Form Authentication fails

10 views
Skip to first unread message

Jeroen

unread,
Jul 16, 2012, 5:42:50 AM7/16/12
to Google-Search-...@googlegroups.com
While crawling an intranet sometimes the Form Authentication fails. At that moment the GSA crawls the login form instead of the real content of the page.
Is there any solution to keep the earlier crawled content in the index and ignore the login form if it appears?

Dave Watts

unread,
Jul 16, 2012, 9:56:19 AM7/16/12
to google-search-...@googlegroups.com
Unfortunately, I don't think there is. You might try setting up a Head
Requestor Deny Rule, but I'm pretty sure that only comes into play at
serve time.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
http://training.figleaf.com/

Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on
GSA Schedule, and provides the highest caliber vendor-authorized
instruction at our training centers, online, or onsite.
Reply all
Reply to author
Forward
0 new messages