Uri Uniqueness with trailing slash

8 views

Skip to first unread message

Jeff Richmond

unread,

May 9, 2024, 10:51:13 PM5/9/24

to Abot Web Crawler

I have noticed that URIs with no trailing slash and the same URI with a trailing slash are not considered the same and thus in our usage of this library we are getting duplicate pages, is there anyway to fix that?

Reply all

Reply to author

Forward

0 new messages