How to stop the AhrefsBot bot crawl ?
I set in robots.txt ,
user-agent: AhrefsBot
disallow: /
already ,and block ip in dos.yaml
blacklist:
- subnet:
188.165.15.238
but , not work .still crawl ,
How can i do ? Anyone have any other suggestions or not.
Thank you.