still seeing AhrefsBot and SemrushBot requests coming in

41 views
Skip to first unread message

David Blado

unread,
Apr 26, 2017, 12:00:19 PM4/26/17
to Nginx Ultimate Bad Bot Blocker
I am still seeing AhrefsBot requests coming in. I haven't added AhrefsBot to my robots file. I thought these conf additions would take care of it?

[26/Apr/2017:15:56:26 +0000] "GET /vendor/abx-engineering-inc/reviews/ HTTP/1.1" 444 0 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)" "164.132.161.58"

[26/Apr/2017:15:58:33 +0000] "GET /locations/indio-ca/pallets/ HTTP/1.1" 444 0 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)" "46.229.168.70"

I tried using the scripts to do the install, but got errors:

$ sudo ~/install-ngxblocker 

** Dry Run ** | -x or --exec to download files

awk: {$1="",$2=""; print}
awk:       ^ syntax error
Nothing to update for directory: /etc/nginx/conf.d
awk: {$1="",$2=""; print}
awk:       ^ syntax error
Nothing to update for directory: /etc/nginx/bots.d


I went through the manual installation.

David Blado

unread,
Apr 26, 2017, 12:01:23 PM4/26/17
to Nginx Ultimate Bad Bot Blocker
Guessing the best place to post this really was to github issue.
Reply all
Reply to author
Forward
0 new messages