Anti-bot code
32 views
Skip to first unread message
Clodoaldo Neto
Oct 7, 2011, 7:19:06 AM10/7/11
to foldingh...@googlegroups.com
Following a suggestion from Bas in a previous email I wrote anti bot code for the wiki. It is installed since the 3th October afternoon. Since then there were only two registrations. One confirmed as spammer and the other suspicious. There was another one (Falcon) which was myself testing.
From the log analysis it looks like those registrations were made by humans! As 1) the request times are not fast as a robot would do; and 2) the clients asks for all the page elements like css and js like a normal browser would do.
The code is an initial approach and I have a couple of ideas about how to improve it, including how to make it harder for a human spammer.
I don't know if I should post the code as that would help a robot writer. But if someone is interested I will send it by email.
Clodoaldo
From the log analysis it looks like those registrations were made by humans! As 1) the request times are not fast as a robot would do; and 2) the clients asks for all the page elements like css and js like a normal browser would do.
The code is an initial approach and I have a couple of ideas about how to improve it, including how to make it harder for a human spammer.
I don't know if I should post the code as that would help a robot writer. But if someone is interested I will send it by email.
Clodoaldo
Bas Couwenberg
Oct 7, 2011, 7:38:09 AM10/7/11
to foldingh...@googlegroups.com
Nice work, many thanks!
Let's see what the other accounts will do now, a dormant spam account
(Gasanbieter) was used today which was registered September 24th.
Hopefully these will be the only source of spam now, as they are
quickly purged :)
--
Disclaimer: Any errors in spelling, tact, or fact are transmission errors.
Clodoaldo Neto
Oct 8, 2011, 12:22:32 PM10/8/11
to foldingh...@googlegroups.com
2011/10/7 Bas Couwenberg <linux...@gmail.com>
Nice work, many thanks!
Let's see what the other accounts will do now, a dormant spam account
(Gasanbieter) was used today which was registered September 24th.
Hopefully these will be the only source of spam now, as they are
quickly purged :)
This last one (Ryan46boyle) acts very much like a humam. I will have to implement the tricks I have been thinking about but I will not have much time as I'm back to full speed at work with a new project and this weekend I'm busy with family and friends.
Clodoaldo
173.208.87.10 - - [08/Oct/2011:13:49:23 +0000] "GET /index.php/Custom_black_list HTTP/1.1" 200 4508 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:26 +0000] "GET /skins/common/shared.css?301 HTTP/1.1" 200 17078 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:26 +0000] "GET /skins/modern/main.css?301 HTTP/1.1" 200 13663 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:27 +0000] "GET /skins/modern/print.css?301 HTTP/1.1" 200 110 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:27 +0000] "GET /load.php?debug=false&lang=en&modules=startup&only=scripts&skin=modern&* HTTP/1.1" 200 1829 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:31 +0000] "GET /skins/modern/footer-grad.png HTTP/1.1" 200 141 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:31 +0000] "GET /skins/modern/bullet.gif HTTP/1.1" 200 50 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:32 +0000] "GET /load.php?debug=false&lang=en&modules=jquery%7Cmediawiki&only=scripts&skin=modern&version=20110608T121000Z HTTP/1.1" 200 32456 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:33 +0000] "GET /load.php?debug=false&lang=en&modules=jquery.checkboxShiftClick%2Cclient%2Ccookie%2Cplaceholder%7Cmediawiki.language%2Cutil%7Cmediawiki.legacy.ajax%2Cwikibits&skin=modern&version=NaNNaNNaNTNaNNaNNaNZ HTTP/1.1" 200 10414 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:34 +0000] "GET /index.php?title=Special:UserLogin&returnto=Custom_black_list HTTP/1.1" 200 3435 "http://fahwiki.net/index.php/Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:39 +0000] "GET /index.php?title=Special:UserLogin&type=signup&returnto=Custom_black_list HTTP/1.1" 200 3757 "http://fahwiki.net/index.php?title=Special:UserLogin&returnto=Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:53 +0000] "POST /index.php?title=Special:UserLogin&action=submitlogin&type=signup&returnto=Custom_black_list HTTP/1.1" 200 3321 "http://fahwiki.net/index.php?title=Special:UserLogin&type=signup&returnto=Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
173.208.87.10 - - [08/Oct/2011:13:49:54 +0000] "GET /load.php?debug=false&lang=en&modules=jquery.checkboxShiftClick%2Cclient%2Ccookie%2Cplaceholder%7Cmediawiki.language%2Cutil%7Cmediawiki.legacy.ajax%2Cajaxwatch%2Cwikibits&skin=modern&version=NaNNaNNaNTNaNNaNNaNZ HTTP/1.1" 200 11150 "http://fahwiki.net/index.php?title=Special:UserLogin&action=submitlogin&type=signup&returnto=Custom_black_list" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)"
linux...@gmail.com
May 25, 2013, 12:32:49 PM5/25/13
to foldingh...@googlegroups.com, clodoaldo....@gmail.com
I got tired of manually reverting the talk page spam, so I wrote a script to automate it :-)
The script takes a list of usernames, deletes their talk page and blocks the user indefinitely.
$ ./fahwiki-revert-spam.pl --help
Usage: fahwiki-revert-spam.pl [OPTIONS]
-b, --base-url <URL> Base URL to wiki (http://fahwiki.net/)
-u, --username <NAME> Wiki login username ()
-p, --password <PASS> Wiki login password ()
-s, --spammer <NAME>, [NAME] Usernames of spammers ()
-f, --file <PATH> Path to spammers file ()
-v, --verbose Enable verbose output
-h, --help Display this usage information
Actions:
-T, --delete-talk Delete talk page for user
-B, --block-user Block the user
All actions are enabled by default, use --no-<action> to disable
You'll be prompted for the required options if they're not specified with commandline arguments. Specifying a file, or one or more spammers will bypass prompting for it.
The script requires some perl modules:
To quickly get a list of usernames I copy/pasted the RecentChanges page to a text file out of which I filtered all users who created a talk page with:
cat /tmp/wiki-spammers.raw | perl -e 'while(<>){ if(/^\s+\(diff.*N\! User talk:(\S+?); \d+:\d+/){ print "$1\n"; } }' >> /tmp/wiki-spammers
I'm attaching it so others can use it too.
Regards,
Bas
The script takes a list of usernames, deletes their talk page and blocks the user indefinitely.
$ ./fahwiki-revert-spam.pl --help
Usage: fahwiki-revert-spam.pl [OPTIONS]
-b, --base-url <URL> Base URL to wiki (http://fahwiki.net/)
-u, --username <NAME> Wiki login username ()
-p, --password <PASS> Wiki login password ()
-s, --spammer <NAME>, [NAME] Usernames of spammers ()
-f, --file <PATH> Path to spammers file ()
-v, --verbose Enable verbose output
-h, --help Display this usage information
Actions:
-T, --delete-talk Delete talk page for user
-B, --block-user Block the user
All actions are enabled by default, use --no-<action> to disable
You'll be prompted for the required options if they're not specified with commandline arguments. Specifying a file, or one or more spammers will bypass prompting for it.
The script requires some perl modules:
- File::Slurp
- Term::Prompt
- WWW::Mechanize
- URI::Escape
To quickly get a list of usernames I copy/pasted the RecentChanges page to a text file out of which I filtered all users who created a talk page with:
cat /tmp/wiki-spammers.raw | perl -e 'while(<>){ if(/^\s+\(diff.*N\! User talk:(\S+?); \d+:\d+/){ print "$1\n"; } }' >> /tmp/wiki-spammers
I'm attaching it so others can use it too.
Regards,
Bas
Bas Couwenberg
Jul 13, 2013, 2:10:06 PM7/13/13
to foldingh...@googlegroups.com
I've been using an updated version of the script, which gets the list of spammer by checking the newusers for recently created talk pages.
Just run the script with -t/--new-talk and be done, just make sure to check for false positives by viewing the page yourself first.
Usage: fahwiki-revert-spam.pl [OPTIONS]
Options:
Just run the script with -t/--new-talk and be done, just make sure to check for false positives by viewing the page yourself first.
Usage: fahwiki-revert-spam.pl [OPTIONS]
Options:
-b, --base-url <URL> Base URL to wiki (http://fahwiki.net/)
-u, --username <NAME> Wiki login username ()
-p, --password <PASS> Wiki login password ()
-s, --spammer <NAME>, [NAME] Usernames of spammers ()
-f, --file <PATH> Path to spammers file ()
-t, --new-talk Get spammers from new talk pages
-v, --verbose Enable verbose output
-h, --help Display this usage information
Actions:
-T, --delete-talk Delete talk page for user
-B, --block-user Block the user
All actions are enabled by default, use --no-<action> to disable
--
---
You received this message because you are subscribed to the Google Groups "Folding@Home Wiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to foldinghome-wi...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Clodoaldo Neto
Jul 13, 2013, 3:18:28 PM7/13/13
to Folding@Home Wiki
Bas,
Thank you very much for your dedication to the wiki. I hope all the best for you.linux...@gmail.com
Jul 13, 2013, 9:08:28 PM7/13/13
to foldingh...@googlegroups.com, clodoaldo....@gmail.com
You're welcome. As mentioned in the "Spam in FahWiki.net" thread, it's not much trouble. I'm doing fine, and happy to see that the script won't be needed anymore.
Regards,
Bas
Op zaterdag 13 juli 2013 21:18:28 UTC+2 schreef Clodoaldo het volgende:
Regards,
Bas
Op zaterdag 13 juli 2013 21:18:28 UTC+2 schreef Clodoaldo het volgende:
Reply all
Reply to author
Forward
0 new messages