Blocking spam signups

[Laurent Savaëte]

Hi everyone,

Over the past 10 days, we've been getting "attacked" by spammers (spam bots?) that are registering spam accounts with emails in various *.ru domains. (I guess we had been lucky enough to stay under their radar so far) The account name (in Russian) translates to various click baity links. I've been banning/renaming those by hand, but we're getting close to a hundred such accounts in that short period of time, and I'd rather spend my time on something more productive. I activated the spam protection measures in config, but after a few days, I must say they don't seem very effective. Probably because the UserSpamScorer class seems to search for signs of spam in English, so cyrillic spam might be getting through?

Is there any way to completely ban signups from certain email domains?

Is there a way I could add a list of words in config to block account names that include "congrats" in Russian or others that I've seen? "https://" in the username should also be a clear spam sign :)

Thanks for any suggestions!

Laurent for madada.fr

[Oliver Lineham]

A hundred spam accounts in ten days sounds wonderful, I wish that were the case here in NZ.

I've had a few battles with spam signups. There have been two types:

1. The biggest problem was bots signing up 60-80 accounts per hour. In the worst run, 7000 accounts were signed up over 3 days. They used real people's real email addresses, so could never activate the accounts to put spam in their profiles. Even so, these are the worst kind of signups as huge numbers of victims receive activation emails. Some of those people report the emails as spam, ruining our email reputation and getting our sending IPs on DNS blocklists. Then, mail to authorities and real users start bouncing. Disaster!

Eventually my solution to this was to use some IP/subnet lists (from fireHOL) to block all anonymous proxies and known compromised IPs. Sorry to anyone who wanted to use a public VPN or Tor with our service.

I'm not sure if picture will make it through to the google group, but here's signups per hour by the bots, until I blocked all anonymous proxies (the gaps was when I blocked particular proxies, until they switched to new ones):

2. A smaller problem has been spam sign-ups where the spammers do confirm the email and then post spammy profile information. I've found a couple of businesses in India and Vietnam selling this as a service, to provide SEO link juice to their customers. The rate is much lower - an account every few minutes - and is is more annoying than harmful. After observing the traffic, times of day, and toying with them in real time (changing the profile information they entered, for example) I've concluded these aren't bots but people paid to do this as a job.

I have blocked access to setting profile information, leaving the name the only opportunity for spam - and that doesn't give them SEO "link juice". So most of them have given up. Sorry to anyone who wanted to customise their profile on our service.

With all of that, we're now down to about 3-5 spam accounts per hour.

It's unfortunate the Alaveteli "deleting a user" instructions are so impractical. I also fear at some point these non-bot spammers will start posting actual FOI requests. If that happens, it'll probably be game over for our site.

Oliver

--
You received this message because you are subscribed to the Google Groups "Alaveteli Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alaveteli-dev/daf3c8aa-0b7b-8a1b-8a5e-997069838cb7%40gmail.com.

[Natalka Onyshchenko]

We (Ukrainian instance) had a horrible spam infestation last year, spam accounts sent FOI requests to authorities who were very pissed at us as a result.

I ended up simply modifying alaveteli_spam_checker.rb file to include relevant Cyrillic spam terms (we're using a fork of Mysociety repository anyway). It worked. The unfortunate side effect is that sometimes a valid request does not go through because it uses a suspicious word in a legit context but that's rare.

And we use fireHOL lists as well.

[Gareth Rees]

Hi all,

This is a problem we suffer on WhatDoTheyKnow, too.

> They used real people's real email addresses… huge numbers of victims receive activation emails

Ugh! I'm not sure we've seen this on a large scale (though have on other services we run).

> Sorry to anyone who wanted to use a public VPN or Tor with our service.

We block signups from Tor by redirecting to a specific page built in to Alaveteli [1] (/tor) to at least give some route for genuine users to contact us and figure out a workaround. We haven't received any contacts yet.

> I have blocked access to setting profile information… I've concluded these aren't bots but people paid to do this as a job.

In 0.41.0.0 we reduced the prominence of the about_me text so that it's only visible while logged in [2].

We also reached the same conclusion that these are humans.

> It's unfortunate the Alaveteli "deleting a user" instructions are so impractical.

We've added a "ban for spamming" quick action to user admin pages [3]. This completely prevents about me text being shown. I guess a good improvement would be to also hide the user name of banned users when viewing not logged in.

> I also fear at some point these non-bot spammers will start posting actual FOI requests

We have had a handful of instances of this – I think in ~2017 where we had spammers posting links to video download services. I'll try to dig out our notes about how we handled if its of interest. We probably just blocked a range of IPs at country level and then hid/destroyed the requests.

> the UserSpamScorer class seems to search for signs of spam in English, so cyrillic spam might be getting through?

It does increase the spam score if non-alpha characters are included in the name [4].

> Is there any way to completely ban signups from certain email domains?

We did start a list of suspicious domains [5], but as you can imagine it's a bit of a losing battle.

BLOCK_SPAM_SIGNUPS needs to be set to true (or ENABLE_ANTI_SPAM to true to enable every spam check) to actually block (not just alert) potential spam signups.

Might need to tweak the UserSpamScorer scoring config that's used on signup [7].

> Is there a way I could add a list of words in config to block account names that include "congrats" in Russian or others that I've seen? "https://" in the username should also be a clear spam sign :)

Would need to add to DEFAULT_SPAM_NAME_FORMATS [8]. 

It's been a while since I looked at this code, but at a quick glance I think you can add to these in the theme with something like:

    UserSpamScorer.spam_name_formats = UserSpamScorer::DEFAULT_SPAM_NAME_FORMATS + [/my/, /array.*of/, /custom/, /regexps/]

If not, then that's definitely something we should enable!

Best,

Gareth

[1] https://github.com/mysociety/alaveteli/pull/6666

[2] https://github.com/mysociety/alaveteli/pull/6688

[3] https://github.com/mysociety/alaveteli/pull/4608

[4] https://github.com/mysociety/alaveteli/blob/develop/lib/user_spam_scorer.rb#L5

[5] https://github.com/mysociety/alaveteli/blob/develop/lib/user_spam_scorer.rb#L28

[6] https://github.com/mysociety/alaveteli/blob/develop/config/general.yml-example#L955-L960

[7] https://github.com/mysociety/alaveteli/blob/fa6f8d63913f09c5b5ccc31654cf66718ac65abf/app/controllers/concerns/user_spam_check.rb#L33-L54

[8] https://github.com/mysociety/alaveteli/blob/develop/lib/user_spam_scorer.rb#L71

[Laurent Savaëte]

Hi everyone,

Thanks all for your replies. I'm conscious that what we have here is pretty small scale, but I'd rather act before we're completely swamped.

Last week, some emails started getting blocked by microsoft, which is worrysome considering how prevalent their mail service is in the (French?) administration. While I don't think it's directly related to the spam accounts, it served as a good reminder that our site can be rendered useless quite easily.

I believe our attack was human as well, the delay between account creations was too long to be bots. I banned the accounts and renamed them to "<spammer>" and after a few days of this dance, it seems to have stopped. The "about me" field was not used in this case.

Gareth, I'll try the various options you suggested and report back.

Thanks again,

Laurent

[Gareth Rees]

I believe our attack was human as well, the delay between account creations was too long to be bots. I banned the accounts and renamed them to "<spammer>" and after a few days of this dance, it seems to have stopped. The "about me" field was not used in this case.

Oh, this reminds me of another observation.

In many cases we’ve seen accounts get created, sit dormant for months – even years – and then they come back to populate the about_me with spam. Presumably this is to get under the radar of any sign-up time detection.

Hopefully the reduced prominence of the about_me text so that it's only visible while logged in has helped reduce the appeal of this tactic.

--

Gareth Rees

[elha...@gmail.com]

Wow, as a newby on the block regarding running the platform (we've been in the air with only lower authorities for a year now) and NO technical knowledge but supported by MySociety for the time being, I just wanted to say: yes please, share your experiences and solutions in general but re this in particular and we will too. I had no idea and feel so naive. Will buckle up. Thanks!