Wiki availability

[Peter Hicks]

All,

The Open Rail Data wiki is currently under heavy load - there are numerous individual IP addresses scraping the site and requesting content which has a tangible resource overhead to produce such as page diffs.  As a result, the server it's running on is consuming all its available CPU even when I allocate more.

I put in place rules to block much of this dynamic content yesterday, using a mix of AWS load balancer rules and in-app configuration, but it appears to have started up again from around 0530 this morning.

If the heavy load doesn't disappear over the next 12 or so hours, I'll look and see what other protections I can put in place.  Of course, if anyone from NR or RDG would like to contribute to hosting costs (as I've always run the site for free) with any spare budget they have, please get in touch separately.

Peter

[Marsh Lane]

HI Peter,

Re: The Wiki

Can I suggest signing up for Cloudflare's free option and turning Attack Mode on.  We found that with a couple of our sites, it was easily managed (could be switched off after a couple of days when it calms down and then switched back on with a mouse click if needed) and reduced the 'hit' the site was taking by about 95%.

Richard

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/-BCINMJZQ-NEXRFhTnD69n0Oj7BRFExIUVkZPmrqywqDOcAluLcnFuhFj6NUsjpFncX5GfZv0wz0aadOb4IhmMz6vMm2C_TlCoi_fqgYVMM%3D%40poggs.co.uk.

[russell pirie]

100% put behind Cloudflare and can add the "are you human" check, should remove most bots.

[Evelyn Snow]

I've heard some good things about iocaine, and I think it might be a bit less obtrusive for normal
users than cloudflare or anubis.

Evelyn

[Peter Hicks]

Thanks both - I’ll give it a go and see if I can neatly fudge the existing AWS architecture to work with Cloudflare.

Sent from Proton Mail for iOS.

-------- Original Message --------

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/e15ef2b6-b63e-4607-bd28-ad760fc76adan%40googlegroups.com.

[Adrian Bradshaw]

Media Wiki seems to be something of a target these days. We use it for our Help pages on the Railcam UK website and had the website taken down by the same sort of attack that you are seeing.

We "fixed" it by moving the Wiki off the main server and dropping it on some shared web-hosting. That only worked for us because the address changed - which didn't matter in our case - I just adjusted the links in the Help menu. Of course, when the bots find the new location, we may have the same problem, but at least it won't take the rest of the website with it!

Adrian

---

Adrian Bradshaw – Director

Railcam UK Limited

@: adr...@railcam.uk

Railcam.uk – Bringing The Lineside To Your Armchair

Railcam UK Limited | Registered in England and Wales | registration number 10167844

Thomas House | Meadowcroft Business Park | Pope Lane | Whitestake | Preston | PR4 4AZ

-------- Original Message --------

Subject:	[openraildata-talk] Wiki availability
Date:	2026-03-11 06:45
From:	"'Peter Hicks' via A gathering place for the Open Rail Data community" <openrail...@googlegroups.com>
To:	A gathering place for the Open Rail Data community <openrail...@googlegroups.com>

[russell pirie]

Yup bots are a real pain these days, we are constantly fighting them. Found the best solution was to hide behind Cloudflare.

Regards

Russell Pirie

UK Departure Boards LTD

Company 12166351

VAT 435869454

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/3a60dc0c8b02dcb4f1db90332feeeaf6%40railcam.uk.

[Peter Hicks]

All,

We are back.  Not all traffic is going via Cloudflare due to DNS changes not being instant, but come tomorrow, it should be sorted.

Expect intermittent availability for the next couple of days until things settle down.

Peter

--

[Seb Dazeley]

If you're getting bots still hitting your AWS IP directly, they might not obey the DNS change. You could set up a firewall or other checks in your proxy to reject traffic that isn't from Cloudflare, or change the IP of the server. I'm not familiar with AWS though. 

This probably won't be an issue if it's just AI crawlers instead of a targeted attack. 

Kind regards, 

Seb 

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/BYBmSI9VJAduGJWO4x0e7AzgAwlCfsDjQfKfya6sgqkuSJ-Q3KKCKLKS66-9tRpEJUbxfaIcZhP0D4wP09X_C5LW10vA8bmZt9N1vdgpSMU%3D%40poggs.co.uk.

[Peter Hicks]

On Thursday, 12 March 2026 at 08:48, Seb Dazeley <sebda...@gmail.com> wrote:

If you're getting bots still hitting your AWS IP directly, they might not obey the DNS change. You could set up a firewall or other checks in your proxy to reject traffic that isn't from Cloudflare, or change the IP of the server. I'm not familiar with AWS though. 

This probably won't be an issue if it's just AI crawlers instead of a targeted attack. 

Already on it - technically the ELB IPv4 addresses can change over time, hence the requirement to point at a CNAME rather than an 'A' record, but I'm already looking at how I can get identify Cloudflare-originated requests to the ELB and block others from being able to route through.  It just needs the DNS to be updated everywhere otherwise I'm going to be blocking legitimate users!

Peter