Open Rail Data Wiki availability

9 views

Skip to first unread message

Peter Hicks

unread,

6:10 AM (6 hours ago) 6:10 AM

to A gathering place for the Open Rail Data community

All,

Over the last few days, the number of hits per hour on the wiki has increased substantially - see below:

It appears there are multiple bots scraping un-cacheable (dynamic) content from the site at the same time, consuming the entirety of the available CPU resources allocated to the wiki.

Whilst I put workarounds in place to stop this, please accept my apologies for the intermittent availability of the site.

Peter

Peter Hicks

unread,

7:08 AM (5 hours ago) 7:08 AM

to openrail...@googlegroups.com

All,

I've blocked a number of IP addresses at Meta which were persistently scraping the site history, which has improved things substantially. Note the different scales on the X and Y axes - the takeaway being that there are now significant fewer requests:

I am planning up updating Mediawiki over the next week or two and I will likely put in an aggressive caching layer and further restrict expensive operations that hit the database to logged-in users.