Bots? are slowing down our site

[Edward Warga]

Hello fellow AtoM users,

My institution runs AtoM version 2.7.3 - 192 with Library Host as our host and tech support. Our site has been running so slow lately. Pages take a long time to load, some elements never load on the page (like hierarchy). I've been working with Manny at Library Host. He has seen aggressive bots slowing down their ArchivesSpace instances, and we think the same thing might be going on with AtoM.

I found the following threads in the forum, and passed them on to Manny (I am not the system admin). I'm going to see if these help him troubleshoot the issues in AtoM. Any other advise from the group is welcome. How are folks dealing with Bots? It seems to be a problem with AI Bots run amuck. 
https://groups.google.com/g/ica-atom-users/c/IkIgVMcnW-M/m/2R5XZxIpAgAJ
https://groups.google.com/g/ica-atom-users/c/PwbPZcqJuDU/m/J3nMkB5lBAAJ
https://groups.google.com/g/ica-atom-users/c/TEmSM5MvHXs/m/0ds-wo0ZAgAJ

All the best,

Ed Warga

St. Cloud State University

[Dan Gillean]

Hi Ed, 

We've seen the same with many of our clients of late. Consequently, the AtoM 2.9 release now includes a configurable JavaScript challenge that can be configured that should reduce bot interference. Ask Library Host to upgrade you and send this documentation link: 

• https://www.accesstomemory.org/docs/latest/admin-manual/security/js-challenge/

I'm sure the AtoM team would also welcome follow-up feedback on whether this resolves the bot issues! 

Cheers, 

Dan Gillean, MAS, MLIS
Business & User Experience Analyst
Artefactual Systems, Inc.
604-527-2056

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ica-atom-users/1375b13a-f03c-4a84-b99a-28c56f7e83ean%40googlegroups.com.

[Edward Warga]

Hi Dan et al,

I am following up on this thread from summer 2025. We have since implemented AtoM 2.9 with the JSON security challenge. It is now mid September 2025, and I think the Bots have found us again. Our site has been slowing down lately, so I reached out to my system admin at LibraryHost and asked for a wellness check on our AtoM site -  https://archives.stcloudstate.edu/ 

Manny our sysadmin says, 
"Checking the site now, I can see that it is sluggish. It is very busy with dozens of requests a second and CPU utilization up around 60-70%, which is significantly higher average. I doubt all this traffic is due to humans this early in the morning. I don't know if the bot mitigations added by the AtoM dev team are working as intended, but they have been implemented. The only mitigation that comes to mind for me is going back up to the next server size (past Premium) which doubles RAM and core count again, which we tried earlier this year before upgrading to the latest AtoM."

I'd like to ask Dan and the forum, is there anything else we can try? How do we know the security check feature is working? That's it for now. Looking forward to any responses you all may have!

Best, Ed Warga - St. Cloud State University Library

[Dan Gillean]

Hi Edward, 

I will mostly defer to actual system administrators - there are some really smart folks in this forum and hopefully they might chime in. 

My first suggestion would be to have Library Host review the webserver logs, and use Nginx rules to outright block IP ranges that are consistently proving to be problem agents by sending repeated requests. Of those prior forum threads you found in your original message, one included the command our system admin use to search the Nginx access logs, and the other one from Jim Adamson showed a possible configuration block that you can use to block the bad actors you find. Also one user mentioned 3 specific bots they blocked that immediately brought down the traffic, here. 

Alternatively, a quick search for "block bots with Nginx" turns up dozens of results. Perhaps discuss some of these options with your hosting provider?

You can also try adding a robots.txt file to your site (also noted in one of those linked threads), though of course the worst of the bot actors don't respect these... 

Finally, I did see this other thread (here) that included some tips for getting a third-party tool, Anubis, set up with AtoM. 

Hopefully others can share their experiences, tips, tricks, and workarounds!

Dan Gillean, MAS, MLIS
Business & User Experience Analyst
Artefactual Systems, Inc.
604-527-2056

he / him

To view this discussion visit https://groups.google.com/d/msgid/ica-atom-users/c1c24b74-8d8f-43e2-b67e-cd0b129ea94dn%40googlegroups.com.