ArchivesSpace crashing following upgrade

[Palchick, Andra]

Hello all,

 

I’m reaching out about an issue we’ve been experiencing since a recent upgrade, from 3.2 to 4.1.1.

 

Both our PUI and SUI have been regularly crashing, and displaying one of the two attached error messages. We are usually able to reconnect after many refreshes, but it does happen several times a day. Unfortunately, we haven’t noticed a regular pattern of when it happens, or what functions may trigger it.

 

After reaching out to ASpace tech support, we’ve been informed that this is likely due to a heavy load of AI bots, which forces our instance to crash. They’ve offered to set up a separate staff instance for us, which would not be affected by the bots accessing our catalog, but this unfortunately, would not solve for the PUI – meaning our catalog would remain basically unusable/unreliable for researchers.

 

We never experienced this issue prior to upgrading, though tech support has assured us that they are not related. Has anyone else experienced a similar issue? And if so, were you able to find a solution that also resolved the problem on the user-end?

 

Many thanks for any guidance!
Andra

 

Andra Palchick
Archivist

Library and Archives

 

Philadelphia Museum of Art

_______________________________________

t: 215-684-7649 
PO Box 7646, Philadelphia, PA 19101-7646
philamuseum.org 

 

[Schanz, Megan]

We had a bot issue on our ArchivesSpace instance caused by AI bots in late January, not at all related to a version upgrade. We did a low-tech solution of just blocking them by a combination of user-agent and IP range along with adding in rate limiting.

Nginx Configuration

But we were experiencing similar errors when the bot "attacks" were occurring when the site was throwing errors and basically unusable to real users. This solution has at least been working for us until we have time to invest in putting up a captcha (we like the ones that don't require human interaction to get through to the site).

- Megan

_____________________________________

Megan Schanz

Application Developer & Systems Administrator

Michigan State University Libraries

---

From: 'Palchick, Andra' via Archivesspace_Users_Group <Archivesspac...@lyrasislists.org>
Sent: Thursday, March 19, 2026 10:58 AM
To: archivesspac...@lyrasislists.org <archivesspac...@lyrasislists.org>
Subject: [ArchivesSpace Users Group] ArchivesSpace crashing following upgrade

 

--
You received this message because you are subscribed to the Google Groups "Archivesspace_Users_Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to Archivesspace_User...@lyrasislists.org.
To view this discussion visit https://groups.google.com/a/lyrasislists.org/d/msgid/Archivesspace_Users_Group/MW6PR01MB8229853BA3F344C4CD27D3638D4FA%40MW6PR01MB8229.prod.exchangelabs.com.

[Blake Carver]

 

The tuning docs here outline things to check and/or change

https://docs.archivesspace.org/provisioning/tuning/

I agree bots can be a BIG problem. There are many different ways to fight the bots, and easy/cheap/fast one is a solid robots.txt like this one:

https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/master/robots.txt/robots.txt

That should help and should make a difference, but it won't stop all of them.

Looking at the proxy logs will tell you how much of a problem the bots are causing.

Also be sure to check your ArchivesSpace logs for FATAL or ERROR lines. Sometimes after an upgrade the indexer can get stuck in a loop and it never finishes up. That can cause those problems you're seeing. Check the logs for "Indexed" to see if that's happening. It will need to be on debug level for "Indexed" to show up. The indexer running nonstop will cause performance issues.

---

From: 'Palchick, Andra' via Archivesspace_Users_Group <Archivesspac...@lyrasislists.org>
Sent: Thursday, March 19, 2026 10:58 AM
To: archivesspac...@lyrasislists.org <archivesspac...@lyrasislists.org>
Subject: [ArchivesSpace Users Group] ArchivesSpace crashing following upgrade

 

Hello all,

--

[Jeffrey Mudge]

We also had this experience last year and implemented Cloudflare’s Turnstile bot detection for all PUI traffic with great success. Robots.txt changes were ineffective since the AI bots were not respecting them.

Jeffrey Mudge

Digital Initiatives Coordinator

Wheaton College Library (IL)

 

From: archivesspac...@lyrasislists.org <archivesspac...@lyrasislists.org> on behalf of Schanz, Megan <scha...@msu.edu>
Date: Thursday, March 19, 2026 at 10:15 AM
To: archivesspac...@lyrasislists.org <archivesspac...@lyrasislists.org>, Palchick, Andra <Andrienne...@philamuseum.org>
Subject: [ArchivesSpace Users Group] Re: ArchivesSpace crashing following upgrade

EXTERNAL MESSAGE: Use caution with replies, links, and attachments.

To view this discussion visit https://groups.google.com/a/lyrasislists.org/d/msgid/Archivesspace_Users_Group/SJ1PR12MB614842ED0532B94DDDAEB1A8CE4FA%40SJ1PR12MB6148.namprd12.prod.outlook.com.