Memory Spikes / Server Crashes

[Liam Lloyd-Tucker]

Hi folks!

I'm running archivematica on an AWS EC2 instance (8 cores, 16 GB memory), and about once a day the memory usage on my instance spikes above 70% and the instance becomes unreachable. I'm still digging into what exactly is causing this problem, but I'm wondering if anyone has seen something similar and might be able to point me in a fruitful direction. I'll note that I am running multiple copies of the MCPClient (I've seen the issue with between 2 and 4 copies running).

Best,
Liam

[Joseph Anderson]

We've experienced this a few times, once due to processing large video files, but in our case boosting our memory to 16gb fixed it (so probably not your issue). Another time was related to Elastic Search which we eventually disabled because we weren't really using any search functionality in AM. And the last time it was related to JHOVE getting caught in a loop when processing various TIFF files, and the solution to that ended up being that the default java heap for JHOVE was set too low by default: https://github.com/archivematica/Issues/issues/1683#issuecomment-2148066040

[Liam Lloyd-Tucker]

In my case the problem seems to have been caused by Prometheus (perhaps specifically during video processing). We weren't using Prometheus, so I turned it off and haven't seen any crashes since.