Error 504 and Restoring from database backup

33 views
Skip to first unread message

elizabet...@mcgill.ca

unread,
Aug 27, 2020, 11:00:25 AM8/27/20
to ica-ato...@googlegroups.com
HI all,
Our AtoM 2.4 instance throws 504 errors whenever we try to access a fonds or collection.  We see that mysql is hogging CPU. This started after someone ran a job to delete a fairly sizeable hierarchy.  We think that mysql is unable to complete the transaction. When mysql is restarted (we did this a number of times), mysql restarts all its processes, hogs CPU, and Atom throws 504 errors when fonds are accessed.  
We only have about 68K records in our system. The delete job should have deleted approx 800-1000 items. The delete job was started at about 12:30 yesterday and 10 hours later it still seemed to be running because CPU usage was still high. 
Our thought is to restore from backup, as per instructions here: https://groups.google.com/g/ica-atom-users/c/5GyIKKCpRHA/m/c4euDQmDAAAJ

Would these instructions apply to AtoM 2.4?  
Also, could you explain if the php symfony tools:upgrade-sql step is required if an upgrade is not being performed?

Thank you!

Dan Gillean

unread,
Aug 27, 2020, 5:29:06 PM8/27/20
to ICA-AtoM Users
Hi Elizabeth, 

Very strange, sorry to hear about this frustrating situation. 

Overall, yes, those previously linked instructions should work for 2.4 as well - though I've basically just copied the Upgrading instructions in that post, so you can always look at the Upgrading docs for 2.4 here to confirm the commands: 
I'd probably suggest running the upgrade task once you load your data anyway, just in case. If the migrations have already been applied, then it shouldn't do anything bad to your load. Make sure you definitely do complete the prior step of dropping and recreating the database! 

 
Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


On Thu, Aug 27, 2020 at 11:00 AM elizabet...@mcgill.ca <elizabet...@mcgill.ca> wrote:
HI all,
Our AtoM 2.4 instance throws 504 errors whenever we try to access a fonds or collection.  We see that mysql is hogging CPU. This started after someone ran a job to delete a fairly sizeable hierarchy.  We think that mysql is unable to complete the transaction. When mysql is restarted (we did this a number of times), mysql restarts all its processes, hogs CPU, and Atom throws 504 errors when fonds are accessed.  
Our thought is to restore from backup, as per instructions here: https://groups.google.com/g/ica-atom-users/c/5GyIKKCpRHA/m/c4euDQmDAAAJ

Would these instructions apply to AtoM 2.4?  
Also, could you explain if the php symfony tools:upgrade-sql step is required if an upgrade is not being performed?

Thank you!

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/19985335-eaf8-4266-adfb-9059309dc15dn%40googlegroups.com.

José Raddaoui

unread,
Aug 31, 2020, 8:12:06 AM8/31/20
to AtoM Users
Hi Elizabeth, 

You are totally right about what happened and I hope you can restore your data properly. Some extra information if you're planning to delete that hierarchy again after the restoration ...

Deleting big hierarchies is a demanding operation due to the nested set updates required in the process and, when the operation timeouts in the browser, the rollback takes even longer, which is what you're seeing after restarting MySQL. AtoM includes a task to delete those hierarchies from the CLI avoiding the time constraints from a web request, knowing the slug of the top-level description, run the following command from the AtoM folder to delete the hierarchy (removing memory limitations too in case the hierarchy is that big):

php -d memory_limit=-1 symfony tools:delete-description <slug>


Additionally, if you're working with big description hierarchies you may want to consider upgrading to a more recent version, we have improved the deletion process in AtoM 2.5.4 to a point where I think you should be able to perform that deletion through the interface too, see the related ticket:


And in AtoM 2.6.x, thanks to the upgrade to MySQL 8, we have changed some important hierarchical queries to use CTE:


Best regards,
Radda.
Reply all
Reply to author
Forward
0 new messages