Very slow in generating exports.

[Alberto Pereira]

Hi all,

I wonder if any of you has experienced this. The generation of reports and other exports is very slow: it can take hours for a few thousand descriptions. I don't see anything weird in the logs (gearman, atom-worker or web server) in an Ubuntu os.
Are there common reasons for this to happen or what can I do to try and debug this?

Thanks!

Regards

[Dan Gillean]

Hi Alberto, 

If you are able to increase the amount of memory allocated in your server, this will likely help the most. It's possible that more CPUs might also help. 

If you're really looking to dig deep on this: 

• We have some basic tips on resource allocation monitoring with htop in our Troubleshooting docs, here. It might be good to determine what resources are being consumed, how close to spiking they are etc, while running a report or export, to see where the bottleneck is. There are also many other monitoring tools out there. 
• It's possible to deploy different AtoM components in different VMs - i.e. one VM for the database, one for Elasticsearch, etc - this too would allow you allocate more resources where they are needed, and gain some performance improvements overall. 
• Gearman, AtoM's job scheduler, has its own documentation - you could look into configuration options for adding more workers, parallel processing, etc
• I'm not sure, but it might also help to review and adjust some of your PHP execution limits - specifically, check out memory_limit and other related directives
• If the database is the culprit, you could look into using MySQL's slow query log to see if you can determine where the bottleneck is. I haven't tried using this myself, however, so I can't offer any guidance. 

Let us know what you try, and if it helps!

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1442e107-76cd-4a0a-a969-c98a3314737f%40googlegroups.com.