Dspace 7.6.1 performance issues

[akshay ts]

Hi all,

We upgraded our DSpace server from 5.8 to 7.6.1 and since then we have been having performance issues. For example, sometimes instead of the top level communities showing on the home page has "Error fetching items", or item pages loading slowly, or searches timing out/taking a long time to load. 

Our OS is RHEL 9, and our server has 16GB RAM, and 4 CPU's. RAM division wise we have given Solr 2GB, Tomcat 2GB and pm2 has been given the arguments "max_memory_restart": "1500M" and "node_args": "--max-old-space-size=2048", each node process seems to settle at a bit over 1GB in memory.

The repository has close to 300k items with close to 3.5k collections.

We plan on updating to 7.6.2 soon to take advantage of some of the improvements that have occurred there (for example, we are hoping it will allow non-admin staff to load the workflow tasks pages), however are there any suggestions as to what else we might do to improve performance or what we may have missed?  For reference in our logs we are regularly seeing "GET /items/<uuid> HTTP/1.1" pages taking greater than 10 or 15 seconds (We're also not seeing a corresponding significant length of time for that uuid's /server/api calls found by doing a grep so we're theorizing it's something to do with angular).

It should also be noted that in addition to pages regularly taking a long time to load, we have regular issues with 502 errors, and the hdl.handle.net redirections seem to have problems/get lost somewhere (potentially because of the page performance issues?).

Thanks and Regards,

Akshay Karthik Thana Sekar

[Technologiczny Informator]

Hi,

Have you applied all the recommendations on this page?

https://wiki.lyrasis.org/display/DSDOC7x/Performance+Tuning+DSpace

And especially cluster mode?

Regards,

Mariusz

[Alan Orth]

Dear Ashkay,

If you do some log analysis I bet you will see tons of requests from bots causing high load on the Angular frontend. Part of this is due to Angular server side rendering (SSR) being slow, see:

https://github.com/DSpace/dspace-angular/issues/3110

In our case, even with SSR disabled, bots were making way too many requests, from way too many IPs (hundreds, thousands!), at the same time. I ended up implementing strict rate limiting based on IP blocks and user agents in nginx. Now our site is much faster.

Regards,

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/b99bb45a-5c40-49b3-9a99-9270b41fc346n%40googlegroups.com.