DSpace 7.6.1 timeouts in production

[Gabriel Martins]

Dear All,

I am experiencing timeout issues when trying to make the DSpace 7 platform available in production with HTTPS when the front end is served by dspace-ui-deploy. However, I am able to access it normally when served by 'yarn start:dev'."

I tried following the documentation strictly and also looked into similar questions on this forum, but unfortunately, I haven't had any success yet. The issue seems to persist despite following the recommended steps. It's possible that there might be a specific configuration or setting that is causing the timeouts when using HTTPS with dspace prod build.

If anyone has encountered a similar issue, I would greatly appreciate any guidance or suggestions. Thank you in advance for your help!

[Abel Gómez]

Hi Gabriel,

can it be related to this?

https://groups.google.com/g/dspace-tech/c/ByXn_9oABOo

As explained in the docs (https://wiki.lyrasis.org/display/DSDOC7x/Technology+Overview), Server Side Rendering (SSR) is disabled when running in dev mode; and as I explained in the link above, I have experienced severe performance issues in my production site when bots were crawling it.

Maybe that's why you are having issues when running in production mode (and not in dev mode).

I think the performance issues I suffered from are caused by SSR and caching, but I have not received any "educated opinion" on this issue.

Any similar experience would be really appreciated.

Abel

[Max Nuding]

Hi Abel, hi Carolyn,

a couple notes from us, we've had similar experiences.

> Since those tenths of requests were coming from the Angular frontend,
I suspected it had to be related to the server-side caching features,
but I was unable to find the root cause.

The Angular frontend is doing SSR (Server-Side Rendering), which means
it is constructing the whole page before sending it to the client. This
is meant to help bots/search engine crawlers by sending a complete page,
but absolutely kills performance when enough crawlers are active. You're
seeing so many requests from the Angular frontend, because "this is just
how it works"!. If you open  your DSpace instance in  browser and watch
the network tab in the browser console you can see, that you also
generate tens of requests per page.

We haven't found any good solutions yet, but manage to mitigate the
issues a bit. We extended our robots.txt.ejs to block a slew of unwanted
bots (Mostly AI and SEO bots) and we use *a lot* more hardware than the
recommended amount. Our frontend (together with the proxy) are running
on 8 cores with 7 frontend instances and 16GB of RAM (although we could
probably get by with 12GB of RAM). Additionally, we have a second server
with the backend, solr and database on 6 cores and another 16 GB of RAM.

We have found the requirements listed in the wiki to be very inaccurate,
especially the frontend needs a *lot* of CPU power (but not as much
memory).


Best regards

Max

> --
> All messages to this mailing list should adhere to the Code of
> Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
> ---
> You received this message because you are subscribed to the Google
> Groups "DSpace Technical Support" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dspace-tech...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/dspace-tech/a002c287-2c7d-4552-a502-60750935ec15n%40googlegroups.com
> <https://groups.google.com/d/msgid/dspace-tech/a002c287-2c7d-4552-a502-60750935ec15n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Herr Max Nuding
Softwareentwickler / Abt. IT-Dienste für Forschung und Lehre
Kommunikations-, Informations-, Medienzentrum (KIM)
Universität Konstanz
78467 Konstanz
Tel: +49.7531.88-4658
Raum: B707