Very Slow Workflow Tasks Page (Dspace 7.5)

[Aroldo Rique Júnior]

Hello everybody

Our
Workflow tasks page is taking a long time to load, over a minute. Something similar also happens on the new item submission page.

My mpf-ui.json:

{
    "apps": [
        {
           "name": "mpf-ui",
           "cwd": "/dspace/mpf-ui/",
           "script": "dist/server/main.js",
           "instances": "-1",
           "exec_mode": "cluster",
  "node_args": "--max_old_space_size=2048",
           "env": {
              "NODE_ENV": "production"
           }
        }
    ]
}

We currently have over 150,000 items, spread across 270 collections, spread across various communities and subcommunities.

We created 3 groups of people, a group of Administrators, another one of Submitters and another one of Editors. These 3 groups are administrators, submitters and editors of all collections.

We've already increased the memory of Solr, Tomcat and the machine (SLES 15), but the slowness (and user complaints) continues...

Att,

Aroldo Rique

[Aroldo Rique Júnior]

We ran new tests. We put a user as editor of only one collection. The workflow tasks page takes less time to load, between 15-20 seconds. But when testing with a user who is part of the group of editors of all 270 collections, the page load time rises to almost 2 minutes, which is almost making the work of these users unfeasible.

Att,

Aroldo

[Aroldo Rique Júnior]

Is it possible to create groups with roles of admins, submitters, and editors by community instead of by collection? This would decrease the number of groups and could improve the performance of database queries.

[Aroldo Rique Júnior]

Is it possible to create groups with roles of admins, submitters, and editors by community instead of by collection? This would decrease the number of groups and could improve the performance of database queries.

[Aroldo Rique Júnior]

We are back to Dspace 6 until we resolve the issues.

Em sex., 1 de set. de 2023 às 14:58, Aroldo Rique Júnior <aro...@gmail.com> escreveu:

[DSpace Technical Support]

Hi Aroldo,

Sorry for the delay in responses.

From the slow workflow tasks page, were you able to determine which REST API requests were taking a longer time?  My suspicion is that it could provide important clues as to what might be causing the slow behavior.   You also could look to make sure no specific errors are thrown in the UI or backend by using our troubleshooting guide: https://wiki.lyrasis.org/display/DSPACE/Troubleshoot+an+error#Troubleshootanerror-DSpace7.x(orabove)

We have recently had a similar report of performance issues when a Group has a very large number of EPersons: https://github.com/DSpace/DSpace/issues/9052  We're investigating this one, but I'm *not entirely sure* if this could be the same issue impacting you.  (From your later responses though you are implying that you think it could be related to how Groups are working.)

If we can narrow down which REST endpoint or scenario is causing the problem, then that makes it easier to reproduce on our end.   That'd allow us to create a ticket, so that I can assign it to someone to investigate further.  Basically, it'd be helpful if you could gather some clues as to what might be causing the issue.  Without those clues, it'll be very difficult for us to debug the problem.

Let us know if you have time to gather more clues.  If so, I'd recommend starting with the Troubleshooting guide & looking for errors or REST API calls (in your browser's DevTools "Network" tab) which are taking a very long time to respond.

Tim

[Aroldo Rique Júnior]

Sorry Tim, I think I caused a bit of confusion here. I will, from now on, only answer on dspace-tech.

I added a ticket: https://github.com/DSpace/DSpace/issues/9053

I also added a postgresql log file, after the longer rest api call I mentioned earlier.

Att,

Aroldo

Em qua., 6 de set. de 2023 às 14:54, DSpace Community <dspace-c...@googlegroups.com> escreveu:

Hi Aroldo,

(This has gotten slightly confusing as we are now talking on dspace-community when I initially responded on dspace-tech.  I'd appreciate it if we can keep this discussion to one list.)

Could you check your logs for any additional clues?  It'd be really helpful to check to see if Solr or dspace.log (or similar) are providing any other hints on what might be going on.  

Essentially, I'm looking for clues that other developers could use to try to reproduce your setup.  It's excellent that you've narrowed down the REST API call... but if we can find more clues about what might be going on in your logs when that REST API call occurs, that can really help us determine where to look in our codebase for possible issues.

It's still very possible that this is related to the other issue that I mentioned on dspace-tech: https://github.com/DSpace/DSpace/issues/9052.  There are permissions checks going on here behind the scenes, and I wonder if something there is acting slowly.  So, based on what you've said so far, it's very difficult to say if the 100 workflow items is the real cause or if it's more related to how your Groups/EPersons are configured.  If we can find any other clues, that'd be really helpful.

If it's easier for you to simply add all that you can find into a Ticket (https://github.com/DSpace/DSpace/issues), that's OK with me.  I just need as many clues as you can gather so that we can find a volunteer to reproduce the issue & investigate what may be occurring.

Thanks!

Tim

On Wednesday, September 6, 2023 at 12:19:07 PM UTC-5 aro...@gmail.com wrote:

Hi Tim, thanks for the reply.

The 2 rest calls that take the most time are these:

Workflow task page (mais problem):
server/api/discover/search/objects?sort=score,DESC&page=0&size=10&configuration=workflow&embed=thumbnail&embed=item%2Fthumbnail

We have a team of 14 submitters who can submit new items in any of the 270 collections. We also have another team of 10 editors, who can accept or reject submitted items in all collections. When we had 100 items submitted in the queue for editing, the above rest call took 2 minutes to complete.

Deposit a new item (the delay here doesn't bother you that much):
server/api/workflow/workflowitems?projection=full

Att,

Aroldo

Em qua., 6 de set. de 2023 às 13:27, DSpace Community <dspace-c...@googlegroups.com> escreveu:

This discussion is taking place on the dspace-tech list.  Please try to keep your emails to a single list if at all possible.  Cross-posting to all lists can make it difficult for others to locate the answer in the future.

https://groups.google.com/g/dspace-tech/c/Me5TNlKcTEk

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-communi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/12fa82fd-1ba9-4c75-b4d8-c6f7b2600b61n%40googlegroups.com.

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-communi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-community/888d6994-4493-4e08-ab49-7333a2b143c1n%40googlegroups.com.