Suddenly very slow ~ 9 seconds per page

[Chris Sykes]

Hi,

I've installed reviewboard using the docker image, which has been running for 3-4 months, but recently we've noticed severe performance problems.

Currently loading any page from reviewboard takes around 9 seconds. This figure is consistent and is the same whether targeting nginx or the docker container's port directly.

Logging is set to debug, but no useful log messages to figure out what's going on. This is all I can see:

2022-08-04 15:58:39,097 - INFO -  - root - Reloading logging settings
2022-08-04 15:58:44,451 - INFO -  - root - Reloading logging settings
2022-08-04 15:59:36,402 - INFO -  - root - Reloading logging settings
2022-08-04 16:00:17,551 - INFO -  - root - Reloading logging settings
2022-08-04 16:03:56,574 - INFO -  - root - Reloading logging settings
2022-08-04 16:04:34,745 - INFO -  - root - Reloading logging settings
2022-08-04 16:04:36,428 - INFO -  - root - Reloading logging settings
2022-08-04 16:04:50,750 - INFO -  - root - Reloading logging settings

Caching appears to be working fine (can't upload image but 97% hit rate)

Any ideas on what to check? Am worried that there's no log messages.

[Christian Hammond]

Hi Chris,

That usually points to either a cache server issue (I know you checked this), slowdown with the database (usual culprit — make sure that's not hitting RAM/disk space issues or showing locked queries), or something on the filesystem level (if you're mounted over NFS and there's any need for disk access, this can happen).

If you access /api/info/ on that server, do you still see the delay? This has minimal database access and can be a quick way to see if it's an issue on the Review Board server itself or something that the server is talking to.

Christian

--
Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups "Review Board Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/75863d96-405c-4fd9-adb7-5906e40a33c8n%40googlegroups.com.

--

Christian Hammond

President/CEO of Beanbag

Makers of Review Board

[Chris Sykes]

Hi Christian,

I think we've traced the problem to a DNS server being down and docker not redirecting to the working alternative, so it couldn't find the database's IP.

I've added an extra hosts entry into the container to skip that and it's back down to 1.4seconds.

Thanks a lot for your help!

Chris

[Christian Hammond]

Ah, DNS. That'll do it! Glad you got it figured out :)

Christian

To view this discussion on the web visit https://groups.google.com/d/msgid/reviewboard/054a9d79-74e5-4353-824d-b4a9adfc0cean%40googlegroups.com.