Query about RO / RW worker quantity
26 views
Skip to first unread message
Pablo Maldonado
Aug 19, 2020, 12:31:25 AM8/19/20
to schedulix
Hello,
I wanted to know how to determine the number of RO and RW workers.
Pablo maldonado
Ronald Jeninga
Aug 19, 2020, 2:59:53 AM8/19/20
to schedulix
Hi Pablo,
If you find a lot of waiting sessions all the time, the system doesn't perform optimally. (Obviously IDLE sessions are fine; the server waits for the user, not vice versa).
that's actually very easy. If you issue a "SHOW SYSTEM;", either through sdmsh or by clicking the System Information button on the GUI's Main Menu, you'll find exactly what you want to know.
Through sdmsh it looks like:
...
OS.NAME : Linux
OS.ARCH : amd64
OS.VERSION : 3.10.0-1127.18.2.el7.x86_64
WORKER :
ID TYPE NAME STATE TIME
-- ---- ------- ---------- ------------------------
0 RW Worker0 IDLE 19 Aug 2020 06:44:52 GMT
1 RO Worker1 IDLE 19 Aug 2020 06:44:55 GMT
2 RO Worker2 IDLE 19 Aug 2020 06:44:53 GMT
3 RO Worker3 ShowSystem 19 Aug 2020 06:44:56 GMT
4 RO Worker4 IDLE 19 Aug 2020 06:44:51 GMT
5 RO Worker5 IDLE 19 Aug 2020 06:44:54 GMT
...
As you can see, I have a single RW Worker and another 5 RO workers active.
If you don't change the configuration, you'll always have a single RW worker and some number of RO workers.
In order to change the number of RO Workers, you set the parameter WorkerThreads in the server.conf to some other (positive) value.
To change the number of RW workers, you change the value of the parameter WriterThreads.
If you don't have many active users, it'll hardly improve anything if you increase any of the mentioned parameters.
In fact, increasing the number of WriterThreads could even have a negative impact on performance because of the Locking overhead that is added.
Watching the Show System output might give you an indication whether the number of RO workers is configured appropriately or not.
If you find that all workers are busy (i.e. not IDLE) most of the time, increasing the number of RO workers makes sense.
Else you'd only increase overhead without benefit.
Another way of watching the behaviour of the system is to look at the "LIST SESSIONS;" output (also contained in the GUI#s System Information).
[SYSTEM@localhost:2506] SDMS> list sessions;
List of Sessions
THIS SESSIONID PORT START TYPE USER UID IP TXID IDLE STATE TIMEOUT INFORMATION STATEMENT WAIT
---- --------- ---- ----------------------------- --------- -------------------------------- ---- --------- -------- ---- ------ ------- ----------------------------------------- -------------- ----
1002 2506 Tue Aug 18 14:03:48 CEST 2020 JOBSERVER GLOBAL.EXAMPLES.HOST_1.SERVER 1047 127.0.0.1 38212978 2 IDLE 300 jobserver[schedulix@ocelot.independit.de]
1003 2506 Tue Aug 18 14:03:49 CEST 2020 JOBSERVER GLOBAL.EXAMPLES.LOCALHOST.SERVER 1037 127.0.0.1 38212981 2 IDLE 300 jobserver[schedulix@ocelot.independit.de]
1004 2506 Tue Aug 18 14:03:50 CEST 2020 JOBSERVER GLOBAL.EXAMPLES.HOST_2.SERVER 1057 127.0.0.1 38212984 2 IDLE 300 jobserver[schedulix@ocelot.independit.de]
* 1007 2506 Wed Aug 19 08:44:50 CEST 2020 USER SYSTEM 0 127.0.0.1 38212986 0 ACTIVE 0 sdmsh[ronald@ocelot.independit.de] list sessions;
1234321 0 Tue Aug 18 14:03:26 CEST 2020 USER SchedulingThread 2 <null> 38212975 2 IDLE 0 <null>
1234322 0 Tue Aug 18 14:03:26 CEST 2020 USER GarbageThread 2 <null> 38212000 466 IDLE 0 <null>
1234323 0 Tue Aug 18 14:03:26 CEST 2020 USER TriggerThread 2 <null> 38212960 8 IDLE 0 <null>
1234324 0 Tue Aug 18 14:03:26 CEST 2020 USER PoolThread 2 <null> 38212606 179 IDLE 0 <null>
19630127 0 Tue Aug 18 14:03:26 CEST 2020 USER TimerThread 2 <null> 38212963 8 IDLE 0 <null>
If you find a lot of waiting sessions all the time, the system doesn't perform optimally. (Obviously IDLE sessions are fine; the server waits for the user, not vice versa).
HTH
Ronald
Pablo Maldonado
Aug 19, 2020, 11:01:37 AM8/19/20
to schedulix
Hello Ronald
Thanks for your answer, I have 2 worker RO and 1 worker RW, but they are IDLE most of the time, but in the server log I see the following and I don't know why:
WARNING [0,18238(18238)] 08 Aug 2020 01:10:51 GMT Connection (user 0) timed out
WARNING [0,18241(18241)] 08 Aug 2020 01:12:37 GMT Connection (user 0) timed out
WARNING [0,18269(18269)] 08 Aug 2020 01:39:17 GMT Connection (user 0) timed out
WARNING [0,18418(18418)] 08 Aug 2020 04:08:35 GMT Connection (user 0) timed out
WARNING [0,19062(19062)] 08 Aug 2020 14:42:54 GMT Connection (user 0) timed out
WARNING [0,19066(19066)] 08 Aug 2020 14:45:44 GMT Connection (user 0) timed out
WARNING [0,19289(19289)] 08 Aug 2020 18:25:51 GMT Connection (user 0) timed out
WARNING [0,19683(19683)] 09 Aug 2020 00:55:12 GMT Connection (user 0) timed out
WARNING [0,19810(19810)] 09 Aug 2020 02:59:42 GMT Connection (user 0) timed out
WARNING [0,20460(20460)] 09 Aug 2020 13:43:23 GMT Connection (user 0) timed out
WARNING [0,20497(20497)] 09 Aug 2020 14:22:47 GMT Connection (user 0) timed out
WARNING [0,20510(20510)] 09 Aug 2020 14:29:06 GMT Connection (user 0) timed out
WARNING [0,20532(20532)] 09 Aug 2020 14:51:57 GMT Connection (user 0) timed out
WARNING [0,20550(20550)] 09 Aug 2020 15:07:08 GMT Connection (user 0) timed out
WARNING [0,20553(20553)] 09 Aug 2020 15:09:03 GMT Connection (user 0) timed out
WARNING [0,20575(20575)] 09 Aug 2020 15:29:33 GMT Connection (user 0) timed out
WARNING [0,20580(20580)] 09 Aug 2020 15:33:46 GMT Connection (user 0) timed out
WARNING [0,20641(20641)] 09 Aug 2020 16:32:38 GMT Connection (user 0) timed out
WARNING [Listener] 09 Aug 2020 22:04:30 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:31 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:32 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:33 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:34 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:35 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:36 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:37 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:38 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:39 GMT Out of user connects, waiting 1 second
WARNING [Listener] 09 Aug 2020 22:04:40 GMT Out of user connects, waiting 1 second
Pablo Maldonado
Ronald Jeninga
Aug 19, 2020, 11:20:50 AM8/19/20
to schedulix
Hi Pablo,
Ronald
OK, that's something different.
The Connection Timeouts aren't that relevant. If you start an sdmsh (or any other session, like e.g. a session of the Zope server) and just do nothing, the server will close the connection by itself.
The reason for implementing this is, more or less, the other warning.
The user connections are a limited resource. You can configure the maximum number of user connections with the Parameter UserThreads in the server.conf.
The timeout for sessions has been introduced to prevent that loads of people start an sdmsh and don't use it, thereby potentially blocking other users.
It's not water proof though. If you issue an "ALTER SESSION WITH TIMEOUT = 0;", the timeout logic is disabled for that session.
You'll need UserThreads 1 per Jobserver, a few per Zope server, and a few more for power users and jobs.
Hence if you have 15 Jobservers running, a number like 30 would seem fine to me. But 50 or even 100 is a reasonable number too.
On the other hand, if you find messages in your server's log, just increase the number of UserThreads.
An unused UserThread is not expensive at all, it's only an entry in a table.
HTH
Best regards,
Ronald
Pablo Maldonado
Aug 19, 2020, 6:32:03 PM8/19/20
to schedulix
Hi Ronald,
Thank you very much for your help
Pablo maldonado
Reply all
Reply to author
Forward
0 new messages