process-based workers?

[t.c....@gmail.com]

Hi all,

When importing a large number of studies (~7000), my OpenREM server starts running into resource issues after some time. From what I can tell, Huey appears to be configured to use process-based workers rather than thread-based workers. This results in a steadily growing number of sleeping worker processes, each consuming a small but non-zero amount of memory. Over time, this exhausts available RAM and the operating system begins killing processes.

Would it be possible (and supported) to configure Huey to use thread-based workers instead to avoid this behaviour? Are there any known downsides or reasons why process workers are preferred in OpenREM?

Best regards,
Tim

[Ed McDonagh]

Hi Tim

For others reading this, this refers to how workers are dealt with in version 1.0 in the beta releases, and will be the mechanism for the final release.

Looking at the defaults, for Linux this is set to process and number of workers is set to the number of CPUs. However for Windows, this setting is overridden and set to threads and the number of workers is set to 1.

I would have to experiment to see what the relative benefits of the different settings are. I note that the huey docs have some useful information on both the number of workers and the worker type. It might be that some of the processes are CPU heavy and would benefit from the current Linux setting, but others, such as importing 7000 studies at once, would benefit from thread based workers.

If you are able to test the performance, then feed back on this would be very helpful. I don't think the person who implemented this initially reads this list and isn't actively contributing to the development currently, so I can't ask him how he came to the default settings we have. However, from my brief review of the code and the docs, changing between the two looks like a safe thing to do, at least for testing.

Kind regards

Ed

--
You received this message because you are subscribed to the Google Groups "OpenREM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrem+u...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openrem/9111287d-fddf-452a-83b8-929c3c8383c6n%40googlegroups.com.

[t.c....@gmail.com]

Thanks Ed,

I'll do some testing... I was wondering though, do you see the same behaviour on your server? I.e. when retrieving, do you see the number of openrem_rdsr.py processes increasing above the #workers limit?

According to the Huey documentation that shouldn't happen; also for process type it should be constrained.

regards, Tim

Op woensdag 11 februari 2026 om 00:08:09 UTC+1 schreef Ed McDonagh:

[t.c....@gmail.com]

I believe I have identified the cause of the increasing number of openrem_rdsr.py processes on my system.

Although the openrem_rdsr.py script delegates the actual processing to Huey, the openrem_rdsr.py process itself remains active until it explicitly exits. Queue management is currently handled outside of Huey (in background.py) and relies on an increasing delay mechanism ("_sleep_for_linear_increasing_time()"). As a result, each invocation remains alive in a sleeping state for progressively longer periods.

When multiple RDSRs arrive within a short timeframe, new instances of the script are started while earlier ones are still waiting. This leads to a rapid accumulation of processes.

Reducing the sleep interval does not resolve the issue — it only affects how frequently the waiting processes wake up. The underlying problem is that the entry script continues running instead of exiting after enqueueing the task.

It may be preferable to let Huey manage concurrency control entirely and modify openrem_rdsr.py so that it exits immediately after enqueueing the task to Huey/Redis.

Switching from processes to threads would not address this behaviour, as openrem_rdsr.py is already spawned as a separate process by Orthanc for each incoming RDSR.

Best regards, Tim

Op woensdag 11 februari 2026 om 09:52:01 UTC+1 schreef t.c....@gmail.com: