Managing resource usage via cgroups v2 on an AP

[Matthew West]

Apologies for not immediately condor-relevant question but given the heavy use of cgroups by this community, I figured it would be a good place to ask.

What is y'all's preferred cgroup config for minimizing the resource usage by users logged on to an AP? Anything beyond the usual suspects of  

• CPUQuota
• MemoryHigh/Max
• TasksMax

to make sure that both the schedd and all human users can get work done? 

Cheers,
Matt

 

-- 
Matthew T. West (he,him,his) | Systems Programmer/Administrator
UNC Charlotte | University Research Computing (office of OneIT)
9214 South Library Lane| Room 301 | Charlotte, NC 28223
Phone: 704-687-8766
mwe...@charlotte.edu | http://www.charlotte.edu/urc/

[Oliver Freyermuth]

Hi Matt,

good question indeed — and I think it's quite relevant for all AP operators, especially given increasing memory / CPU hogging by editors with LLM integration and more, often used on APs...

Beyond the usual suspects, which are incredibly useful, we also use:
MemorySwapMax

We found keeping swap active is often useful (to swap out unused / rarely accessed memory hoarders, e.g. from such stuff mentioned before) and rather allows the system to use this memory for file cache for example.
However, to prevent things from going haywire, limiting swap is then very helpful.
We also combine this with zswap to make it more useful, but I'm digressing from the topic of cgroups a bit.

Cheers and thanks for asking,
Oliver

Am 15.07.26 um 20:48 schrieb 'Matthew West' via HTCondor Users:

> Apologies for not immediately condor-relevant question but given the heavy use of cgroups by this community, I figured it would be a good place to ask.
>
> What is y'all's preferred cgroup config for minimizing the resource usage by users logged on to an AP? Anything beyond the usual suspects of
>

> * CPUQuota
> * MemoryHigh/Max
> * TasksMax

>
> to make sure that both the schedd and all human users can get work done?
>
> Cheers,
> Matt
>
> --
> Matthew T. West (he,him,his) | Systems Programmer/Administrator
> UNC Charlotte | University Research Computing (office of OneIT)
> 9214 South Library Lane| Room 301 | Charlotte, NC 28223
> Phone: 704-687-8766
> mwe...@charlotte.edu |http://www.charlotte.edu/urc/
>

> --
> Archives for older messages are found at https://www-auth.cs.wisc.edu/lists/htcondor-users/ <https://www-auth.cs.wisc.edu/lists/htcondor-users/>
> ---
> You received this message because you are subscribed to the Google Groups "HTCondor Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to htcondor-user...@g-groups.wisc.edu <mailto:htcondor-user...@g-groups.wisc.edu>.
> To view this discussion visit https://groups.google.com/a/g-groups.wisc.edu/d/msgid/htcondor-users/ec1db57d-2e38-4aac-a291-2e58350b8d32%40charlotte.edu <https://groups.google.com/a/g-groups.wisc.edu/d/msgid/htcondor-users/ec1db57d-2e38-4aac-a291-2e58350b8d32%40charlotte.edu?utm_medium=email&utm_source=footer>.


--
Oliver Freyermuth
Universität Bonn
Physikalisches Institut, Raum 1.055
Käthe-Kümmel-Straße 1
53115 Bonn
--
Tel.: +49 228 73 2367
Fax: +49 228 73 7869
--

[Matthew West]

Good afternoon Oliver,

Yes! MemorySwapMax is definitely essential as I discovered in testing. And I have not heard of ZSwap before, but seem at least worth a look.

• Is it worth turning on *Accounting for CPU, Mem, IO, etc if they aren't already?
• Would restricting IOPs & Bandwidth for users on an AP impact job performance, or just keep someone from banging on the local or networked storage system?
• I know there are some (e)BPF fans here, so figure somebody has got an interesting filter script floating around.

Cheers and thanks all,
Matt

On 7/15/26 3:51 PM, Oliver Freyermuth wrote:

[Caution: Email from External Sender. Do not click or open links or attachments unless you know this sender.]

[Greg Thain]

Matthew/Oliver:

There's lots of good stuff in this thread.  A couple more I'd consider: 
We see more and more users abusing /tmp space.  By default, HTCondor
uses /tmp to authenticate between clients and daemons on the same
machine, so we often can't just make a per-user /tmp mount.  We can
however, install a file system quota on /tmp to help out with the problem.

For a long time now, we've used pam to set an rlimit on cpu for
processes that are children of login for users (on the order of a couple
of hours).  This doesn't prevent programs that spawn lots of relatively
short lived processes, but does tend to be effective at killing out of
cpu-control processes that probably should not be running unmanaged on
an AP.


-greg