[slurm-users] Creating /run/user/$UID - for Podman runtime
John Snowdon via slurm-users
-
Apptainer
-
Podman (rootless)
-
Docker (rootless)
Michael DiDomenico via slurm-users
prolog/epilog to create the directories and clean them up. it's only
a couple lines of bash.
> slurm-users mailing list -- slurm...@lists.schedmd.com
> To unsubscribe send an email to slurm-us...@lists.schedmd.com
--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com
Paul Edmon via slurm-users
/etc/containers/storage.conf and pointed the following variables to /tmp:
storage.conf:runroot = "/tmp/containers-user-$UID/storage"
storage.conf:graphroot = "/tmp/containers-user-$UID/storage"
storage.conf:rootless_storage_path = "/tmp/containers-user-$UID/storage"
We also have a prune script which cleans up /tmp periodically keeping it
clean.
I like your solution for subuid, we put together a puppet module that
does much the same thing: https://github.com/fasrc/puppet-subuid
-Paul Edmon-
Paul Edmon via slurm-users
https://forge.puppet.com/modules/southalc/podman/readme
-Paul Edmon-
On 9/5/25 9:20 AM, Michael DiDomenico via slurm-users wrote:
John Snowdon via slurm-users
Sent: 05 September 2025 14:29
To: slurm...@lists.schedmd.com <slurm...@lists.schedmd.com>
Subject: [slurm-users] Re: Creating /run/user/$UID - for Podman runtime
We recently setup the same thing (Rocky 8). What we did was we set
/etc/containers/storage.conf and pointed the following variables to /tmp:
storage.conf:runroot = "/tmp/containers-user-$UID/storage"
storage.conf:graphroot = "/tmp/containers-user-$UID/storage"
storage.conf:rootless_storage_path = "/tmp/containers-user-$UID/storage"
We also have a prune script which cleans up /tmp periodically keeping it
clean.
I like your solution for subuid, we put together a puppet module that
-Paul Edmon-
On 9/5/25 9:20 AM, Michael DiDomenico via slurm-users wrote:
> for what it's worth, we found the simplest solution was just to run a
> prolog/epilog to create the directories and clean them up. it's only
> a couple lines of bash.
>
> On Fri, Sep 5, 2025 at 7:59 AM John Snowdon via slurm-users
> <slurm...@lists.schedmd.com> wrote:
>> We are in the middle of implementing an extensive range of container support on our new HPC platform and have decided to offer our users a wide suite of technologies to better support their workloads:
>>
>> Apptainer
>> Podman (rootless)
>> Docker (rootless)
>>
>>
Roger Moye via slurm-users
John, we ran into the same issues that you did. One thing we discovered was that podman relies heavily on the $TMPDIR variable if it is set. It seemed that in spite of making changes to storage.conf, podman still tried to use $TMPDIR
for some of its state information. Since $TMPDIR on our cluster was pointed at an NFS mount, that created all sorts of issues.
We implemented similar solutions as discussed in this thread. However, jobs that are going to run podman had to be configured to unset $TMPDIR. This allowed the rest of our podman config to work as intended. So the existence of $TMPDIR kept interfering
with our solution. Easy to fix since our compute jobs are created using an automated build process.
Roger Moye
HPC Architect
713.898.0021 Mobile
QUANTLAB Financial, LLC
3 Greenway Plaza
Suite 200
Houston, Texas 77046
From: John Snowdon via slurm-users <slurm...@lists.schedmd.com>
Sent: Friday, September 5, 2025 2:55 AM
To: slurm...@lists.schedmd.com
Subject: [slurm-users] Creating /run/user/$UID - for Podman runtime [External Email]
Caution: This email originated from outside of the organization. Do not click links or open attachments unless you recognize and know the content is safe.
-----------------------------------------------------------------------------------
The information in this communication and any attachment is confidential and intended solely for the attention and use of the named addressee(s). All information and opinions expressed herein are subject to change without notice. This communication is not to be construed as an offer to sell or the solicitation of an offer to buy any security. Any such offer or solicitation can only be made by means of the delivery of a confidential private offering memorandum (which should be carefully reviewed for a complete description of investment strategies and risks). Any reliance one may place on the accuracy or validity of this information is at their own risk. Past performance is not necessarily indicative of the future results of an investment. All figures are estimated and unaudited unless otherwise noted. If you are not the intended recipient, or a person responsible for delivering this to the intended recipient, you are not authorized to and must not disclose, copy, distribute, or retain this message or any part of it. In this case, please notify the sender immediately at 713-333-5440
Christopher Samuel via slurm-users
> What I'm struggling with now is running Podman under Slurm itself on our
> compute nodes.
run from the job_container/tmpfs plugin in order to stop jobs from the
same user on a node using podman (via podman-hpc) trashing each other.
The details (including the script and config) are in our public support
ticket where I was flailing around looking for how to do this with
CloneNSScript here: https://support.schedmd.com/show_bug.cgi?id=23228
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Berkeley, CA, USA
John Snowdon via slurm-users
Sent: 06 September 2025 17:14
To: slurm...@lists.schedmd.com <slurm...@lists.schedmd.com>
Subject: [slurm-users] Re: Creating /run/user/$UID - for Podman runtime
> What I'm struggling with now is running Podman under Slurm itself on our
> compute nodes.
We found that we had to make /run/user/$UID private per job via a script
run from the job_container/tmpfs plugin in order to stop jobs from the
same user on a node using podman (via podman-hpc) trashing each other.
The details (including the script and config) are in our public support
ticket where I was flailing around looking for how to do this with
All the best,
Chris
--