[slurm-users] MUNGE security issue (CVE-2026-25506)

120 views
Skip to first unread message

Tim Wickberg via slurm-users

unread,
Feb 10, 2026, 1:06:23 PM (12 days ago) Feb 10
to slurm...@lists.schedmd.com, slurm-a...@schedmd.com
MUNGE just announced a serious security issue as CVE-2026-25506:

https://github.com/dun/munge/security/advisories/GHSA-r9cr-jf4v-75gh

The issue is entirely within MUNGE - specifically in the munged daemon.
However, as most Slurm installations explicitly trust MUNGE as the
source of authentication, Slurm's security is immediately undermined if
an attacker were to exploit that issue and gain access to the MUNGE key.
On Slurm clusters you should treat that security issue as if it were
local-root exploit.

If you have switched to Slurm's built-in authentication ("auth/slurm"),
you are not impacted by this issue. (However, we do not recommend
switching over as a mitigation - you cannot safely change authentication
types in Slurm while jobs are currently running.)

I strongly encourage sites to get install the fixed packages from the
upstream distros and restart the munged daemon on all nodes. If you
install and restart munged promptly then Slurm should be unaffected.

You do not need to rebuild Slurm for this issue. The MUNGE ABI/API
remain unchanged, so there is no need to recompile Slurm, and there are
no patches to apply to Slurm related to this issue.

If you are concerned that an attacker could have compromised the MUNGE
key itself, you may want to rotate that key. Unfortunately there's no
mechanism to safely handle this key rotation with the cluster
operational, and I do not recommend key rotation as long as you're
installing and restarting MUNGE ASAP.

- Tim

--
Tim Wickberg
Director, System Software, Slurm and Slinky, NVIDIA

--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com

Ole Holm Nielsen via slurm-users

unread,
Feb 10, 2026, 4:01:19 PM (12 days ago) Feb 10
to slurm...@lists.schedmd.com
On 2/10/2026 7:04 PM, Tim Wickberg via slurm-users wrote:
> MUNGE just announced a serious security issue as CVE-2026-25506:

Just a small note on Munge [1] for those of you with RPM-based systems:

Build Munge 0.5.18 RPM packages by:

wget
https://github.com/dun/munge/releases/download/munge-0.5.18/munge-0.5.18.tar.xz

rpmbuild -ta munge-0.5.18.tar.xz

and install the packages from the directory ~/rpmbuild/RPMS/x86_64/ (or
whatever your architecture is).

Best regards,
Ole

[1] https://github.com/dun/munge/releases
--
Ole Holm Nielsen
PhD, Senior HPC Officer
Department of Physics, Technical University of Denmark

Sean Mc Grath via slurm-users

unread,
Feb 11, 2026, 5:27:00 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com, Ole Holm Nielsen
FYI, (which others probably already know). Munge needs to be updated on the slurmctld node(s) before being updated on the slurmd nodes in my limited testing. Similar to how slurm is updated. Updating munge on a slurmd node before the slurmctld caused errors on the slurmd node for the one instance I did that.

---
Sean McGrath

Systems Administrator
Research IT
IT Services
Trinity College Dublin

00 353 - (0)1 896 3725
https://www.tcd.ie/itservices/
https://www.tchpc.tcd.ie/

From: Ole Holm Nielsen via slurm-users <slurm...@lists.schedmd.com>
Sent: Tuesday 10 February 2026 20:59
To: slurm...@lists.schedmd.com <slurm...@lists.schedmd.com>
Subject: [slurm-users] Re: MUNGE security issue (CVE-2026-25506)
 
[External Email] This email originated outside of Trinity College Dublin. Do not click links or open attachments unless you recognise the sender and know the content is safe.

Paul Edmon via slurm-users

unread,
Feb 11, 2026, 5:32:33 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com

Interesting. That was not my experience, unless you were also rotating the keys or moving from a much older version. We were upgrading from 0.5.15 -> 0.5.18 and simply upgrading and restarting munged was sufficient and caused no issue.  That said we have not rotated our munge keys but will be doing so at our next regular maintenance which is known to be disruptive.


-Paul Edmon-

william--- via slurm-users

unread,
Feb 11, 2026, 5:39:52 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com
In passing, this does not work on CentOS 7 (don't ask...)

$ rpmbuild -ta --clean munge-0.5.18.tar.xz
error: Failed build dependencies:
systemd-rpm-macros is needed by munge-0.5.18-1.el7.x86_64

>From brief reading, the required macro is provided by the system package on
CentOS 7 but munge has an explicit check. As CentOS 7 isn't supported, it's
just another reason to get rid of it.

On my Rocky Linux 9/10 systems this worked fine (with appropriate el9/el10),
pushed from Foreman:

munged --version
rpm -Uvh /home/apps/slurm/munge/munge-libs-0.5.18-1.el10.x86_64.rpm
/home/apps/slurm/munge/munge-0.5.18-1.el10.x86_64.rpm
munged --version
systemctl restart munge

William


-----Original Message-----
From: Ole Holm Nielsen via slurm-users <slurm...@lists.schedmd.com>
Sent: 10 February 2026 20:59
To: slurm...@lists.schedmd.com
Subject: [slurm-users] Re: MUNGE security issue (CVE-2026-25506)

On 2/10/2026 7:04 PM, Tim Wickberg via slurm-users wrote:
> MUNGE just announced a serious security issue as CVE-2026-25506:

Just a small note on Munge [1] for those of you with RPM-based systems:

Build Munge 0.5.18 RPM packages by:

wget
https://github.com/dun/munge/releases/download/munge-0.5.18/munge-0.5.18.tar
.xz

rpmbuild -ta munge-0.5.18.tar.xz

and install the packages from the directory ~/rpmbuild/RPMS/x86_64/ (or
whatever your architecture is).

Best regards,
Ole

[1] https://github.com/dun/munge/releases
--
Ole Holm Nielsen
PhD, Senior HPC Officer
Department of Physics, Technical University of Denmark

--
slurm-users mailing list -- mailto:slurm...@lists.schedmd.com To
unsubscribe send an email to mailto:slurm-us...@lists.schedmd.com

Hagdorn, Magnus via slurm-users

unread,
Feb 11, 2026, 5:41:33 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com
On Wed, 2026-02-11 at 05:30 -0500, Paul Edmon via slurm-users wrote:
>  
> Interesting. That was not my experience, unless you were also
> rotating the keys or moving from a much older version. We were
> upgrading from 0.5.15 -> 0.5.18 and simply upgrading and restarting
> munged was sufficient and caused no issue.  That said we have not
> rotated our munge keys but will be doing so at our next regular
> maintenance which is known to be disruptive.
>  
the same here - we upgraded on the compute nodes first. it seems to
have just worked.
Regards
magnus

--
Dr. Magnus Hagdorn
Charité – Universitätsmedizin Berlin
Geschäftsbereich IT | Scientific Computing
 
Campus Charité Mitte
BALTIC - Invalidenstraße 120/121
10115 Berlin
 
magnus....@charite.de
https://www.charite.de
HPC Helpdesk: sc-hpc-...@charite.de

Ole Holm Nielsen via slurm-users

unread,
Feb 11, 2026, 5:52:17 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com, Sean Mc Grath
Hi Sean,

On 2/11/26 11:24, Sean Mc Grath wrote:
> FYI, (which others probably already know). Munge needs to be updated on
> the slurmctld node(s) before being updated on the slurmd nodes in my
> limited testing. Similar to how slurm is updated. Updating munge on a
> slurmd node before the slurmctld caused errors on the slurmd node for the
> one instance I did that.

I'm surprised by this experience. As long as you stay with the Munge
0.5.XX versions, I would think that Munge's protocols are interoperable
between minor versions - assuming that you don't rotate the Munge key as
explained in Tim's mail.

The Munge Release notes [2] don't seem to mention issues with upgrading,
and Slurm isn't mentioned at all.

Maybe someone can correct me here, and I'd be happy to add a correction to
my Slurm Wiki page on the topic of Munge [1].

I upgraded Munge on-the-fly from 0.5.17 to 0.5.18 yesterday in the order
login-nodes, slurmctld, slurmdbd, slurmd's, and we haven't encountered any
issues.

Best regards,
Ole

[1]
https://wiki.fysik.dtu.dk/Niflheim_system/Slurm_installation/#install-the-latest-munge-version
[2] https://github.com/dun/munge/releases

Sean Mc Grath via slurm-users

unread,
Feb 11, 2026, 5:56:16 AM (11 days ago) Feb 11
to Ole Holm Nielsen, slurm...@lists.schedmd.com
I think my advice to upgrade the slurmctld nodes first was wrong. Sorry about that. Please disregard it. Subsequent upgrades I have been performing don't seem to have the same issue. It was probably a false positive from the first test instance I upgraded.

Not rotating keys here and upgrading from munge-0.5.13.




From: Ole Holm Nielsen <Ole.H....@fysik.dtu.dk>
Sent: Wednesday 11 February 2026 10:50
To: slurm...@lists.schedmd.com <slurm...@lists.schedmd.com>
Cc: Sean Mc Grath <smc...@tcd.ie>
Subject: Re: [slurm-users] Re: MUNGE security issue (CVE-2026-25506)
 
[External Email] This email originated outside of Trinity College Dublin. Do not click links or open attachments unless you recognise the sender and know the content is safe.

Sebastian Sitkiewicz via slurm-users

unread,
Feb 11, 2026, 5:56:45 AM (11 days ago) Feb 11
to Hagdorn, Magnus, slurm...@lists.schedmd.com
Hello community,

the same here, worked fine for updating on slurmd nodes first (munge-0.5.13 -> munge-0.5.18)

Cheers,
Sebastian
----- Original Message ----

--
dr inż. Sebastian Sitkiewicz 

Politechnika Wrocławska 
Wrocławskie Centrum Sieciowo-Superkomputerowe
Dział Usług Obliczeniowych
Wyb. Wyspiańskiego 27
50-370 Wrocław 

Ole Holm Nielsen via slurm-users

unread,
Feb 11, 2026, 6:05:12 AM (11 days ago) Feb 11
to slurm...@lists.schedmd.com
On 2/11/26 11:38, william--- via slurm-users wrote:
> In passing, this does not work on CentOS 7 (don't ask...)
>
> $ rpmbuild -ta --clean munge-0.5.18.tar.xz
> error: Failed build dependencies:
> systemd-rpm-macros is needed by munge-0.5.18-1.el7.x86_64
>
>>From brief reading, the required macro is provided by the system package on
> CentOS 7 but munge has an explicit check. As CentOS 7 isn't supported, it's
> just another reason to get rid of it.

FWIW, Munge 0.5.16 [1] was tested on CentOS Linux Stream 9, Stream 8,
7.9.2009, 6.10
Maybe later Munge versions silently broke on CentOS 7 since there is no
way to download the old RPMs and test it?

Best regards,
Ole

[1] https://github.com/dun/munge/releases/tag/munge-0.5.16

Laura Hild via slurm-users

unread,
Feb 11, 2026, 9:13:37 AM (11 days ago) Feb 11
to wil...@signalbox.org.uk, william....@gmail.com, slurm-users
> In passing, this does not work on CentOS 7 (don't ask...)

How old of CentOS 7? I know someone who couldn't compile it on 7.4 but could compile it on 7.7 and then run it on 7.4. This was using the Fedora specfile, with the Version bumped and the libmunge.so.2.0.0 %file changed to .2.0.1. 🤷‍♀️

william--- via slurm-users

unread,
Feb 11, 2026, 12:40:13 PM (11 days ago) Feb 11
to Laura Hild, slurm-users
There are quite a few differences in the spec file for 0.5.16 (which does build on CentOS 7) and 0.5.18 (which does not).

0.5.16 spec file does say that it supports 7.9.2009, and the 0.5.18 does not.

A key difference is that the older spec file has:
BuildRequires: %{?el7:systemd}%{!?el7:systemd-rpm-macros}
And the new:
BuildRequires: systemd-rpm-macros

That is enough to cause the rpmbuild to fail.

I have found that just by editing that one line in the spec file to go back to the version from 0.5.16, it builds OK on our CentOS 7.9.2009. However it fails to install:

# rpm -Uvh /home/apps/slurm/munge/munge-0.5.18-1.el7.x86_64.rpm /home/apps/slurm/munge/munge-libs-0.5.18-1.el7.x86_64.rpm
error: Failed dependencies:
/usr/bin/systemd-sysusers is needed by munge-0.5.18-1.el7.x86_64
munge-libs(x86-64) = 0.5.13-1.el7 is needed by (installed) munge-devel-0.5.13-1.el7.x86_64

A bit of reading shows that system-sysusers does not exist for CentOS 7. So if I read it correctly your suggestion is to use the spec file from 0.5.16 and edit these lines:

< Version: 0.5.16
---
> Version: 0.5.18

< %{_libdir}/libmunge.so.2.0.0
---
> %{_libdir}/libmunge.so.2.0.1

I tried that and it failed to find a file:
install: cannot stat 'src/etc/munge.tmpfiles.conf': No such file or directory

This file is created in 0.5.16 but not in 0.5.18.

Overall I am curious how some people have been able to build it when there do seem to be multiple issues.

William

Laura Hild via slurm-users

unread,
Feb 11, 2026, 12:53:56 PM (11 days ago) Feb 11
to wil...@signalbox.org.uk, slurm-users
> So if I read it correctly your suggestion is to use the spec file from 0.5.16 and edit these lines:

I meant use Fedora's, like from https://src.fedoraproject.org/rpms/munge, not the spec file from upstream.

chris.m.dunlap--- via slurm-users

unread,
Feb 11, 2026, 10:06:20 PM (11 days ago) Feb 11
to slurm...@lists.schedmd.com
Support for CentOS 7 was dropped from the specfile for 0.5.17 since EL7 was EOL.

I've added a "munge-0.5.18-el7.spec" to the release page:
https://github.com/dun/munge/releases/tag/munge-0.5.18

Directions for building with it are towards the bottom of the announcement.

-Chris

Timony, Mick via slurm-users

unread,
Feb 12, 2026, 12:53:11 PM (10 days ago) Feb 12
to slurm-users
Hi All,

If I am not updated the munge key is it possible to install a newer munge package without pausing the cluster (marking partitions as down and suspending jobs)? 

I am concerned about job failures, especially on GPU nodes if I need to pause jobs. 


Thanks

--
Mick Timony
Senior DevOps Engineer
LASER, Longwood, & O2 Cluster Admin
Harvard Medical School
--

From: william--- via slurm-users <slurm...@lists.schedmd.com>
Sent: Wednesday, February 11, 2026 12:38 PM
To: 'Laura Hild' <l...@jlab.org>
Cc: 'slurm-users' <slurm...@lists.schedmd.com>
Subject: [slurm-users] Re: MUNGE security issue (CVE-2026-25506)
 

Jason Simms via slurm-users

unread,
Feb 12, 2026, 1:06:47 PM (10 days ago) Feb 12
to Timony, Mick, slurm-users
We did the update yesterday and no need to pause. All went well. 

Jason L. Simms, Ph.D., M.P.H.
Research Computing Manager
Swarthmore College
Information Technology Services

Ole Holm Nielsen via slurm-users

unread,
Feb 12, 2026, 1:44:05 PM (10 days ago) Feb 12
to slurm...@lists.schedmd.com
On 2/12/2026 6:50 PM, Timony, Mick via slurm-users wrote:
> If I am not updated the munge key is it possible to install a newer
> munge package without pausing the cluster (marking partitions as down
> and suspending jobs)?
>
> I am concerned about job failures, especially on GPU nodes if I need to
> pause jobs.

Yes, several people already wrote on this list that they upgraded the
Munge package on running clusters without any bad side effects on Slurm
jobs.

/Ole
Reply all
Reply to author
Forward
0 new messages