[slurm-users] Build 22.05.8 for Debian Trixie?

[Steffen Grunewald via slurm-users]

Good afternoon,

we're still running our HPC cluster on Debian 12 Bookworm, which comes with
Slurm 22.05.8 - no issues so far.
Yesterday, another machine was added as a submit node (slurm-client only),
with Debian 13 Trixie / Slurm 24.11.5.
Now on that the commands (sinfo etc.) don't work and the slurmctld logs
"Incompatible versions of client and server code".
Since upgrading the whole cluster is not an option yet: Has someone attempted
(and possibly succeeded) to build 22.05.8 (or 23.11.4? which IIRC would be
the last compatible one) for Trixie, and would be willing to share ideas?

Thanks,
Steffen

--
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~

--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com

[Rikka Göring via slurm-users]

Hi Steffen,

that error is expected with a Slurm 24.11 client talking to a 22.05 controller — the RPC/protocol versions are simply too far apart. SchedMD generally recommends keeping Slurm components reasonably close in version, and upgrading the controller + client commands together (or controller first).

If upgrading the cluster isn’t an option yet, the most practical workaround is to run matching 22.05.x client tools on the Debian 13 submit node (e.g., build/package Slurm 22.05.8 from source for Trixie, or use a small Bookworm chroot/container with slurm-client). In general, I’d expect you’ll need client commands that match the controller’s major version as long as slurmctld stays on 22.05.

Best regards,
Rikka Göring (GenericRikka)

Maintainer, sysutils/slurm-wlm
FreeBSD HPC Modernization Project (unofficial)
https://kavocado.net

---

Von: Steffen Grunewald via slurm-users <slurm...@lists.schedmd.com>
Gesendet: Freitag, 12. Dezember 2025 14:02
An: Slurm users <slurm...@lists.schedmd.com>
Betreff: [slurm-users] Build 22.05.8 for Debian Trixie?

 

[Steffen Grunewald via slurm-users]

Hi,

On Fri, 2025-12-12 at 14:16:25 +0000, Slurm users wrote:
> Hi Steffen,
>
> that error is expected with a Slurm 24.11 client talking to a 22.05 controller

that's why I'm looking for some 23.11 which in theory would be inside the
compatibility window...

> If upgrading the cluster isn’t an option yet, the most practical workaround
is to run matching 22.05.x client tools on the Debian 13 submit node (e.g.,
build/package Slurm 22.05.8 from source for Trixie

This is where I'm constantly failing, for multiple reasons: the *.so libraries
are installed 0555 instead of 0755 which they had been set before, perl versions
aren't properly detected (perl/5.40.1 vs perl5/5.40) etc.

, or use a small Bookworm chroot/container with slurm-client)

This indeed might make sense but is too user-unfriendly, I'm afraid.

. In general, I’d expect you’ll need client commands that match the controller’s major version as long as slurmctld stays on 22.05.

For the theory part above, I found some 23.11.4 backport attempt but even that one
throws an error "slurm_load_partitions: Unexpected message received" with server-
side "Invalid Protocol Version 10240" (the 24.11 one was 10752, btw).

So this makes me believe that the server version must be >= the client one, despite
windows usually being transparent from both sides - and I must find a way to build
22.05 slurm-client, slurm-wlm-basic-plugins and slurm-wlm-jwt-plugin packages for
Trixie; or have to wait until the controller can be upgraded. (I hope to keep the
DB; there's an intermediate 23.11.7-1 build that might still fit trixie for this.
That's a different story though.)

Maybe I'd "just" build the binaries within Trixie? (I'm so used to packages, and
packaging, but this is a single point of failure...) In theory that should work,
and I don't need everything now...


Thanks, S

[Christopher Samuel via slurm-users]

On 12/15/25 11:14 am, Steffen Grunewald via slurm-users wrote:

> that's why I'm looking for some 23.11 which in theory would be inside the
> compatibility window...

Problem is that for versions:

compute <= slurmctld <= slurmdbd

so your compute nodes need to be the same as (or 2 versions earlier)
than slurmctld - but not any later version.

--
Chris Samuel : http://www.csamuel.org/ : Philadelphia, PA, USA

[Steffen Grunewald via slurm-users]

Good morning,

On Mon, 2025-12-15 at 12:18:02 -0500, Slurm users wrote:
> On 12/15/25 11:14 am, Steffen Grunewald via slurm-users wrote:
>
> > that's why I'm looking for some 23.11 which in theory would be inside the
> > compatibility window...
>
> Problem is that for versions:
>
> compute <= slurmctld <= slurmdbd
>
> so your compute nodes need to be the same as (or 2 versions earlier) than
> slurmctld - but not any later version.

Thanks, that clarifies things that aren't expressed as clearly in the
context of the compat window itself - the information although is there
in the upgrade order instructions.

Since we were upgrading from Bookworm, we put the 22.05 packages on hold
and kept all available fingers crossed, and it seems that Trixie can live
with the old packages.

Whether we'll be equally lucky when upgrading the controller node (we need
to use 23.11 as an intermediate, and aren't quite sure whether we should
run that in Bookworm or Trixie) - time will have to tell.

Thanks so far,

Steffen