[slurm-users] Nodes required for job are DOWN, DRAINED or reserved for jobs in higher priority partitions

[sportlecon sportlecon via slurm-users]

JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
26 cpu myscript user1 PD 0:00 4 (Nodes required for job are DOWN, DRAINED or reserved for jobs in higher priority partitions)
Anyone can help to fix this?

--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com

[John Hearns via slurm-users]

Output of sinfo and squeue

Look at slurmd log in an example node also

Tail -f is your friend 

[Brian Andrus via slurm-users]

Run 'sinfo -R' to see the reason any nodes may be down.

It may be as simple as running 'scontrol update state=resume nodename=xxxx' to bring them back, if they are down. It depends on the reason they went down (if that is the issue).

Otherwise, check the job requirements to see what it is asking for that does not exist 'scontrol show job xxx'

Brian Andrus

[Steffen Grunewald via slurm-users]

On Sat, 2025-01-04 at 08:11:21 -0000, Slurm users wrote:
> JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
> 26 cpu myscript user1 PD 0:00 4 (Nodes required for job are DOWN, DRAINED or reserved for jobs in higher priority partitions)
> Anyone can help to fix this?

Not without a little bit of extra information,
e.g. "sinfo -p cpu" and maybe "scontrol show job=26"

Best,
Steffen

--
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~