[slurm-users] launch failed requeued held

[sportlecon sportlecon via slurm-users]

slurm 24.11 - squeue displays reason "launch failed requeued held"

--
slurm-users mailing list -- slurm...@lists.schedmd.com
To unsubscribe send an email to slurm-us...@lists.schedmd.com

[John Hearns via slurm-users]

You need to find the node which the job started on.

Then look at the slurmd log on that node. You may find an indication of the reason for the failure.

[John Hearns via slurm-users]

Generally, the troubleshooting steps which you should take for Slurm are:

squeue to look at the list of running/queued or held jobs

sinfo to show which nodes are idle, busy or down

scontrol show node  to get more detailed information on a node

For problem nodes - indeed just log into any node to see what a healthy node looks like
systemctl status slurmd
cat /var/log/slurm/slurmd.log

On your slurm controller look at the slurmctld and slurmdbd logs

On Tue, 7 Jan 2025 at 11:30, sportlecon sportlecon via slurm-users <slurm...@lists.schedmd.com> wrote: