[slurm-users] Add prolog and epilog to sbatch's job

[Franck HOUSSEN]

Hello,

How to add an epilog to the job.log I got from running "sbatch -o job.log job.sh" ?
I would like to run "scontrol show job $SLURM_JOB_ID" in the epilog (track job info).

I tried to play with Epilog and TaskEpilog in slurm.conf: but seems not to work.
Are these the correct ones ? Or did I misunderstood something along the way ?

Franck

Note: I got what I expected with srun and SrunEpilog. Would like to get the same with sbatch.

>> more sbatch_epilog.sh
#!/bin/bash
#
# Slurm epilog

echo ""
echo "==============SLURM===EPILOG=============="
SLURM_JOB_INFO="$(/path/to/scontrol show job "$SLURM_JOB_ID")"
echo "print scontrol show job $SLURM_JOB_ID"
echo "print $SLURM_JOB_INFO"
echo "=========================================="

[Bjørn-Helge Mevik]

I don't remember exactly when it comes to TaskEpilog, but the Epilog
does not put its output into the job's --output or --error file(s).
Hopefully, SchedMD will add an easy way for prologs/epilogs to write to
the job's file someday.

In the mean time, we use a dirty hack for it. Basically, our Epilog
script uses information from "scontrol show job" to get the name of the
--output file, and redirects output directly to it. (If the file
doesn't yet exist, the epilog must create it, with correct ownerships
and permissions, and depending on your setup (e.g., rootsquash), you
have to su to the job user first. I told you it was a dirty hack. :)

--
Regards,
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo

[Hadrian Djohari]

We put this exact function in epilogctld

--

Hadrian Djohari
Manager of Research Computing Services, [U]Tech
Case Western Reserve University
(W): 216-368-0395
(M): 216-798-7490