[slurm-users] JobComp file not rotating

[Stuart Barkley]

I am using:

JobCompType=jobcomp/filetxt
JobCompLoc=/var/log/slurm/jobcomp.log

This file is not being reopened on SIGUSR2 which makes rotating the
log file difficult. slurmctld(8) says SIGUSR2 should be used for
logrotate but only mentions the main slurmctld log file.

Using SIGHUP does reopen the JobComp log file but also reloads the
full slurm configuration file.

I've become cautious about restarting slurmctld and rereading the
configuration file because a number of configuration errors can cause
slurmctld to stop running. Yes, I know I should reload the
configuration file immediately after making changes and make sure
slurmctld restarts correctly but I do occasionally forget.

We are just getting our feet wet with Slurm so I want the JobComp log
in addition to the accounting database. Getting the accounting
database setup correctly is going to take additional time and I want
to ensure we have at least some information about jobs that have run.

Stuart
--
I've never been lost; I was once bewildered for three days, but never lost!
-- Daniel Boone

[Paul Brunk]

Hi Stuart:

 

I've long seen similar on my server.  If I don't intervene then I don't really get a per-day completed job log, but rather some empty ones and some that span days.  I can live with this until I do something about it--the job completion entries are all there.  I just can't infer from the log file names (date stamps) which completed job log a given day's jobs will appear in.

 

==

Paul Brunk, system administrator

Georgia Advanced Resource Computing Center

Enterprise IT Svcs, the University of Georgia