[slurm-users] JobState of RaisedSignal:53 Real-time_signal_19; slurm 23.02.4

[Robert Kudyba]

The user is launching a Singularity container for RStudio and the final option for --rsession-path does not exist.

scontrol show job 420719
JobId=420719 JobName=r2.sbatch
  UserId=ouruser(552199) GroupId=user(500) MCS_label=N/A
  Priority=1428 Nice=0 Account=ouracct QOS=xxx
  JobState=FAILED Reason=RaisedSignal:53(Real-time_signal_19) Dependency=(null)

From slurmctld.log:

[2023-11-10T11:40:20.569] _slurm_rpc_submit_batch_job: JobId=420719 InitPrio=1428 usec=272
[2023-11-10T11:40:20.973] sched: Allocate JobId=420719 NodeList=node001 #CPUs=2 Partition=xxx
[2023-11-10T11:40:21.143] _job_complete: JobId=420719 WTERMSIG 53
[2023-11-10T11:40:21.144] _job_complete: JobId=420719 done

I think I may know the reason but wanted to see if this error meant something else.Here is the snippet in the sbatch file:

singularity exec --cleanenv rstudio_4.2.sif \
  /usr/lib/rstudio-server/bin/rserver --www-port ${PORT} \
      --auth-none=0 \
      --auth-pam-helper-path=pam-helper \
      --auth-stay-signed-in-days=30 \
      --auth-timeout-minutes=0 \
      --rsession-path=/path/to/4.2/rsession.sh

Thanks.

[John Pellman]

At a glance my guess would be that the user needs to place a double dash (--) after rstudio_4.2.sif or that the command to be executed needs to be double-quoted.  Otherwise singularity might be trying to parse the rstudio flags as its own. 

 WTERMSIG 53 mostly just indicates that the rstudio process was killed with the SIGRTMAX-11 signal (not sure why exactly as I don't believe SLURM makes use of this signal).