[slurm-users] Alocating a subset cores to each job

[Nadav Toledo]

Hello everyone,
Sorry for might be a trivial question for most of you.

I am trying to understand cpu allocation in slurm.
The goal is to launch a batch job on one node. while the batch itself will run several jobs in parallel each allocated a subset of the cpus given to the batch.

We've tried the following:

#!/bin/bash
#SBATCH --nodelist=server5
#SBATCH --sockets-per-node=2
#SBATCH --cores-per-socket=10
nproc &
srun  --ntasks=1 --cpus-per-task=2  echo "nproc is `nproc` " &
srun  --ntasks=1 --cpus-per-task=4  echo "nproc2 is `nproc` " &
wait

result is:
20
nproc is 20
nproc2 is 20

(lines order might be different, it doen't matter at this point)

desired output is:
20
nproc is 2
nproc2 is 4
(lines order might be different, it doen't matter at this point)

So the quetsion is:
How can one send a few jobs running in parallel with different cpus allocation on the same node?

Thanks in advance, Nadav

[Kilian Cavalotti]

Hi Nadav,

On Tue, Jun 12, 2018 at 8:18 AM, Nadav Toledo
<nadav...@cs.technion.ac.il> wrote:
> How can one send a few jobs running in parallel with different cpus
> allocation on the same node?

According to https://slurm.schedmd.com/srun.html#OPT_cpu-bind, you may
want to use "srun --exclusive":

By default, a job step has access to every CPU allocated to the
job. To ensure that distinct CPUs are allocated to each job step, use
the --exclusive option.

I agree that the option could be named better, especially since it has
a completely different meaning when used with sbatch, for instance.

Cheers,
--
Kilian