[slurm-users] Slurm Environment Variable for Memory

[Juan A. Cordero Varelaq]

Dear Community,

does anyone know whether there is an environment variable, such as $SLURM_CPUS_ON_NODE, but for the requested RAM (by using --mem argument)?

Thanks

[Bjørn-Helge Mevik]

Yes. It is documented in sbatch(1):

SLURM_MEM_PER_CPU
Same as --mem-per-cpu

SLURM_MEM_PER_NODE
Same as --mem

--
Regards,
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo

[Juan A. Cordero Varelaq]

That variable does not exist somehow on my environment. Is it possible
my Slurm version (17.02.3) does not include it?

Thanks

[Bjørn-Helge Mevik]

"Juan A. Cordero Varelaq" <bioinform...@us.es> writes:

> That variable does not exist somehow on my environment. Is it possible my
> Slurm version (17.02.3) does not include it?

It's documented in our version, 17.02.7, and I've verified that they get
set in our systems, at least.

[Chris Samuel]

On Monday, 20 August 2018 4:43:57 PM AEST Juan A. Cordero Varelaq wrote:

> That variable does not exist somehow on my environment. Is it possible
> my Slurm version (17.02.3) does not include it?

They should be there, from the NEWS file they were introduced in 2.3.0.rc1.
Is something else nuking your shells environment perhaps?

17.02.11 is the last released version of 17.02.x and all previous versions
have been pulled from the SchedMD website due to CVE-2018-10995.

cheers,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC

[Juan A. Cordero Varelaq]

I am just running an interactive job with "srun -I --pty /bin/bash" and
then run "echo $SLURM_MEM_PER_NODE", but it shows nothing. Does it have
to be defined in any conf file?

[Williams, Gareth (IM&T, Clayton)]

I don't think it is what you want anyway.  They are probably set if you request memory specifically. It is not how much memory the node has.

I think many sites set a default memory per core (we do).

Gareth

Get Outlook for Android

---

From: slurm-users <slurm-use...@lists.schedmd.com> on behalf of Juan A. Cordero Varelaq <bioinform...@us.es>
Sent: Monday, August 20, 2018 9:21:39 PM
To: slurm...@lists.schedmd.com
Subject: Re: [slurm-users] Slurm Environment Variable for Memory

 

[Bjørn-Helge Mevik]

"Juan A. Cordero Varelaq" <bioinform...@us.es> writes:

> I am just running an interactive job with "srun -I --pty /bin/bash" and then
> run "echo $SLURM_MEM_PER_NODE", but it shows nothing. Does it have to be
> defined in any conf file?

If you read srun(1) or sbatch(1), you will see that the SLURM_MEM_*
variables correspond to what you have specified with the corresponding
--mem* switches. If you haven't specified the switches, there is no
value to set the variables to (perhaps srun/sbatch use the DefMemPer*
parametres from slurm.conf if they are set, I haven't checked).

[Chris Samuel]

On Monday, 20 August 2018 9:21:39 PM AEST Juan A. Cordero Varelaq wrote:

> I am just running an interactive job with "srun -I --pty /bin/bash" and
> then run "echo $SLURM_MEM_PER_NODE", but it shows nothing. Does it have
> to be defined in any conf file?

My apologies - I've just tested here (with Slurm 17.11.7) and you are indeed
correct, they only appear when launched with sbatch and salloc and not when
you launch jobs directly with srun!

Also you only ever get ${SLURM_MEM_PER_CPU} *or* ${SLURM_MEM_PER_NODE} but not
both together, so you'll need to check for both.

Hope this helps!

All the best,

[Chris Samuel]

On Tuesday, 21 August 2018 6:17:59 PM AEST Chris Samuel wrote:

> My apologies - I've just tested here (with Slurm 17.11.7) and you are indeed
> correct, they only appear when launched with sbatch and salloc and not when
> you launch jobs directly with srun!

I think the confusion is because they are *input* environment variables to
srun, so srun consumes them. It doesn't create them. I hadn't noticed this
until I search back from when I found them in the manual page and saw they
were listed as:

INPUT ENVIRONMENT VARIABLES
Some srun options may be set via environment variables. These environ‐
ment variables, along with their corresponding options, are listed
below. Note: Command line options will always override these settings.


Which is why they are set by sbatch/salloc but not directly inside of srun.

All the best!