How to update a partition ?

681 views
Skip to first unread message

Kartikey Sarode

unread,
Nov 2, 2021, 12:50:34 AM11/2/21
to google-cloud-slurm-discuss
Hi,

We have deployed a multi-partition Slurm cluster using https://github.com/SchedMD/slurm-gcp.

Now, we are looking to increase the maximum number of nodes in a partition from 1000 to 5000.
I'm unable to find anything specific in the documentation that will help me with the commands to do so.

One doc recommended the below approach:
1. Update the partition information using scontrol.
2. Restart slurmctld and slurmdbd in the controller.
3. Run "scontrol reconfigure".

Is the above approach correct or do I need to do something else ?

Thanks,
Kartikey

Alex Chekholko

unread,
Nov 2, 2021, 3:08:52 PM11/2/21
to Kartikey Sarode, google-cloud-slurm-discuss
Hi Kartikey,

Here is how I remember doing it with this software configuration:
1) drain the cluster so all the compute nodes get turned off
2) update the number of nodes and partition definition in slurm.conf
3) restart slurmctld on the controller

You're done!  The login node doesn't have running slurm daemons so nothing to change there, and the compute nodes mount the central slurm.conf, so you can't easily change it for running nodes and all new nodes have slurmd load the new config.

Regards,
Alex


--
You received this message because you are subscribed to the Google Groups "google-cloud-slurm-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-slurm-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-slurm-discuss/701685a0-94b3-43a8-9384-95fc40ec574an%40googlegroups.com.

Kartikey Sarode

unread,
Nov 3, 2021, 10:03:48 AM11/3/21
to google-cloud-slurm-discuss
Thanks Alex,

I will try this and update how it goes !

Regards,
Kartikey

Reply all
Reply to author
Forward
0 new messages