Dask on Slurm GCP

[Will]

Hello,

I would like to run Dask on my slurm google cloud cluster using the dask-jobqueue (to communicate with slurm), however, I could not find documentation Dask slurm gcp so I was wondering before installing and configuring Dask on my cluster if google cloud slurm is compatible with Dask.

More specifically, would Slurm GCP be able to create the machines, adjust the need of machines, and destroy them once the computation is done if it is directed by a Dask script?

Thanks a lot in advance for any advice or documentation.

Best,

Will

 

[Joseph Schoonover]

Hey Will,

I haven't tried this before, but the Dask documentation suggests that its compatible with a number of common HPC job schedulers, including Slurm.

Have you come across this documentation before : https://jobqueue.dask.org/en/latest/examples.html#slurm-deployments   ?

The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.

Dr. Joseph Schoonover Chief Executive Officer Senior Research Software Engineer j...@fluidnumerics.com

--
You received this message because you are subscribed to the Google Groups "google-cloud-slurm-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-slurm-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-slurm-discuss/7da8e12d-d504-4857-a014-f235d0e9640fn%40googlegroups.com.

[Joseph Schoonover]

To answer you latter question, slurm-gcp will handle compute node creation and deletion (autoscaling) for you when jobs are submitted to Slurm.

[William]

Hi Joseph,

Thanks a lot for your help and advice (again :) ) 

I saw Dask can run Slurm but I did not know if it makes a difference if it is slurm or slurm-gcp, and did not find documentation specifically for slurm-gcp. 

For instance, should I install Dask on the login or controller node?

Does queue represent the partitions in the cluster function of Dask? 

I'll give it a try and let you know how it goes :)

Thanks again!

Best,

Guilhem

[Joseph Schoonover]

Hey William,

To submit to a particular partition, you could try using the queue variable - you'll have to dig around the dask documentation a bit more to find out. Of course, if you're ok with experimenting to figure this out, if the queue option fails, you can use the extra option to specify the partition.

cluster = SLURMCluster(memory='8g',
                       processes=1,
                       cores=2,

                       extra=['--partition=my-partition']) 

Since Dask is a python package, you can use pip (or pip3) to install. From what I can tell, the job-queue package is not needed on the compute nodes (though I could be wrong).

pip3 install --user dask-jobqueue

If dask and other dependencies are needed on compute nodes, the --user flag should install the packages in your home directory, making them accessible on compute nodes. I'm interested to hear how this goes

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-slurm-discuss/0682a84f-5abc-474e-966c-a75ec0014b33n%40googlegroups.com.