Can we still customize/update compute images?

[Woonki Chung]

Hi,

I have just tried the latest version 4.0.3. One of the main differences from the previous version 3 is that it doesn't create compute images any more under both "VM Instances" and "Images". Could you please explain how it works now?

I used to update those images to make any changes, to add new packages, or just to update the OS before, while the cluster is running. How can I do such things in the current version? 

Also, is there any easy way to add packages to the compute images while setting up the cluster initially? In the previous version, I could edit scripts/startup.sh file to add any packages.

Thanks,
--

Woonki Chung, Ph.D.

HPC Program Manager

University Information Services

Georgetown University

[Joseph Schoonover]

Hey Dr. Chung,

While I'm not a developer of schedmd/slurm-gcp, I can recommend a strategy that has worked well for me. You can "bake" vm images on top of the schedmd images and use those in the partitions.image metadata.

The image baking is done with Google Cloud build and Packer.

You can see examples of how this works in action at 
https://github.com/FluidNumerics/hpc-apps-gcp

For example, see the wrf image baking scripts at : https://github.com/FluidNumerics/hpc-apps-gcp/tree/main/wrf

And the slurm-gcp deployment here : https://github.com/FluidNumerics/hpc-apps-gcp/tree/main/wrf/tf/slurm

Check out the wrf example 

--
You received this message because you are subscribed to the Google Groups "google-cloud-slurm-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cloud-slurm-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-slurm-discuss/CAMYSRGJ9hsdbg6Lz6rM5EO8mVVF0PUhvcT5GosB%3DjdrvgpzVnQ%40mail.gmail.com.

[Wyatt Gorman]

Hi Woonki,

Happy to have a discussion about this directly, but as of the latest version of Slurm on GCP, the images are pre-built and hosted by SchedMD, and are used directly without the requirement to make build an image at deployment time. You can continue to modify the image by creating your own image. This is done by spinning up a VM with the public image pointed to in the tfvars, making your modifications, saving the image, and pointing your tfvars to your new image.

You can continue to use the custom compute/controller install scripts found in the scripts directory.

Again, happy to have a call to discuss.

Wyatt Gorman HPC Solutions Manager https://cloud.google.com/hpc

To view this discussion on the web visit https://groups.google.com/d/msgid/google-cloud-slurm-discuss/CA%2B1e_JWBuPM%3Da2DwgAmZfHTQdySgGGdR1MQfPYFoiZ8QhAafUw%40mail.gmail.com.

[Woonki Chung]

Hi Joseph,

Thank you for the info. For now, creating an image through a VM with gcloud commands is good enough for me.. Thanks

On Mon, Jun 14, 2021 at 11:26 PM Joseph Schoonover <j...@fluidnumerics.com> wrote:

[Woonki Chung]

Hi Wyatt,

Glad to hear from you again. Yes, it makes sense and it looks cleaner in fact. 

I just tried creating a customized image from the schedmd-slurm one and deploying a cluster from the image. It seems to be working well as expected. I was able to add many packages to the image as well. When I updated an image again into the image family as usual, new compute nodes got updated accordingly. So it seems to work well as before.

Thank you for your quick response and for working on this great project! ;)