Warewulf v4 Features

[Maciek Dems]

Hi,

I have been using Warewulf 3 in my small HPC cluster environment and it works well. I have contributed a patch to use mdadm to create RAID0 for scratch (the path became outdated before it was merged, due to the problems with mdadm license, but I use it successfully in my production environment).

I consider switching to WW4 and contributing to it (currently I don’t know Go, but gimmie a week 😉). However, before I make my decision I would like to ask for a state of the following features:

• Is provision through Mellanox Infiniband working? Setting it up with WW3 was a pain in the…, but finally I managed to have it going. This is crucial for me.

• Is it possible to create a RAID partition for stateless diskfull operation (I need several local disks on my nodes to work as a single large scratch space).

• How about management: users (I would like to use NIS), syncing some files (wwsh file ... in WW3) etc.

Regards,
Macie

[Gregory M. Kurtzer]

Hi Maciek,

Apologies for the latent reply, I've been a bit overwhelmed with emails recently.

1. I honestly don't know if PXE over Mellanox is working! Does Mellanox support iPXE? I'd assume it does, so I'd think it would work, but it hasn't been tested.

2. Right now, I've put no effort into stateful/diskful provisioning. The mechanism is there as can be found inside the /init file, as is the ability to define a root partition for a given node, but just no code there yet as it is still 100% stateless.

3. User management is pulling statically from the files on the master node, but Warewulf doesn't do a huge amount here as the files are generated from a full templating system. I'd suggest to write a script that does the needed getpw* systemcalling and persist those to a file that the Warewulf templates ingest for the node's /etc/passwd(.ww) within the runtime overlay.

Hopefully that helps and looking forward to having your help on the project!

Greg

--
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAPb13SEmy3PDyn%2BqfGFceL1GmH%2B_XQ1qs9-5eBopwDqnv41Jtg%40mail.gmail.com.

--

Gregory M. Kurtzer

CEO and Founder, CtrlCmd: http://ctrl-cmd.com

Executive Director, Next Generation of High Performance Computing: http://hpcng.org

[Maciek Dems]

Thanks for your answer.

1. Mellanox Infiniband can use IPXE, but it’s a bit tricky. WW3 supports this. I understand that no one tried it with WW4?

2. I am asking about the stateless system. But I need to use mdadm to create a RAID0 array and then format it. Doing this on every reboot is fine.

3. Actually I don’t want to touch the /etc/passwd on the nodes, but I have configured NIS to do the job.Actually I think that this is the correct approach for every cluster.

4. Another question: how to make a container in WW4? In WW3 it was based on the chroot and I like this, as it allows me to install all the software I need and provides hybridization, which is good to save the memory on the nodes. Is this still possible with WW4?

Best regards,
Maciek

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAApQTTjRmLVg2-QrK2foQzp%3Dcpm1D1_Q%2BLBrGLPz1vJqX-_CPA%40mail.gmail.com.

[Gregory M. Kurtzer]

Hi Maciek,

1. That should work just fine, but I haven't tested it. If it worked with WW3, I'd assume it would work with WW4 as it leverages a very similar PXE setup.

2. Ahhh, then this should be pretty easy. In the system overlay, check out the default system overlay and feel free to add a script to do that in /warewulf/init.d/

3. That can be done then by configuring each node to use LDAP. In my experience, that works for most "normal" scale clusters, but the bigger the system gets, the worse it does.

4a. You can make a chroot in exactly the same way as you did in WW3, just put it into /var/warewulf/${NAME}/rootfs

4b. Hybridization is not currently done by the VNFS in WW4, as I opted to just recommend swap on compute nodes. This was mostly necessitated by RedHat's breakage of the root file system and how critical components are now linking to /usr which breaks almost all useful hybridization. So at this point, the easier way to mitigate is just put everything in the VNFS, and if you have a local drive, enable swap and the VNFS tmpfs will be swapped by the kernel as applications need the system memory.

Greg

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAPb13SFikedQ4xhq3gfyS3yAkn8GHb5M-uKM47NVEqAbr20L7g%40mail.gmail.com.