WW 3.6.99, uEFI PXE Boot

Novosielski, Ryan

unread,

Oct 6, 2015, 6:46:14 PM10/6/15

to Warewulf

Anyone ever seen something like this before? It seems like it's a SYSLINUX thing, but 6.02 works even worse than the, I'm guessing, supplied 6.03 (file size matches). Some browsing on the internet seems to possibly implicate firmware, but to me that doesn't sound like the most likely thing. Server is a Lenovo nx360 if it matters.

>>Start PXE over IPv4.
Station IP address is 10.0.81.200

Server IP address is 10.0.81.101
NBP filename is /warewulf/syslinux64.efi
NBP filesize is 199952 Bytes
Downloading NBP file...

Succeed to download NBP file.

Downloading NBP file...

Succeed to download NBP file.
Getting cached packet
My IP is 0.0.0.0
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping
core_udp_sendto: stalling on configure with no mapping

...this continues forever.

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novo...@rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'

Novosielski, Ryan

unread,

Oct 7, 2015, 3:15:41 PM10/7/15

to ware...@lbl.gov

Is anyone able to confirm that at least they do have EFI booting working with Warewulf?

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novo...@rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'

________________________________________
From: Novosielski, Ryan [novo...@ca.rutgers.edu]
Sent: Tuesday, October 06, 2015 6:42 PM
To: Warewulf
Subject: [Warewulf] WW 3.6.99, uEFI PXE Boot

--
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
For more options, visit https://groups.google.com/a/lbl.gov/d/optout.

Gregory M. Kurtzer

unread,

Oct 7, 2015, 6:02:30 PM10/7/15

to Warewulf

I have tried to, but due to some known but unfixed bugs in Syslinux I stopped working on it. Your client output looked different from what I noticed but that would be client implementation specific, but it did seem to be failing at about the same point.

I have not been following up with it because we have been able to boot using legacy mode. Which actually turned out to be beneficial due to a side effect of one of our vendors has a very high memory footprint when booting via uEFI (on the order of hundreds of megabytes lost to the operating system).

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.

For more options, visit https://groups.google.com/a/lbl.gov/d/optout.

--

Gregory M. Kurtzer
Technical Lead and HPC Systems Architect
High Performance Computing Services (HPCS)
University of California
Lawrence Berkeley National Laboratory
One Cyclotron Road, Berkeley, CA 94720

Novosielski, Ryan

unread,

Oct 7, 2015, 7:45:43 PM10/7/15

to ware...@lbl.gov

It looks like the bugs are fixed now, but I can't figure out how to build a new syslinux64.efi successfully from the git respository. There hasn't been a release which includes the alleged fixes.

At any rate, I've switched to legacy boot which is using BIOS PXE I guess and now things are working. That method has its own problems on my platform (no console between seeing the Ethernet BIOS banners and Warewulf loading), but when you have Warewulf booting correctly, that's not really a problem.

Thanks Greg.

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novo...@rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'
________________________________________

From: Gregory M. Kurtzer [gmku...@lbl.gov]
Sent: Wednesday, October 07, 2015 6:02 PM
To: Warewulf
Subject: Re: [Warewulf] RE: WW 3.6.99, uEFI PXE Boot

I have tried to, but due to some known but unfixed bugs in Syslinux I stopped working on it. Your client output looked different from what I noticed but that would be client implementation specific, but it did seem to be failing at about the same point.

I have not been following up with it because we have been able to boot using legacy mode. Which actually turned out to be beneficial due to a side effect of one of our vendors has a very high memory footprint when booting via uEFI (on the order of hundreds of megabytes lost to the operating system).

On Wed, Oct 7, 2015 at 12:13 PM, Novosielski, Ryan <novo...@ca.rutgers.edu<mailto:novo...@ca.rutgers.edu>> wrote:
Is anyone able to confirm that at least they do have EFI booting working with Warewulf?

--
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist

|| \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'
________________________________________

From: Novosielski, Ryan [novo...@ca.rutgers.edu<mailto:novo...@ca.rutgers.edu>]

|| \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'

--
You received this message because you are subscribed to the Google Groups "Warewulf" group.

To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2Bunsu...@lbl.gov>.
To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
For more options, visit https://groups.google.com/a/lbl.gov/d/optout.

--
You received this message because you are subscribed to the Google Groups "Warewulf" group.

To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2Bunsu...@lbl.gov>.
To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
For more options, visit https://groups.google.com/a/lbl.gov/d/optout.

--
Gregory M. Kurtzer
Technical Lead and HPC Systems Architect
High Performance Computing Services (HPCS)
University of California
Lawrence Berkeley National Laboratory
One Cyclotron Road, Berkeley, CA 94720

--
You received this message because you are subscribed to the Google Groups "Warewulf" group.

To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.
To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com<https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Allen, Benjamin S.

unread,

Aug 29, 2016, 5:52:31 PM8/29/16

to ware...@lbl.gov

syslinux 6.04 Pre1 appears to fix this bugs. I was able to boot via uEFI on HPE XL170r servers via:

cp syslinux-6.04-pre1/efi64/efi/syslinux.efi /var/lib/tftpboot/warewulf/syslinux64.efi
cp syslinux-6.04-pre1/efi32/efi/syslinux.efi /var/lib/tftpboot/warewulf/syslinux32.efi
cp syslinux-6.04-pre1/efi64/com32/elflink/ldlinux/ldlinux.e64 /var/lib/tftpboot/warewulf/ldlinux.e64
cp syslinux-6.04-pre1/efi32/com32/elflink/ldlinux/ldlinux.e32 /var/lib/tftpboot/warewulf/ldlinux.e32

Available at https://www.kernel.org/pub/linux/utils/boot/syslinux/Testing/6.04/

Note, it appears in UEFI mode these machines have 71MB less total memory compared to legacy mode.

Ben

> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF152B%40umdexmbx02.

Ryan Novosielski

unread,

Aug 29, 2016, 6:41:22 PM8/29/16

to ware...@lbl.gov

Thank you, Ben! Great news — it’s uncomfortable to be so reliant on something that’s headed toward legacy.

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/1B1A28A1-E160-4ECD-B4E4-CC9EC04082A5%40alcf.anl.gov.

> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>

--

____
|| \\UTGERS, |---------------------------*O*---------------------------
||_// the State | Ryan Novosielski - novo...@rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
|| \\ of NJ | Office of Advanced Research Computing - MSB C630, Newark
`'

signature.asc

Glenn Johnson

unread,

Jan 9, 2017, 4:17:44 PM1/9/17

to Warewulf, novo...@rutgers.edu

Hi,

I think I am running into this same issue. I had setup WW in a VM and all worked well. I am now trying to deploy on Lenovo nx360 M5 servers but am not able to PXE boot. I tried using the syslinux files but that did not help, nor does legacy mode.

I see

>>Start PXE over IPv4.

PXE-E18: Timeout. Server did not respond

I do see some activity in the logs.

Jan 9 14:49:45 argon-head xinetd[2039]: START: tftp pid=45911 from=172.29.18.150

Jan 9 14:49:45 argon-head in.tftpd[45912]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 14:49:45 argon-head in.tftpd[45912]: tftp: client does not accept options

Jan 9 14:49:45 argon-head in.tftpd[45913]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 14:49:45 argon-head in.tftpd[45913]: Client 172.29.18.150 finished /warewulf/lpxelinux.0

Jan 9 15:03:36 argon-head in.tftpd[48831]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 15:03:36 argon-head in.tftpd[48831]: tftp: client does not accept options

Jan 9 15:03:36 argon-head in.tftpd[48832]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 15:03:36 argon-head in.tftpd[48832]: Client 172.29.18.150 finished /warewulf/lpxelinux.0

Jan 9 15:05:42 argon-head in.tftpd[49266]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 15:05:42 argon-head in.tftpd[49266]: Client 172.29.18.150 finished /warewulf/lpxelinux.0

Jan 9 15:10:26 argon-head in.tftpd[50172]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 15:10:26 argon-head in.tftpd[50172]: tftp: client does not accept options

Jan 9 15:10:26 argon-head in.tftpd[50173]: RRQ from 172.29.18.150 filename /warewulf/lpxelinux.0

Jan 9 15:10:26 argon-head in.tftpd[50173]: Client 172.29.18.150 finished /warewulf/lpxelinux.0

Any ideas?

Thanks.

>> || \\ and Health | novo...@rutgers.edu<mailto:novos...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

>> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
>> `'
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Warewulf" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.
>> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

>> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
>> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Warewulf" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.
>> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

>> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
>> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>>
>>
>>
>> --
>> Gregory M. Kurtzer
>> Technical Lead and HPC Systems Architect
>> High Performance Computing Services (HPCS)
>> University of California
>> Lawrence Berkeley National Laboratory
>> One Cyclotron Road, Berkeley, CA 94720
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Warewulf" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

>> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

Ryan Novosielski

unread,

Jan 9, 2017, 4:31:27 PM1/9/17

to Glenn Johnson, Warewulf

I suspect you have something else going on. Legacy mode works fine for me on that hardware out of the box with WW 3.7, and uEFI mode I noticed now works for me with SYSLINUX 6.04pre1 swapped in (it didn’t a few months ago when I tried it — I don’t know what changed in the meantime, but I just got a system board replaced and accidentally tried it again). FYI, re: revs:

[root@node001 updates]# ./dmidecode -s system-product-name
Lenovo NeXtScale nx360 M5: -[5465AC1]-
[root@node001 updates]# ./dmidecode -s bios-version
-[THE122RUS-2.00]-
[root@node001 updates]# ./dmidecode -s baseboard-version
STC

Incidentally, the Lenovo machines have a couple hundred megs difference in memory with uEFI boot on.

> >> || \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> >> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> >> `'
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >>
> >>
> >>
> >> --
> >> Gregory M. Kurtzer
> >> Technical Lead and HPC Systems Architect
> >> High Performance Computing Services (HPCS)
> >> University of California
> >> Lawrence Berkeley National Laboratory
> >> One Cyclotron Road, Berkeley, CA 94720
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

signature.asc

Glenn Johnson

unread,

Jan 9, 2017, 5:43:59 PM1/9/17

to Warewulf, gle...@gmail.com, novo...@rutgers.edu

Okay; I have gotten a little further. Using legacy with UEFI PXE disabled gets me to the point where it begins the boot, configures the network on the client but then hangs with

Trying to reach the master node at xxx.xxx.xxx.xxx .........

Also, exactly how do you swap in SYSLINUX? I copied the files according to a previous post but it did not seem like they were getting read. Do I need to rebuild the bootstrap?

Thanks.

> >> || \\ and Health | novo...@rutgers.edu<mailto:novos...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> >> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> >> `'
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

> >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

> >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >>
> >>
> >>
> >> --
> >> Gregory M. Kurtzer
> >> Technical Lead and HPC Systems Architect
> >> High Performance Computing Services (HPCS)
> >> University of California
> >> Lawrence Berkeley National Laboratory
> >> One Cyclotron Road, Berkeley, CA 94720
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> >> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

Ryan Novosielski

unread,

Jan 9, 2017, 6:33:15 PM1/9/17

to Glenn Johnson, Warewulf

I believe you get the files from here:

https://www.kernel.org/pub/linux/utils/boot/syslinux/Testing/6.04/

…and you need the appropriate pair for whatever you’re doing, from this list:

http://www.syslinux.org/wiki/index.php?title=Install#file_list

So, for example, for uEFI X64, you need a replacement syslinux.efi and ldlinux.e64. On my system, those are in /var/lib/tftpboot/warewulf as syslinux64.efi and ldlinux.e64. What files are where is all laid out in /etc/dhcp/dhcpd.conf on my system, which is a pretty default CentOS 7 install. You’ll see the various architecture types called out and what directory and filename the pieces are found at. You should also have existing files in there that will make it clear what goes where.

> > >> || \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> > >> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> > >> `'
> > >>
> > >> --
> > >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> > >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> > >>
> > >> --
> > >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > >> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> > >> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> > >>
> > >>
> > >>
> > >> --
> > >> Gregory M. Kurtzer
> > >> Technical Lead and HPC Systems Architect
> > >> High Performance Computing Services (HPCS)
> > >> University of California
> > >> Lawrence Berkeley National Laboratory
> > >> One Cyclotron Road, Berkeley, CA 94720
> > >>
> > >> --
> > >> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > >> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> > >> To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

signature.asc

Ryan Novosielski

unread,

Jan 9, 2017, 6:40:17 PM1/9/17

to ware...@lbl.gov, Glenn Johnson

This page also might be helpful:

http://www.syslinux.org/wiki/index.php?title=PXELINUX#UEFI

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/A1FD341E-8026-46E2-BB03-7CD548E8C77A%40rutgers.edu.

signature.asc

Glenn Johnson

unread,

Jan 9, 2017, 9:49:31 PM1/9/17

to Warewulf, gle...@gmail.com, novo...@rutgers.edu

Ryan,

Thanks for all of your help on this. I will try the UEFI boot more tomorrow. I am curious though why the legacy boot did not work. Looking at the code I see where it is hanging in initramfs/base. It essentially is failing to ping the master host IP. The failed ping seems odd given that it retrieved the pxe files from that same host. The network parameters look right on the client so I am not sure what is blocking.

Thanks.

>>>>> || \\ and Health | novo...@rutgers.edu<mailto:novos...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

>>>>> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
>>>>> `'
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "Warewulf" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

>>>>> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

>>>>> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
>>>>> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "Warewulf" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

>>>>> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

>>>>> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
>>>>> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Gregory M. Kurtzer
>>>>> Technical Lead and HPC Systems Architect
>>>>> High Performance Computing Services (HPCS)
>>>>> University of California
>>>>> Lawrence Berkeley National Laboratory
>>>>> One Cyclotron Road, Berkeley, CA 94720
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "Warewulf" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

>>>>> To post to this group, send email to ware...@lbl.gov<mailto:warewulf...@lbl.gov>.

Ryan Novosielski

unread,

Jan 10, 2017, 1:42:19 AM1/10/17

to ware...@lbl.gov, gle...@gmail.com

They may use different network protocols or ports or what have you. It's been awhile since I had to get this to work from scratch.

--
____
|| \\UTGERS, |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - novo...@rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
|| \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
    `'

To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/21580eb3-1585-406e-8433-ff9e0a159e9a%40lbl.gov.

Aurelien Bouteiller

unread,

May 11, 2017, 5:03:57 PM5/11/17

to Warewulf

I can confirm that swapping 6.04 permits PXE UEFI boot.

There is however a serious performance problem/limitation: when obtaining the initfs.gz file with TFTP, the transfer starts at normal speed but quickly degrades to kb/minutes. If you have lots of drivers (nvidia, mic, etc), the initfs will be too large and trigger the performance bug, making boot impossible (watchdog will reboot the machine before it completes). I have found that an initfs larger than 10MB is unusable.

As usual, Bios PXE works as intended.

If anybody knows how to fix this, I can take.

> || \\ and Health | novo...@rutgers.edu<mailto:novos...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> `'
>
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.
> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.
> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>
>
>
> --
> Gregory M. Kurtzer
> Technical Lead and HPC Systems Architect
> High Performance Computing Services (HPCS)
> University of California
> Lawrence Berkeley National Laboratory
> One Cyclotron Road, Berkeley, CA 94720
>
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> To post to this group, send email to ware...@lbl.gov<mailto:warewul...@lbl.gov>.

Jason Stover

unread,

May 11, 2017, 5:22:47 PM5/11/17

to ware...@lbl.gov

Hi Aurelien,

Generally, I try making the included modules as minimal as possible
since a lot of them aren't needed until the OS init starts up anyway.
It may make the image a bit larger, but I keep the /lib/modules/[kver]
of the kernel I'm booting in the VNFS image. Then, when init takes
over, it can still load any modules it needs, and they don't need to
be in the bootstrap.

If the device isn't needed during the provision step, you don't need
to include it going this route ... i.e. nvidia, mic can be left out,
etc...

-J

Allen, Benjamin S.

unread,

May 11, 2017, 5:51:06 PM5/11/17

to ware...@lbl.gov

You can convert to using HTTP instead of TFTP and see if it makes a difference. You would need to serve /var/lib/tftpboot/warewulf via Apache, if you have selinux enable you'll have to deal with allowing httpd access to that directory. Then test by editing a /var/lib/tftpboot/warewulf/pxelinux.cfg/01-... file similar to:

KERNEL http://<master>/warewulf/bootstrap/287/kernel
APPEND ro initrd=http://<master>/warewulf/bootstrap/287/initfs.gz ...

Of course if there's any UEFI firmware updates for the machine it may be useful to apply them.

Ben

> > || \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> > || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> > `'
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> >
> >
> > --
> > Gregory M. Kurtzer
> > Technical Lead and HPC Systems Architect
> > High Performance Computing Services (HPCS)
> > University of California
> > Lawrence Berkeley National Laboratory
> > One Cyclotron Road, Berkeley, CA 94720
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com<https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> > To post to this group, send email to ware...@lbl.gov.
> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF152B%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/4b80cf1d-bc6f-40ec-a8e5-87a68fc4415f%40lbl.gov.

Ryan Novosielski

unread,

May 11, 2017, 6:31:15 PM5/11/17

to ware...@lbl.gov

I don’t, but I have to say thank you for pointing that out. I was having screwy problems with our new equipment — boots sometimes, sometimes not. We’re using UEFI on it (or were trying to at least until we had any problem with it). I wonder if this is related at all. I was seeing very slow download speeds as well for part of the boot, as you are.

This is still with HPE XL170r and 6.04pre1? Mine is Lenovo NeXtScale nx360 M5. I’m not sure which SYSLINUX I’m using at this point.

> > || \\ and Health | novo...@rutgers.edu<mailto:novo...@rutgers.edu> - 973/972.0922<tel:973%2F972.0922> (2x0922)

> > || \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
> > `'
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1520%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf%2B...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF1528%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> >
> >
> > --
> > Gregory M. Kurtzer
> > Technical Lead and HPC Systems Architect
> > High Performance Computing Services (HPCS)
> > University of California
> > Lawrence Berkeley National Laboratory
> > One Cyclotron Road, Berkeley, CA 94720
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov<mailto:warewulf+u...@lbl.gov>.

> > To post to this group, send email to ware...@lbl.gov<mailto:ware...@lbl.gov>.

> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com<https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAN7etTz1hMt3%2B0HV7xFeCf2Le2XX6ACnnR94yXJzN0t7LS92-g%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Warewulf" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> > To post to this group, send email to ware...@lbl.gov.
> > To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ADFBD4B618944F499D05722136ECD99388E6BF152B%40umdexmbx02.
> > For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/4b80cf1d-bc6f-40ec-a8e5-87a68fc4415f%40lbl.gov.

> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.

--

signature.asc

Allen, Benjamin S.

unread,

Jun 2, 2017, 12:17:17 PM6/2/17

to ware...@lbl.gov

UEFI performance on the HPE XL170r appears to be as good I'd expect with legacy BIOS booting, but HPE is typically ahead of the game with UEFI compared to other vendors.

I found I can reproduce the slow initrd.gz loading over TFTP via VirtualBox in EFI mode using the virtio-net adapter type. Throughput starts around 3Mb / second and drops from there. Testing with HTTP, it can push a blazing 6Mb/s fairly consistently. For comparison the VNFS transfer drives 55Mb/s in this VM testbed. A packet capture of the TFTP session isn't particularly enlightening. If anyone wants to test this using KVM, virtio, and the UDK EFI version that ships with it, I'd be interested if there's an improvement.

Note, the virtio-net adapter is the only adapter the EDKII / UDK2014.SP1 based EFI implementation, which ships with VirtualBox, to have an EFI network driver. Also note, Warewulf's bootstrap needs kargs to include wwkmods=virtio_net,virtio_pci for virtio_net to be loaded, as well as "drivers += kernel/drivers/virtio/" added to bootstrap.conf.

I'm thinking the slow transfer could either be a bug in UDK2014 or syslinux itself. I came across a similar bug in Grub2, where Grub2 wasn't taking exclusive access of the network adapter from EFI. Google'ing for syslinux related issues hasn't been particularly enlightening however.

Ben

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/2F2DD652-7CD7-4665-95CC-0DFEE855DAC6%40rutgers.edu.

Allen, Benjamin S.

unread,

Jun 25, 2017, 11:50:25 PM6/25/17

to ware...@lbl.gov

I noticed today that booting Linux via iPXE's ipxe.efi (http://boot.ipxe.org/ipxe.efi) on a VirtualBox EFI guest with a Virtio network driver is as fast as BIOS mode. I'd be interested if this holds true for other slow to EFI boot with syslinux platforms. To that end if anyone can test on real hardware where this is a problem, here is a quick set of changes to boot a node via iPXE.

Example snippet of dhcpd.conf:

option pxelinux.pathprefix "http://192.168.2.100/WW/boot";

if exists user-class and option user-class = "iPXE" {
filename "/warewulf/ipxe/test.cfg";
} else {
if option architecture-type = 00:09 {
# filename "/warewulf/syslinux64.efi";
filename "/warewulf/ipxe/ipxe.efi";
} elsif option architecture-type = 00:07 {
# filename "/warewulf/syslinux64.efi";
filename "/warewulf/ipxe/ipxe.efi";
} elsif option architecture-type = 00:06 {
# filename "/warewulf/syslinux32.efi";
filename "/warewulf/ipxe/ipxe.efi";
} else {
# filename "/warewulf/lpxelinux.0";
filename "/warewulf/ipxe/undionly.kpxe";
}
}

Example iPXE test.cfg, derive yours from a node's /var/lib/tftpboot/warewulf/pxelinux.cfg/01-...:

#!ipxe

set base http://192.168.2.100/WW/boot/warewulf
kernel ${base}/bootstrap/12/kernel ro initrd=initfs.gz wwhostname=n01 wwkmods=virtio_net,virtio_pci net.ifnames=0 biosdevname=0 nosplash wwmaster=192.168.2.100 wwnetmask=255.255.255.0 wwnetdev=eth1 wwhwaddr=08:00:27:9b:39:94
initrd ${base}/bootstrap/12/initfs.gz
boot

Serve tftpboot via HTTP, /etc/httpd/conf.d/warewulf-http.conf:

Alias /WW/boot /var/lib/tftpboot

<Directory /var/lib/tftpboot>
AllowOverride None
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
</Directory>

Lastly if selinux is enabled, allow httpd access to /var/lib/tftpboot:

yum install policycoreutils-python
semanage fcontext -a -t public_content_t '/var/lib/tftpboot(/.*)?'
restorecon -Rv /var/lib/tftpboot

Thanks,

Ben

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/5F2BE82E-AEAD-4830-8EC2-C5008A308BDF%40alcf.anl.gov.

tst...@nyu.edu

unread,

Oct 3, 2017, 6:29:59 PM10/3/17

to Warewulf, novo...@ca.rutgers.edu

Hi,

How can I get warewulf to work with 4K disk sizes?

Thanks

Tony

Renato Golin

unread,

Oct 4, 2017, 4:04:54 AM10/4/17

to ware...@lbl.gov, novo...@ca.rutgers.edu

This smells like a driver problem. Note that at PXE time, the "Station
IP address" is 10.0.81.200 while after it loads the NBP file, it's
0.0.0.0. No wonder UDP doesn't work.

One possibility is that the loader is resetting the network config,
another is that it's trying to use a different, unconfigured device.

Also, have you tried using the new iPXE support in the development
branch? Seems to work well on x86 and ARM machines we tested with
OpenHPC.

cheers,
--renato

> --
> You received this message because you are subscribed to the Google Groups
> "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit

> https://groups.google.com/a/lbl.gov/d/msgid/warewulf/ab2302b0-5097-449a-940c-2aeb62fb11d1%40lbl.gov.

Tony S Teague

unread,

Oct 4, 2017, 2:39:53 PM10/4/17

to ware...@lbl.gov, novo...@ca.rutgers.edu

So you got warewulf woking on 4k disks.

Tony

> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAMSE1kfGkFALqH87VbbeADdPniJwwLaRBGmfmeS6fiuvJmHbnw%40mail.gmail.com.

Renato Golin

unread,

Oct 5, 2017, 5:15:12 AM10/5/17

to ware...@lbl.gov, Ryan Novosielski

On 4 October 2017 at 19:39, Tony S Teague <tst...@nyu.edu> wrote:
> So you got warewulf woking on 4k disks.

Sorry, was that a question?

I was only commenting on the little I could see about the error messages.

I don't know what you mean by "4k disk size", but I'm guessing it's
not 4Kb flash. :)

--renato

Ryan Novosielski

unread,

Oct 5, 2017, 9:39:31 AM10/5/17

to ware...@lbl.gov

Tony,

This question does not appear to have anything to do with the subject or the quoted content. Am I misunderstanding something? If not, please don’t thread hijack.

--
____
|| \\UTGERS, |---------------------------*O*---------------------------
||_// the State |         Ryan Novosielski - novo...@rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
|| \\    of NJ | Office of Advanced Research Computing - MSB C630, Newark
    `'

--

Reply all

Reply to author

Forward