[ANN] Pixiecore, simple all-in-one netbooting

[David Anderson]

Booting a linux system over the network is quite tedious. You have to set up a TFTP server, configure your DHCP server to recognize PXE clients, and send just the right set of magical options to get them to boot, often fighting rubbish PXE ROM implementations.

Pixiecore reduces that to: `pixiecore -kernel vmlinuz -initrd initrd.cpio.gz`

With that, any PXE client that shows up on your network will be netbooted into the given linux kernel and ramdisk.

Of note is that Pixiecore does not require replacing or changing your existing DHCP server. Instead, Pixiecore uses a feature of the PXE spec (very poorly named "ProxyDHCP" - it's not a proxy, and it's barely DHCP) to complement the normal DHCP network configuration with PXE configurations for eligible clients, without directly interacting with the network's main DHCP server. Your non-PXE clients and your regular DHCPd are completely unaware of Pixiecore and don't need any reconfiguration.

https://github.com/danderson/pixiecore has more details, including a Docker image and a description of the madness that is navigating the PXE specification.

Bug reports are welcome, given the complexity of the PXE spec and the bugginess of PXE firmwares, I'm quite sure that I'll have to iterate some. Currently, Pixiecore has been tested and works with ipxe, and with the horrible boot rom that Realtek provides in its RTL8168 NIC.

- Dave

[andrewc...@gmail.com]

Thanks, this seems cool.

One example of what this can be used for:

My university used a PXE boot system which had an initrd which performed an rsync of the root filesystem from a central server on boot. This way all PC's had identical linux installs which could be updated centrally.

[aren...@gmail.com]

After reading your "How it works" section: Kudos for working through this nightmare!

[Tim K]

Wow, very cool. I don't imagine you wrote this just for fun.

Tim

[Brad Fitzpatrick]

This is great!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Axel Wagner]

Cool stuff :) I just have to say: The README is an amazing read :) Very
nice and clever stuff you did there :) I used PXE in the past for linux
install parties, where we would gather new students and install them
linux. But using PXE to distribute the installation media had the
downside that we couldn't have internet (because DHCP) access during
installs. So this looks actually kind of perfect for that :)

David Anderson <da...@natulte.net> writes:

> Booting a linux system over the network is quite tedious. You have to set
> up a TFTP server, configure your DHCP server to recognize PXE clients, and
> send just the right set of magical options to get them to boot, often
> fighting rubbish PXE ROM implementations.
>
> Pixiecore reduces that to: `pixiecore -kernel vmlinuz -initrd
> initrd.cpio.gz`
>
> With that, any PXE client that shows up on your network will be netbooted
> into the given linux kernel and ramdisk.
>

> Of note is that Pixiecore does *not* require replacing or changing your

> existing DHCP server. Instead, Pixiecore uses a feature of the PXE spec
> (very poorly named "ProxyDHCP" - it's not a proxy, and it's barely DHCP) to
> complement the normal DHCP network configuration with PXE configurations
> for eligible clients, without directly interacting with the network's main
> DHCP server. Your non-PXE clients and your regular DHCPd are completely
> unaware of Pixiecore and don't need any reconfiguration.
>
> https://github.com/danderson/pixiecore has more details, including a Docker
> image and a description of the madness that is navigating the PXE
> specification.
>
> Bug reports are welcome, given the complexity of the PXE spec and the
> bugginess of PXE firmwares, I'm quite sure that I'll have to iterate some.
> Currently, Pixiecore has been tested and works with ipxe, and with the
> horrible boot rom that Realtek provides in its RTL8168 NIC.
>
> - Dave
>

[David Anderson]

A quick followup: I've just pushed the first version of Pixiecore's API mode.

In API mode, requests made by PXE clients will be translated into calls to an HTTP API, essentially asking "Should I boot this client? If so, what should I boot it with?" This lets you implement fancy dynamic booting things, as well as construct per-machine commandlines and whatnot.

Hopefully, if I did it right, API mode makes it much easier for people to experiment with netbooting in new and exciting ways, without having to care about working with PXE firmware directly. Just implement a trivial HTTP API that expresses what you want, and Pixiecore will take care of making it happen.

The API documentation is at https://github.com/danderson/pixiecore/blob/master/README.api.md . Feedback welcome.

- Dave

On Mon, Sep 14, 2015 at 8:17 PM, David Anderson <da...@natulte.net> wrote:

[roger peppe]

Nice. Not that I have any immediate plans for using this but it occurred to me that perhaps you could let the HTTP server return a host-relative URL for the image - that way the server won't need to know its absolute address if it's serving the images itself.

[Steph]

Hi Dave,

This looks great! I'm a new developer and I've been toying around with DHCP servers for a day in which time I managed to create an internet nightmare at my company. So pixiecore looks like a great solution. In terms of running pixiecore, should this be done inside of a docker container? Basically,I'm not familiar with golang and don't really know how to run pixiecore. Thanks for any and all help here. 

Best, 

Steph 

[vito...@gmail.com]

How do you build pixiecore?

I'm not familar with GO or go clients.  Any help would be appreciated.

[ji...@blockstream.io]

 David,

 I'm confused by one part of this ...

On Monday, September 14, 2015 at 8:18:23 PM UTC-7, David Anderson wrote:

 ... 

Of note is that Pixiecore does not require replacing or changing your existing DHCP server. Instead, Pixiecore uses a feature of the PXE spec (very poorly named "ProxyDHCP" - it's not a proxy, and it's barely DHCP) to complement the normal DHCP network configuration with PXE configurations for eligible clients, without directly interacting with the network's main DHCP server. Your non-PXE clients and your regular DHCPd are completely unaware of Pixiecore and don't need any reconfiguration.

 ... if I'm reading correctly then it sounds like you're saying that 'pixiecore' is NOT a DHCP server.  That it's able to piggyback on DHCP

 traffic (on a segment, within a given ethernet broadcast/collision domain) to provide additional configuration data (using this "ProxyDHCP" feature/spec.) which a PXE client can use for just the TFTP fetch (bootloader -> kernel + initrd/initramfs chains)?

 If that's the case then, I guess it's necessary to have a separate DHCP server on the network already?

 (I've tried setting up multiple PXE clients on cross-over cables to my MacOS X (10.11.5) running 'pixiecore' (fc0d895c19e7c6a31f9d3b8ad5c8967d7285ef87 from Feb 28, 2016) and the "tinycorelinux" configuration as described in the github

 pages.  When running it I get messages like:

[ProxyDHCP] Couldn't find an IP address to use to reply to f0:de:f1:3b:b9:65: interface en5 has no usable unicast addresses

[ProxyDHCP] Offering to boot f0:de:f1:3b:b9:65 (via 172.17.17.1) 

[ProxyDHCP] Responding to f0:de:f1:3b:b9:65: write udp4 0.0.0.0:67->255.255.255.255:68: sendmsg: network is unreachable

 ... after having statically configure the en5 (thunderbolt connect ethernet cable to the other systems (I've tried four different systems, at least two of which I've PXE booted into Linux using the traditional ISC/Linux tools and configuration in the past).  I've done this with and without ethernet cross-over cabling (many ethernet adapters know support auto-MDI crossing) and I've done with direct and through an old, simple, Netgear switch ... but always with no DHCP server on that segment!

 I'll try again with a DHCP server to see if that works.  But you might make it clear in the docs if 'pixiecore' REQUIRES a separate DHCP server, or perhaps add an option to detect and optionally provide DHCP services.

[David Anderson]

On Sun, Aug 7, 2016 at 5:11 PM, <ji...@blockstream.io> wrote:

 David,

 I'm confused by one part of this ...

On Monday, September 14, 2015 at 8:18:23 PM UTC-7, David Anderson wrote:

 ... 

Of note is that Pixiecore does not require replacing or changing your existing DHCP server. Instead, Pixiecore uses a feature of the PXE spec (very poorly named "ProxyDHCP" - it's not a proxy, and it's barely DHCP) to complement the normal DHCP network configuration with PXE configurations for eligible clients, without directly interacting with the network's main DHCP server. Your non-PXE clients and your regular DHCPd are completely unaware of Pixiecore and don't need any reconfiguration.

 ... if I'm reading correctly then it sounds like you're saying that 'pixiecore' is NOT a DHCP server.  That it's able to piggyback on DHCP

 traffic (on a segment, within a given ethernet broadcast/collision domain) to provide additional configuration data (using this "ProxyDHCP" feature/spec.) which a PXE client can use for just the TFTP fetch (bootloader -> kernel + initrd/initramfs chains)?

 If that's the case then, I guess it's necessary to have a separate DHCP server on the network already?

That's correct. ProxyDHCP allows a third-party server to provide _just_ network booting information, while the network's primary DHCP server provides network configuration. The idea being that you can run Pixiecore on an existing DHCP-managed network without changing any other configurations.

While at this point I probably know enough of DHCP to implement full DHCP support as well... I'm a bit resistant to doing so, because really good DHCP support is a huge can of worms in terms of supporting buggy implementations, and I'd much rather point people to dnsmasq if they need a quick and dirty DHCP server :).

 (I've tried setting up multiple PXE clients on cross-over cables to my MacOS X (10.11.5) running 'pixiecore' (fc0d895c19e7c6a31f9d3b8ad5c8967d7285ef87 from Feb 28, 2016) and the "tinycorelinux" configuration as described in the github

 pages.  When running it I get messages like:

[ProxyDHCP] Couldn't find an IP address to use to reply to f0:de:f1:3b:b9:65: interface en5 has no usable unicast addresses

[ProxyDHCP] Offering to boot f0:de:f1:3b:b9:65 (via 172.17.17.1) 

[ProxyDHCP] Responding to f0:de:f1:3b:b9:65: write udp4 0.0.0.0:67->255.255.255.255:68: sendmsg: network is unreachable

 ... after having statically configure the en5 (thunderbolt connect ethernet cable to the other systems (I've tried four different systems, at least two of which I've PXE booted into Linux using the traditional ISC/Linux tools and configuration in the past).  I've done this with and without ethernet cross-over cabling (many ethernet adapters know support auto-MDI crossing) and I've done with direct and through an old, simple, Netgear switch ... but always with no DHCP server on that segment!

The initial error you're getting is that Pixiecore can't find a usable source IP address on en5. It needs that address to transmit the ProxyDHCP response. Do you have any IPs configured on en5?

 

 I'll try again with a DHCP server to see if that works.  But you might make it clear in the docs if 'pixiecore' REQUIRES a separate DHCP server, or perhaps add an option to detect and optionally provide DHCP services.

Are you connecting your machine directly to a system that you need to boot? If that's the case, you'll run into a separate issue if you try to run both Pixiecore and a DHCP server on the same machine, which is that they both want to bind to the DHCP server port, and so one of them will fail. I have a working solution for that on linux using some raw socket trickery, but since I have no macs, I don't have an answer for your setup yet :(.

The closest thing to an answer I have is to make sure you connect both the machine running Pixiecore and the system you're trying to boot to an ethernet broadcast domain that already has DHCP services provided by some other machine. I'm very much aware that this isn't great, and I'll now go off and think about how to improve that.

- Dave

 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Jim Dennis]

 David,

 Thanks for the fast response.  I have, since writing that, confirmed that I

 can boot at least one of these systems (Lenovo ThinkPad T410) using

 pixiecore (running on my Mac laptop) when both systems are connected

 to my home router (which, of course, provides DHCP).

 So that works.  I'd like to suggest that you make this point more clear in

 the README.md file (I can provide a pull request if you like).  Another

 possibility might be for you to add an option to pixiecore, perhaps built

 on an compile time option to link to https://github.com/krolaw/dhcp4 ... 

 Richard Warburton's DHCP library (client and server support) written

 in Go(lang).

 The other option I'm considering is Astralboot:

 https://github.com/zignig/astralboot which seems to include the DHCP

 server ... but which requires IPFS for the file distribution services.

 I'll test those and write up my recommendations for my boss.

[David Anderson]

On Mon, Aug 8, 2016 at 3:15 PM, Jim Dennis <ji...@blockstream.io> wrote:

 David,

 Thanks for the fast response.  I have, since writing that, confirmed that I

 can boot at least one of these systems (Lenovo ThinkPad T410) using

 pixiecore (running on my Mac laptop) when both systems are connected

 to my home router (which, of course, provides DHCP).

 So that works.  I'd like to suggest that you make this point more clear in

 the README.md file (I can provide a pull request if you like).

A pull request would be most welcome, but failing that I'll get that done later this week. Thanks for the suggestion!

 

  Another

 possibility might be for you to add an option to pixiecore, perhaps built

 on an compile time option to link to https://github.com/krolaw/dhcp4 ... 

 Richard Warburton's DHCP library (client and server support) written

 in Go(lang).

Yeah, I'm thinking about various options. I'm still worried about using a Go dhcp4 package, because those don't have all the compatibility tweaks that battle-tested implementations like ISC dhcpd and dnsmasq have. My slight preference at this point would be to have a way to delegate to a local dnsmasq that Pixiecore runs as needed. Either way, it's clear this needs to happen.

 

 The other option I'm considering is Astralboot:

 https://github.com/zignig/astralboot which seems to include the DHCP

 server ... but which requires IPFS for the file distribution services.

Yeah, Astralboot is implementing a full DHCP server. This has several advantages, the biggest being that the network boot process is simplified when a single server provides network and boot configuration (fewer buggy clients). That's great if it's what you need, of course. My design goal with Pixiecore was to have something that I can drop into an existing network to add netbooting capabilities, without having to change any other network services (basically, I wanted a quick way to do one-off netboot installs on some machines on my LAN :) ). So in a sense Astralboot and Pixiecore are at odds on the fundamental problem they want to solve.

 I'll test those and write up my recommendations for my boss.

I'd be interested in your analysis once you're done, if you can/feel like sharing it. I think I already have all the "lessons learned" for Pixiecore from your messages so far, but I'm generally interested in the space, so it's useful to have additional inputs :).

- Dave