CoreOS on raspberrypi 3 (amd64)

[Andreas Mahnke]

Hello,

I successfully created an amd64 CoreOS image based on Geoffs descriptions and repo (https://github.com/glevand/hikey-coreos) and would like to run it on a raspberrypi 3 (but I have no clue how to get it booted, due to the fact that the raspberry does not support grub). 

Do you have some hints or tips&tricks that I could try? 

Best regards from Hamburg, Germany

Andreas

[Jeffrey Hulten]

Firstly, the rPi machines are not amd64 boxes, so step one is compiling/porting to ARM. Not that I have any idea what is needed to get that working.

[Joshua Perry]

Hi Andreas,

Interesting, this is something that I’ve been pondering as well. I was very close to starting a 32-bit port of CoreOS to run on the RPi2 but ended up going with Snappy Ubuntu Core. I’m definitely interested in this as we are in the middle of implementing the infrastructure necessary to deploy, run, and update container-based applications on embedded CPE and I’d love to use CoreOS instead of Snappy.

The RPi boot process is a little funky but a bit simpler than grub et al.

This post has a lot of low-level information on the boot process as the dev was interested in getting code to run in secure mode before the boot process hands control off to the kernel: http://blog.flexvdi.es/2015/02/25/enabling-hyp-mode-on-the-raspberry-pi-2/

Another dev used this info to boot u-boot instead of the kernel in order to more easily manage the boot process: https://blog.night-shade.org.uk/2015/05/booting-a-raspberry-pi2-with-u-boot-and-hyp-enabled/

I haven’t tried this on the RPi3 yet so I’m not sure if the `kernel_old` parameter is still honored (this isn’t really necessary to boot CoreOS, but the information about the boot process is important and I make use of this for the secure boot implementation that we’re working on).

I haven’t yet found the code that the Ubuntu guys use to boot the RPi 2 using a similar A/B partitioning scheme but that may be a good place to start since CoreOS works similarly. I shot an email off to the snappy-dev list, hopefully they didn’t get the RPi foundation to jigger their closed boot process specially for Snappy.

Let me know if I can help out, I’ve got a handful of RPi 3s sitting here that I can mess with.

Josh 

[Joshua Perry]

Looks like they use u-boot (set as `kernel` in `config.txt`) and a u-boot script to do the a/b partition choosing and basically leave the rest of the RPi boot process as is. I’m not familiar with the CoreOS boot process yet but I’d assume it’s similar.

If you just want to try booting your build, you should be able to just set up the boot chain binaries, point `config.txt` at your kernel image, and configure the kernel parameters. Then I suppose the boot should just work, sans multi partition stuffs.

The Snappy u-boot script: http://people.canonical.com/~ogra/snappy/snappy-system.txt

Boot binaries (including for RPi3): https://github.com/piso77/ubuntu-embedded

Josh

[Geoff Levand]

Hi Andreas,

On Mon, 2016-03-21 at 14:16 -0700, Andreas Mahnke wrote:
> I successfully created an amd64 CoreOS image based on Geoffs
> descriptions and repo (https://github.com/glevand/hikey-coreos) and would like to run it on a raspberrypi 3 (but I have no clue how to get it booted, due to the fact that the raspberry does not support grub).
>
> Do you have some hints or tips&tricks that I could try?

Any arm64 server will have UEFI support. UEFI is
required by all the major distros, including CoreOS.
CoreOS can't support a platform that does not have
UEFI. I don't think raspberrypi 3 has UEFI support.
Is there any work on it?

CoreOS won't work on a platform that doesn't have its
board support in the upstream kernel. Will the
vanilla 4.4 kernel boot on raspberrypi 3? If so, then
we just need to enable the kernel config options for
it in the CoreOS kernel.

Without either of these it is still possible to hack
something together to boot CoreOS.

Let me know the answeres to the above.

-Geoff

[Andreas Mahnke]

Hi Geoff,

I'm not sure about the UEFI support - by now I could not find any indicator that it is supported. 

The Kernel topic looks better though. Some folks where able to boot a 64bit Version 4.5 Kernel on the pi3 using u-boot. (See also https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963&sid=62597535a88a3dfb1e12d59a53db1ed2&start=125)

Maybe this could be a good basis to get CoreOS booting on the pi3 using u-boot as already suggested by Josh above.

Regards,

Andreas

[Geoff Levand]

Hi,

On Thu, 2016-04-07 at 08:45 -0700, Andreas Mahnke wrote:
> I'm not sure about the UEFI support - by now I could not find any
> indicator that it is supported.
>
> The Kernel topic looks better though. Some folks where able to boot a 64bit Version 4.5 Kernel on the pi3 using u-boot. (See also https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963&sid=62597535a88a3dfb1e12d59a53db1ed2&start=125)
>
> Maybe this could be a good basis to get CoreOS booting on the pi3 using u-boot as already suggested by Josh above.

CoreOS just updated to the linux-4.5 kernel. I'll pull
those updates into my dev repos today [1]. I don't
know if all the board support that needed for pi3 is
enabled in the CoreOS kernel config file [2] though.
I'll enable what looks like it needs, but someone will
need to verify.

Just to mention it, CoreOS does not have wifi support.

To boot the system you'll need to write an arm64 CoreOS
disk image [3] to a storage device. If u-boot can boot
from fat32 formated USB storage, then I suggest trying
USB boot first.

[1] https://github.com/glevand/coreos--manifest.git
[2] https://github.com/glevand/coreos--coreos-overlay/blob/master/sys-kernel/coreos-kernel/files/arm64_defconfig-4.5
[3] https://github.com/glevand/hikey-coreos/releases

-Geoff

[Luke Wiersma]

Hey Guys!

I just ran across this group posting and found it very interesting because it's exactly what I would like to do.

I thought I would post some extra info I actually just ran across within the last week that might be useful to note for future development of Geoff's project.

I ran across this post on Phoronix about Raspberry Pi 3's BCM2837 SoC getting support in the Mainline Linux 4.8 Kernel:  http://www.phoronix.com/scan.php?page=news_item&px=ARM-Platforms-Linux-4.8

I also ran across a post on on Hackaday about Raspberry Pi 3 getting new Boot Modes, specifically USB Mass Storage, and Network Boot:  http://hackaday.com/2016/08/04/raspberry-pi-3-gets-usb-ethernet-boot/

Here are the two current post from Gordon Hollingworth about the current state of these two new Boot Modes:  https://www.raspberrypi.org/blog/pi-3-booting-part-i-usb-mass-storage-boot/  &  https://www.raspberrypi.org/blog/pi-3-booting-part-ii-ethernet-all-the-awesome/

Thought this might shed some extra light on the subject!

~Luke

[Bhima Pandava]

I also ran across a project which might help get Core OS running on ARM boards that use U-boot like the RPi and ODROID boards. It aims to enable UEFI on top of U-Boot on ARM.

Here's the post on HackerNews: https://news.ycombinator.com/item?id=12266370

There is also another project which might be helpful that is pointed out in the comment section: http://efidroid.org/

I've got no idea what parts of UEFI that Core OS requires but I would love to be able to run Core OS on a small cluster using ODROID C2 boards.

[Michael R. Lawrence CISSP]

https://github.com/sakaki-/gentoo-on-rpi3-64bit  one for gentoo 

can just overlay coreos with /etc/portage/repos.conf/coreos.conf  

add the git... etc.. 

has XFCE etc.. https://www.dropbox.com/s/jptnzaf1fgr2k5d/sakaki--gentoo-rpi3-ARM64bit.tar.gz?dl=0 

tarball with docker sabayon bits etc..  and packages... pilfer what you will... 

note nano ebuild.... KEYWORDS="~arm64" in gentoo , or related...    repoman.... etc... /ughh... 

[Geoff Levand]

On 06/13/2017 04:16 PM, Michael R. Lawrence CISSP wrote:
> can just overlay coreos with /etc/portage/repos.conf/coreos.conf
> add the git... etc..

If your system has ACPI and can PXE boot, then you can use the PXE
boot files

coreos_production_pxe.vmlinuz
coreos_production_pxe_image.cpio.gz

As I have mentioned in earlier posts in this thread, for hard disk
install CoreOS needs a platform that has UEFI and ACPI support.
Without these the release images and update engine can't work.

Without ACPI support you'll need to hack things up to use a DTB.

-Geoff

[Michael R. Lawrence CISSP]

Gentoo is the base for Google's Chrome os , and thus Coreos... based on Chrome..  (http://gpo.zugaina.org/)

also , as mentioned 

https://wiki.gentoo.org/wiki//etc/portage/repos.conf

/etc/portage/repos.conf/corseos.conf

[core-os]
location = /usr/local/overlay/core-os
sync-type = git
sync-uri = https://github.com/coreos/coreos-overlay.git

auto-sync = yes

make.conf features , buildpkg , getbinpkg 

Sakis repo , also add /usr/portage/packages...

if you can chroot , quickpkg  emerge -k/--binpkg  some.tar.gz...   , also can drop in 64 bit coreos packages folder into /usr/portage , emaint binhost --fix...

 

SAKAKI's Gentoo 64 bit for pi3 is alredy efi boot...(Gentoo Wiki by SAKAKI for pi3 )  use  Etcher for linux or windows no need to even untar , or rufus/DD/winimage32... 

might have to add a few other chrome or etc gentoo overlays... , however , it would give you a boot-able image with xfce , and allow for faster porting... 

anyhow its useful to Cherry pick package from core-os , via overlay in my case. 

Anyhow an Official coreos image for PI3 would be a nice to have esp with rocket , and  small....  especially if I get a few more.. pi3's ...

anyrate , it should give you a bootable Gentoo image , and making it more coreos shouldn''t be an issue... 

icecream/distcc  , cross-gcc-arm64 on amd64 hosts...   i've been bending packages from Sabayon /Pentoo Linux to pi3 based on the Gentoo Image , as they are more related to Gentoo... 

Note, the kernel config used was derived simply by running make bcmrpi3_defconfig on the rpi-4.10.y branch of the Raspberry Pi kernel tree; but, for reference, may be viewed here.

The image may be downloaded from the link below (or via wget, per the instructions which follow).

Variant	Version	Image	Digital Signature
Raspberry Pi 3 Model B 64-bit	v1.0.2	genpi64.img.xz	genpi64.img.xz.asc

Image binhost

I have made available a Portage binhost containing all the packages in this image, at https://isshoni.org/pi64. You may find this useful (to save compilation time) if building your own 64-bit system from scratch. For most users, adding/uncommenting the following lines in /etc/portage/make.conf (on your Pi) will suffice to start using automatically the provided binary packages, where available (and building locally - as usual - where not):

PORTAGE_BINHOST="https://isshoni.org/pi64"
FEATURES="${FEATURES} getbinpkg"