What is correct way to get U-Boot to boot into Raspbian kernel?

[Don Cross]

I'm working on a non-Yocto integration of Mender and U-Boot on a Raspberry Pi 3. Even though this is a 64-bit ARM, the Raspbian kernel is 32-bit (armv7l). When I build U-Boot as 64 bit ARM (CROSS_COMPILE=aarch64-linux-gnu- ARCH=aarch64) and copy u-boot.bin to the Pi as kernel8.img, the Pi's firmware picks it instead of kernel7.img and loads it correctly: I see U-Boot printing stuff to the screen, etc.

All I'm trying to do right now is get U-Boot to load and transfer control to kernel7.img. I'm not even worrying about Mender yet.  Looking at the following, I figured I just need to use U-Boot's "bootz" command:

don@spearmint:/media/don/F160-BF22 $ file kernel7.img

kernel7.img: Linux kernel ARM boot executable zImage (little-endian)

However, bootz is disabled by conditional compilation in 64-bit ARM builds of U-Boot. I hacked the U-Boot code to try to enable bootz (put CONFIG_CMD_BOOTZ=y in rpi_3_defconfig, changed some code in pxe.c to force it to call do_bootz()) but all this does is create a "Synchronous Abort" exception and resets the CPU repeatedly.

So at this point I figured I'm committing a crime against nature or something, and I back up and try building U-Boot as 32-bit ARM, perhaps to match the kernel: CROSS_COMPILE=arm-linux-gnueabi-, ARCH=arm.  This makes things even worse!  Now all I get when I put u-boot.bin on the Pi is the dreaded rainbow screen.

It gets more interesting.  When I use the Linux "file" command on both versions of u-boot.bin (32-bit and 64-bit), I get surprising differences:

The 64-bit build gives me:

don@spearmint:~/casa/uboot $ file u-boot.bin

u-boot.bin: PCX ver. 2.5 image data bounding box [8223, 54531] - [0, 8], 20-bit uncompressed

The 32-bit build gives me:

don@spearmint:~/casa/uboot $ file u-boot.bin

u-boot.bin: COM executable for DOS

What in the world?  This can't possibly be right. No wonder I get a rainbow screen on the 32-bit build.

I've looked at online sources such as this:

https://elinux.org/RPi_U-Boot

It was helpful to get me this far. I've also read about the mkimage utility, and I'm wondering if I need to use it to transform kernel7.img somehow. 

Can anyone give me a clue what the right way is to do this?  Let me know if I can provide more details to help.  I really would like to use Mender but this is blocking all progress.

Thanks in advance,

- Don

[Mirza Krak]

On 28 June 2018 at 00:19, Don Cross <cosin...@gmail.com> wrote:

Hi!

> I'm working on a non-Yocto integration of Mender and U-Boot on a Raspberry
> Pi 3. Even though this is a 64-bit ARM, the Raspbian kernel is 32-bit
> (armv7l). When I build U-Boot as 64 bit ARM
> (CROSS_COMPILE=aarch64-linux-gnu- ARCH=aarch64) and copy u-boot.bin to the
> Pi as kernel8.img, the Pi's firmware picks it instead of kernel7.img and
> loads it correctly: I see U-Boot printing stuff to the screen, etc
>

Great that you are hanging in there. Hopefully I can assist in moving
you forward.

First of I would try to avoid 64-bit on raspberrypi, at least
initially because as you have noticed the support is somewhat limited
and generally packages are not as stable as for the 32-bit platform.
Especially GPU drivers.

I do not get the full picture of what commands you have executed but I
would suggest something like this, v2017.09 is used because that is
what we are using in Yocto, so might as well keep the versions the
same.

$ git clone git://git.denx.de/u-boot.git -b v2017.09
$ cd u-boot/
$ export CROSS_COMPILE=arm-linux-gnueabihf-
$ export ARCH=arm
$ make rpi_3_32b_config
$ make

Above should produce a valid u-boot.bin file.

As you have noticed, the raspberrypi boots either kernel7.img or
kernel8.img from the boot part.

Next steps are to get the u-boot.bin to boot:

- Make a copy of the original kernel7.img > zImage
- Copy u-boot.bin to the boot part on the SD card and call it kernel7.img

That should make sure that U-boot starts up.

Next step is to make sure that U-boot loads the Linux kernel and boots
it, for this you need to provide a boot.scr file which you put it in
the boot partition. U-boot probes for this and will automatically pick
it up.

The script content should be something like this:

fdt addr ${fdt_addr} && fdt get value bootargs /chosen bootargs
fatload mmc 0:1 ${kernel_addr_r} zImage
bootz ${kernel_addr_r} - ${fdt_addr}

You can put above in a file called boot.cmd.in, then we need to apply
an U-boot header to that file with the mentioned mkimage tool with the
following:

mkimage -A arm -T script -C none -n "Boot script" -d boot.cmd.in boot.scr

This will create the boot.scr, which you then copy to the boot part on
the SD card and this should result in that your device boots.

Hopefully this will help you progress.

/ Mirza

[Mirza Krak]

Also the following packages are needed (assuming debian based system):

apt-get install gcc-arm-linux-gnueabihf device-tree-compiler bc

/ Mirza

[Don Cross]

Thank you very much, Mirza!  I'm pretty happy now because I was able to boot the Raspbian kernel from U-Boot for the first time. I was so close before and didn't know it.  I was already using U-Boot v2017.09 source code, and I had already installed the gcc-arm-linux-gnueabihf cross compiler, and I tried the same CROSS_COMPILE and ARCH environment values you suggested, only I got a rainbow screen when I named u-boot.bin as kernel8.img.  But naming it kernel7.img causes it to work! There must be something in the Raspberry Pi's ROM that treats those 2 names differently.

So I guess the next step is to replace the boot.scr with Mender's boot command logic, in include/env_mender.h?  (I think I have applied patches correctly to U-Boot source to make this work.)  If I'm reading what you wrote in your sample boot script correctly...

fdt addr ${fdt_addr} && fdt get value bootargs /chosen bootargs

fatload mmc 0:1 ${kernel_addr_r} zImage

bootz ${kernel_addr_r} - ${fdt_addr}

... the "fatload" part is loading the kernel file zImage from the U-Boot partition, not the ROOTFS A (or B) dual partitions. Is it possible to load the image from my ext4 A/B partitions instead of the fat32 U-Boot partition?  My best understanding is that the actual boot logic for getting Mender to work will look something like this:

    run mender_setup;

    setenv bootargs root=${mender_kernel_root} ${bootargs};

    if test "${fdt_addr_r}" != ""; then

        load ${mender_uboot_root} ${fdt_addr_r} /boot/${mender_dtb_name}

    fi

    load ${mender_uboot_root} ${kernel_addr_r} /boot/${mender_kernel_name}

    bootz ${kernel_addr_r} - ${fdt_addr_r}

    run mender_try_to_recover

Of course, I'm leaving out all my custom definitions for environment variables like mender_kernel_root, etc. In this case, the "load" command (I hope) works on ext4 partitions, so I don't need "fatload"?

I just want to make sure I'm heading down the right road before I waste too much time.

Thanks again!

Don

 

[Mirza Krak]

Yeah the kernel8.img file is "64-bit" mode. 

So I guess the next step is to replace the boot.scr with Mender's boot command logic, in include/env_mender.h?  (I think I have applied patches correctly to U-Boot source to make this work.)  If I'm reading what you wrote in your sample boot script correctly...

fdt addr ${fdt_addr} && fdt get value bootargs /chosen bootargs

fatload mmc 0:1 ${kernel_addr_r} zImage

bootz ${kernel_addr_r} - ${fdt_addr}

... the "fatload" part is loading the kernel file zImage from the U-Boot partition, not the ROOTFS A (or B) dual partitions. Is it possible to load the image from my ext4 A/B partitions instead of the fat32 U-Boot partition?  My best understanding is that the actual boot logic for getting Mender to work will look something like this:

    run mender_setup;

    setenv bootargs root=${mender_kernel_root} ${bootargs};

    if test "${fdt_addr_r}" != ""; then

        load ${mender_uboot_root} ${fdt_addr_r} /boot/${mender_dtb_name}

    fi

    load ${mender_uboot_root} ${kernel_addr_r} /boot/${mender_kernel_name}

    bootz ${kernel_addr_r} - ${fdt_addr_r}

    run mender_try_to_recover

Of course, I'm leaving out all my custom definitions for environment variables like mender_kernel_root, etc. In this case, the "load" command (I hope) works on ext4 partitions, so I don't need "fatload"?

I just want to make sure I'm heading down the right road before I waste too much time.

There is a script in our Yocto environment that could work as inspiration as well and yeah you are on the correct path :). 

Note that the script we use in Yocto actually loads the kernel from /boot on the active root file system. We can do this becaus we make sure it is installed there during build. I do not know if Rasbian does this as well by default, if it does not it might be tricky, and you would be forced to load it from the boot part unless you modify the rootfs images to contain the kernel file. 

Also make sure to check this out, https://tracker.mender.io/browse/MEN-1812. Ongoing work to make it easier to integrate Mender with the likes of Rasbian. 

/ Mirza

[1]. https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-raspberrypi/recipes-bsp/rpi-u-boot-scr/files/boot.cmd.in