Kernel not booting correctly with Yocto / Mender / U-boot
1,257 views
Skip to first unread message
Jon Kolstad
Jul 25, 2018, 9:49:25 AM7/25/18
to Mender List mender.io
Hi!
U-boot environment variables:
Conf file in meta-mylayer/conf/layer.conf:
So I've built an image for the board Firefly-RK3399 using Yocto (Rocko-branch) with Mender, but it doesn't want to boot.
The build uses these primary layers:
- oe-core (Rocko-branch)
- oe-meta-go (Master-branch)
- meta-rockchip (containing the kernel and other board specific configurations) (Rocko-branch)
- meta-mender-core (Rocko-branch)
- meta-mylayer (to provide some necessary basic configuration and patching)
I've previously made a working manual build for the Firefly-RK3399 with Mender, but without Yocto.
I've applied the same custom patches that I had to with the manual build, and the same Mender/U-boot integration points (both through automatic and manual Mender/U-boot patching outlined in https://docs.mender.io/1.5/devices/integrating-with-u-boot/manual-u-boot-integration) for this Yocto build. My patches are by the way made with the Yocto devtool, and they are applied as intended.
The build process works fine, and the flashing is done correctly with respect to the partition layout requirements of Mender. The errors occur when booting.
I've attached a log of the serial output generated when booting the board with the build, below. If you look closely after it is starting autoboot, you can see the following error it produces when it is supposed to read and start the kernel:
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Bad Linux ARM64 Image magic!Note: The filesystem is ext4.
It then restarts, and loops through the same procedure, producing the same error every time, until autoboot is stopped.
The attached log shows one run through to the error, then stopped at the next autoboot. Further are the U-boot environmental variables, and my layer.conf-file.
The problem might still be caused by a faulty Mender U-boot integration, or it might be the kernel (but I think not).
I am stuck on this error, maybe any of you have ideas as to what is wrong, or what I can try next. Even though I've been looking into this for a while, I am by no means an expert. :)
Booting:
DDR Version 1.08 20170320
In
Channel 0: DDR3, 800MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
Channel 1: DDR3, 800MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
256B stride
ch 0 ddrconfig = 0x101, ddrsize = 0x20
ch 1 ddrconfig = 0x101, ddrsize = 0x20
pmugrf_os_reg[2] = 0x32817281, stride = 0x9
OUT
Boot1: 2017-04-07, version: 1.06
CPUId = 0x0
ChipType = 0x10, 1832
SdmmcInit=2 0
BootCapSize=100000
UserCapSize=14910MB
FwPartOffset=2000 , 100000
SdmmcInit=0 20
StorageInit ok = 66005
LoadTrustBL
No find bl30.bin
No find bl32.bin
RunBL31 0x10000
WARNING: plat_rockchip_comm_sram_init:code:ff8d9000, 67000, ef0
NOTICE: BL31: v1.2(debug):b995f80
NOTICE: BL31: Built : 16:53:56, Nov 7 2016
INFO: GICv3 with legacy support detected. ARM GICV3 driver initialized in EL3
INFO: boot cpu mask: 0
INFO: Vesion: 2016.0.01
INFO: plat_rockchip_pmu_init(1211): pd status 3e
INFO: BL31: Initializing runtime services
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0x200000
INFO: SPSR = 0x3c9
U-Boot 2017.09-00595-g515bbbdeca-dirty (Jul 23 2018 - 11:05:59 +0000)
Model: Firefly-RK3399 Board
DRAM: 2 GiB
MMC: dwmmc@fe320000: 1, sdhci@fe330000: 0
*** Warning - bad CRC, using default environment
In: serial@ff1a0000
Out: serial@ff1a0000
Err: serial@ff1a0000
Model: Firefly-RK3399 Board
Net: eth0: ethernet@fe300000
Hit any key to stop autoboot: 0
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Bad Linux ARM64 Image magic!
DDR Version 1.08 20170320
In
soft reset
SRX
Channel 0: DDR3, 800MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
Channel 1: DDR3, 800MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
256B stride
ch 0 ddrconfig = 0x101, ddrsize = 0x20
ch 1 ddrconfig = 0x101, ddrsize = 0x20
pmugrf_os_reg[2] = 0x32817281, stride = 0x9
OUT
Boot1: 2017-04-07, version: 1.06
CPUId = 0x0
ChipType = 0x10, 1899
SdmmcInit=2 0
BootCapSize=100000
UserCapSize=14910MB
FwPartOffset=2000 , 100000
SdmmcInit=0 20
StorageInit ok = 66720
LoadTrustBL
No find bl30.bin
No find bl32.bin
RunBL31 0x10000
WARNING: plat_rockchip_comm_sram_init:code:ff8d9000, 67000, ef0
NOTICE: BL31: v1.2(debug):b995f80
NOTICE: BL31: Built : 16:53:56, Nov 7 2016
INFO: GICv3 with legacy support detected. ARM GICV3 driver initialized in EL3
INFO: boot cpu mask: 0
INFO: Vesion: 2016.0.01
INFO: plat_rockchip_pmu_init(1211): pd status 3e
INFO: BL31: Initializing runtime services
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0x200000
INFO: SPSR = 0x3c9
U-Boot 2017.09-00595-g515bbbdeca-dirty (Jul 23 2018 - 11:05:59 +0000)
Model: Firefly-RK3399 Board
DRAM: 2 GiB
MMC: dwmmc@fe320000: 1, sdhci@fe330000: 0
*** Warning - bad CRC, using default environment
In: serial@ff1a0000
Out: serial@ff1a0000
Err: serial@ff1a0000
Model: Firefly-RK3399 Board
Net: eth0: ethernet@fe300000
Hit any key to stop autoboot: 0
U-boot environment variables:
=> printenv
altbootcmd=run mender_altbootcmd; run bootcmd
arch=arm
baudrate=1500000
board=evb_rk3399
board_name=evb_rk3399
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
boot_efi_binary=load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_r} efi/boot/bootaa64.efi; if fdt addr ${fdt_addr_r}; then bootefi ${kernel_addr_r} ${fdt_addr_r};else bootefi ${kernel_addr_r} ${fdtconti
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}extlinux/extlinux.conf
boot_net_usb_start=usb start
boot_prefixes=/ /boot/
boot_script_dhcp=boot.scr.uimg
boot_scripts=boot.scr.uimg boot.scr
boot_targets=mmc0 mmc1 usb0 pxe dhcp
bootcmd=run mender_setup; load ${mender_uboot_root} ${kernel_addr_r} ${mender_uboot_root}/boot/Image; load ${mender_uboot_root} ${fdt_addr_r} ${mender_uboot_root}/boot/rk3399-firefly-linux.dtb; setenv bootargs r;
bootcmd_dhcp=run boot_net_usb_start; if dhcp ${scriptaddr} ${boot_script_dhcp}; then source ${scriptaddr}; fi;setenv efi_fdtfile ${fdtfile}; setenv efi_old_vci ${bootp_vci};setenv efi_old_arch ${bootp_arch};sete;
bootcmd_mmc0=setenv devnum 0; run mmc_boot
bootcmd_mmc1=setenv devnum 1; run mmc_boot
bootcmd_pxe=run boot_net_usb_start; dhcp; if pxe get; then pxe boot; fi
bootcmd_usb0=setenv devnum 0; run usb_boot
bootcount=1
bootdelay=2
bootlimit=1
cpu=armv8
cpuid#=544d533631362e303000000000088403
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done
efi_dtb_prefixes=/ /dtb/ /dtb/current/
ethaddr=16:54:50:77:00:8b
fdt_addr_r=0x01f00000
fdtcontroladdr=7def3bc8
kernel_addr_r=0x02000000
load_efi_dtb=load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} ${prefix}${efi_fdtfile}
mender_altbootcmd=if test ${mender_boot_part} = 2; then setenv mender_boot_part 3; else setenv mender_boot_part 2; fi; setenv upgrade_available 0; saveenv; run mender_setup
mender_boot_kernel_type=booti
mender_boot_part=2
mender_dtb_name=rk3399-firefly-linux.dtb
mender_kernel_name=Image
mender_setup=setenv mender_kernel_root /dev/mmcblk0p${mender_boot_part}; if test ${mender_boot_part} = 2; then setenv mender_boot_part_name /dev/mmcblk0p2; else setenv mender_boot_part_name /dev/mmcblk0p3; fi; ss
mender_try_to_recover=if test ${upgrade_available} = 1; then reset; fi
mender_uboot_boot=mmc 0:1
mender_uboot_dev=0
mender_uboot_if=mmc
mmc_boot=if mmc dev ${devnum}; then setenv devtype mmc; run scan_dev_for_boot_part; fi
partitions=uuid_disk=${uuid_gpt_disk};name=loader1,start=32K,size=4000K,uuid=${uuid_gpt_loader1};name=loader2,start=8MB,size=4MB,uuid=${uuid_gpt_loader2};name=trust,size=4M,uuid=${uuid_gpt_atf};name=boot,size=11;
pxefile_addr_r=0x00600000
ramdisk_addr_r=0x04000000
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;run scan_dev_for_efi;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfse
scan_dev_for_efi=setenv efi_fdtfile ${fdtfile}; for prefix in ${efi_dtb_prefixes}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${efi_fdtfile}; then run load_efi_dtb; fi;done;if test -e ${devtye
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}extlinux/extlinux.conf; then echo Found ${prefix}extlinux/extlinux.conf; run boot_extlinux; echo SCRIPT FAILED: continuing...; fi
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILe
scriptaddr=0x00500000
serial#=df26ab7f5bf9c0d
soc=rockchip
stderr=serial@ff1a0000
stdin=serial@ff1a0000
stdout=serial@ff1a0000
upgrade_available=0
usb_boot=usb start; if usb dev ${devnum}; then setenv devtype usb; run scan_dev_for_boot_part; fi
vendor=rockchip
Environment size: 5268/131067 bytesConf file in meta-mylayer/conf/layer.conf:
# We have a conf and classes directory, add to BBPATH BBPATH .= ":${LAYERDIR}" # We have recipes-* directories, add to BBFILES BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ ${LAYERDIR}/recipes-*/*/*.bbappend" BBFILE_COLLECTIONS += "../meta-mylayer" BBFILE_PATTERN_../meta-mylayer = "^${LAYERDIR}/" BBFILE_PRIORITY_../meta-mylayer = "6" # MIRRORS SSTATE_MIRRORS = "\ " # MACHINE MACHINE = "firefly-rk3399" # MENDER
MENDER_ARTIFACT_NAME = "mylayer-yocto-build-0" INHERIT += "mender-full" DISTRO_FEATURES_append = " systemd" VIRTUAL-RUNTIME_init_manager = "systemd" DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" VIRTUAL-RUNTIME_initscripts = "" ARTIFACTIMG_FSTYPE = "ext4" MENDER_FEATURES_ENABLE_append = " mender-uboot" Cheers!
- Jon
Mirza Krak
Jul 25, 2018, 10:02:13 AM7/25/18
to Mender List mender.io
So I've built an image for the board Firefly-RK3399 using Yocto (Rocko-branch) with Mender, but it doesn't want to boot.
Cool, I have some experience with RK3288 based boards and hopefully that is helpful in this case :).
The build uses these primary layers:- oe-core (Rocko-branch)- oe-meta-go (Master-branch)
This is probably not related to your error but you should not use "oe-meta-go", golang is part of oe-core as of "pyro" release and it is expected that the version is used.
- meta-rockchip (containing the kernel and other board specific configurations) (Rocko-branch)- meta-mender-core (Rocko-branch)- meta-mylayer (to provide some necessary basic configuration and patching)I've previously made a working manual build for the Firefly-RK3399 with Mender, but without Yocto.I've applied the same custom patches that I had to with the manual build, and the same Mender/U-boot integration points (both through automatic and manual Mender/U-boot patching outlined in https://docs.mender.io/1.5/devices/integrating-with-u-boot/manual-u-boot-integration) for this Yocto build. My patches are by the way made with the Yocto devtool, and they are applied as intended.The build process works fine, and the flashing is done correctly with respect to the partition layout requirements of Mender. The errors occur when booting.I've attached a log of the serial output generated when booting the board with the build, below. If you look closely after it is starting autoboot, you can see the following error it produces when it is supposed to read and start the kernel:
** Unrecognized filesystem type **
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
Bad Linux ARM64 Image magic!Note: The filesystem is ext4.It then restarts, and loops through the same procedure, producing the same error every time, until autoboot is stopped.The attached log shows one run through to the error, then stopped at the next autoboot. Further are the U-boot environmental variables, and my layer.conf-file.The problem might still be caused by a faulty Mender U-boot integration, or it might be the kernel (but I think not).I am stuck on this error, maybe any of you have ideas as to what is wrong, or what I can try next. Even though I've been looking into this for a while, I am by no means an expert. :)
This is an extract of your bootcmd:
run mender_setup;
load ${mender_uboot_root} ${kernel_addr_r} ${mender_uboot_root}/boot/Image;
load ${mender_uboot_root} ${fdt_addr_r} ${mender_uboot_root}/boot/rk3399-firefly-linux.dtb;
setenv bootargs r;
Above does not seem correct to me and should probably be:
load ${mender_uboot_root} ${kernel_addr_r} /boot/Image;
load ${mender_uboot_root} ${fdt_addr_r} /boot/rk3399-firefly-linux.dtb;
--
Jon Kolstad
Jul 30, 2018, 7:16:50 AM7/30/18
to Mender List mender.io, mirza...@northern.tech
Thank you for your quick reply, Mirza.
I also tried,
You are correct in pointing out the bootcmd errors. I was not aware that I didn't copy it correctly, the entire bootcmd is actually:
run mender_setup;load ${mender_uboot_root} ${kernel_addr_r} ${mender_boot_root}/boot/Image;load ${mender_uboot_root} ${fdt_addr_r} ${mender_boot_root}/boot/rk3399-firefly-linux.dtb;setenv bootargs root=${mender_kernel_root} console=ttyFIQ0,1500000n8;booti ${kernel_addr_r} - ${fdt_addr_r} ;reset;\0run mender_try_to_recover;run mender_setup;load ${mender_uboot_root} ${kernel_addr_r} /boot/Image;load ${mender_uboot_root} ${fdt_addr_r} /boot/rk3399-firefly-linux.dtb;setenv bootargs root=${mender_kernel_root} console=ttyFIQ0,1500000n8;booti ${kernel_addr_r} - ${fdt_addr_r} ;reset;\0run mender_try_to_recover;However, I tried printing ${mender_uboot_root} while in the u-boot environment, but it's undefined. Do you know the format of this variable? I looks to be defined in mender_setup, but still. I could perhaps hardcode it to check the result for further debugging.
Do you have any other suggestions, maybe some other configuration or variables I haven't pasted that might cause the issues?
- Jon
Drew Moseley
Jul 30, 2018, 12:00:51 PM7/30/18
to men...@lists.mender.io, Jon Kolstad
Hi Jon,
The mender_uboot_root variable should be something like (from my Beagleboard setup) "mmc 0:2". It's a normal uboot specification of a device and partition.
It is defined in mender_setup as "setenv mender_uboot_root mmc 0:${mender_boot_part}" but may differ depending on your board if, for instance, you are on mmc device 1. The mender_boot_part variable should exist in the default UBoot environment.
HTH,
Drew
The mender_uboot_root variable should be something like (from my Beagleboard setup) "mmc 0:2". It's a normal uboot specification of a device and partition.
It is defined in mender_setup as "setenv mender_uboot_root mmc 0:${mender_boot_part}" but may differ depending on your board if, for instance, you are on mmc device 1. The mender_boot_part variable should exist in the default UBoot environment.
HTH,
Drew
--
You received this message because you are subscribed to the Google Groups "Mender List mender.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mender+un...@lists.mender.io.
To post to this group, send email to men...@lists.mender.io.
Visit this group at https://groups.google.com/a/lists.mender.io/group/mender/.
Kristian Amlie
Aug 1, 2018, 4:20:11 AM8/1/18
to men...@lists.mender.io, Drew Moseley, Jon Kolstad
We talked in person, so I won't go into the details you posted below.
But one patch that may be interesting for you since you are on ARM64 was
posted by a different user. Check out this thread:
https://groups.google.com/a/lists.mender.io/d/msg/mender/GR0jQjlDtYQ/d2i6g7CTBQAJ
Maybe you have this covered already, but I think it might be needed to
get ARM64 to work.
--
Kristian
>> <https://docs.mender.io/1.5/devices/integrating-with-u-boot/manual-u-boot-integration>)
>> Northern.tech AS <https://northern.tech> | @northerntechHQ
>> <https://twitter.com/northerntechhq>
> <https://twitter.com/northerntechhq> | @drewmoseley
> <https://twitter.com/drewmoseley>
But one patch that may be interesting for you since you are on ARM64 was
posted by a different user. Check out this thread:
https://groups.google.com/a/lists.mender.io/d/msg/mender/GR0jQjlDtYQ/d2i6g7CTBQAJ
Maybe you have this covered already, but I think it might be needed to
get ARM64 to work.
--
Kristian
>> <https://twitter.com/northerntechhq>
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Mender List mender.io" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to mender+un...@lists.mender.io
>> <mailto:mender+un...@lists.mender.io>.
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Mender List mender.io" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to mender+un...@lists.mender.io
>> To post to this group, send email to men...@lists.mender.io
>> <mailto:men...@lists.mender.io>.
>> Visit this group at
>> https://groups.google.com/a/lists.mender.io/group/mender/.
>
> --
> Drew Moseley | Technical Solutions Architect | (+1) 480-797-0552 |
> https://mender.io
>
> Northern.tech AS <https://northern.tech> | @northerntechHQ
>> https://groups.google.com/a/lists.mender.io/group/mender/.
>
> --
> Drew Moseley | Technical Solutions Architect | (+1) 480-797-0552 |
> https://mender.io
>
> <https://twitter.com/northerntechhq> | @drewmoseley
> <https://twitter.com/drewmoseley>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Mender List mender.io" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to mender+un...@lists.mender.io
> <mailto:mender+un...@lists.mender.io>.
> --
> You received this message because you are subscribed to the Google
> Groups "Mender List mender.io" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to mender+un...@lists.mender.io
> To post to this group, send email to men...@lists.mender.io
> <mailto:men...@lists.mender.io>.
Jon Kolstad
Aug 1, 2018, 5:01:35 AM8/1/18
to Mender List mender.io, drew.m...@northern.tech, jon....@gmail.com
Thanks alot for the help guys, I have some clues to work on now.
Will update you on any further progress!
- Jon
Reply all
Reply to author
Forward
0 new messages