Booting with sysboot

[David M]

Hi,

At my day job I work with the Nvidia TX2, I need to make changes to the kernel every once in a while and the process of introducing an updated kernel on that platform is a lot easier than it is on this platform. When I want to load a new kernel all I need to do is build it on my desktop and scp it over to TX2 (along with the DTB) to the /boot directory then reset the board and everything is updated. There are some caveats, but for the most part it's very simple.

I would like to bring that same functionality to the Snickerdoodle board. I have been learning about this functionality and apparently it's called 'sysboot'. I've enabled this on my u-boot by going into uboot config menu, searching for distro_defaults and enabling it.

BOOT.bif

Now my boot.bif only consists of the fsbl and uboot.elf.

U-Boot Environmental Variables

Instead of modifying the uEnv.txt I modified the snickerdoodle-uboot/include/configs/zynq_snickerdoodle_black.h and redefined the 'CONFIG_EXTRA_ENV_SETTINGS' and adding a new script:

...  

"kernel_addr_r=0x2080000\0" \

"ramdisk_addr_r=0x4000000\0" \

"pxefile_addr_r=0x3000000\0" \

"fdt_addr=0x2000000\0" \

"extboot=echo Load Linux from EXT... && " \
  "mmcinfo && " \
  "ext4load mmc 0:2 ${fdt_addr} /boot/zynq-snickerdoodle-black.dtb && " \
  "sysboot mmc 0:2 ext2 0x5000000 /boot/extlinux/extlinux.conf\0" \
...

(I apologize for hardcoding everything. This is a first draft, I plan to add more global variables but at the moment I just wanted to get it up and running)

/boot/extlinux/extlinux.conf

This also requires users to write a simple 'extlinux.conf' file within the /boot/extlinux/ directory of the rootfs and copy the kernel uImage and dtb (from the kernel) to the /boot directory.

Mine looks like this:

TIMEOUT 30
DEFAULT primary


MENU TITLE Snickerdoodle SDCard Boot


LABEL primary
      MENU LABEL Primary Kernel (SD Card)
      LINUX /boot/uImage
      FDT devicetree.dtb
      APPEND console=ttyPS0,115200n8 root=/dev/mmcblk0p2 rootfstype=ext4 rw rootwait consoleblank=0 devtmpfs.mount=1
      INITRD /boot/uramdisk.image.gz


LABEL debug
      MENU LABEL Debug Kernel (SD Card)
      LINUX /boot/Image
      FDT zynq-snickerdoodle-black.dtb
      APPEND console=ttyPS0,115200n8 root=/dev/mmcblk0p2 rootfstype=ext4 rw rootwait earlyprintk=/dev/ttyPS0 consoleblank=0 devtmpfs.mount=1
      INITRD /boot/uramdisk.image.gz

I originally used the Linux kernel 'Image' file instead of the 'uImage' file but it took 40 seconds to load Image to the ram address to 0x8000 so I've been using uImage instead. (I'm also using a slower SD Card so this could be modified in the future too)

Bootup Status

So far bootup has been working but I only get as far as the ramdisk. I can see that the ramdisk does recognize /dev/mmcblk0p2 and that it see it is an EXT4 but I don't know how to get it to do the last step of using EXT4 as the ROOTFS.

I'm new at getting this aspect of the kernel boot process and if anybody has any advice on how to get the kernel to switch over I would be very appreciative.

I've started a blog entry about this here (Note: it is in rough form): Snickerdoodle Sysboot

As soon as we can get all the bugs out of this I'll update the entry.

Here is the boot process

U-Boot 2017.01snickerdoodle-07040-g27c02a8-dirty (Jan 22 2018 - 05:52:54 -0500)


Model: snickerdoodle black
Board: Xilinx Zynq
DRAM:  ECC disabled 1 GiB
MMC:   sdhci@e0100000: 0
SF: unrecognized JEDEC id bytes: 20, bb, 18
*** Warning - spi_flash_probe() failed, using default environment


In:    serial@e0000000
Out:   serial@e0000000
Err:   serial@e0000000
Model: snickerdoodle black
Board: Xilinx Zynq
Net:   No ethernet found.
** Bad device size - mmc 0 **
Hit any key to stop autoboot:  3
Load Linux from EXT...
Device: sdhci@e0100000
Manufacturer ID: 3
OEM: 5344
Name: SS08G
Tran Speed: 50000000
Rd Block Len: 512
SD version 3.0
High Capacity: Yes
Capacity: 7.4 GiB
Bus Width: 4-bit
Erase Group Size: 512 Bytes
10326 bytes read in 72 ms (139.6 KiB/s)
Retrieving file: /boot/extlinux/extlinux.conf
623 bytes read in 69 ms (8.8 KiB/s)
Snickerdoodle SDCard Boot
1: Primary Kernel (SD Card)
2: Debug Kernel (SD Card)
Enter choice: 1: Primary Kernel (SD Card)
Retrieving file: /boot/uramdisk.image.gz
5310018 bytes read in 324 ms (15.6 MiB/s)
Retrieving file: /boot/uImage
5669512 bytes read in 328 ms (16.5 MiB/s)
append: console=ttyPS0,115200n8 root=/dev/mmcblk0p2 rootfstype=ext4 rw rootwait consoleblank=0 devtmpfs.mount=1
## Booting kernel from Legacy Image at 02080000 ...
   Image Name:   Linux-4.9.0-snickerdoodle-68561-
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5669448 Bytes = 5.4 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 04000000 ...
   Image Name:   snickerdoodle QSPI ramdisk
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    5309954 Bytes = 5.1 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
   Loading Kernel Image ... OK
   Loading Ramdisk to 1faef000, end 1ffff602 ... OK
   Loading Device Tree to 1fae9000, end 1faee855 ... OK


Starting kernel ...


Uncompressing Linux... done, booting the kernel.
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.0-snickerdoodle-68561-gbe1c511-dirty (cospan@coati) (gcc version 6.2.1 20161114 (Linaro GCC Snapshot 6.2-2016.11) ) #1 SMP PREEMPT Wed Jan 10 09:06:44 EST 2018
[    0.000000] CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt:Machine model: snickerdoodle black
[    0.000000] cma: Reserved 256 MiB at 0x30000000
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] percpu: Embedded 15 pages/cpu @af750000 s30604 r8192 d22644 u61440
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 260096
[    0.000000] Kernel command line: console=ttyPS0,115200n8 root=/dev/mmcblk0p2 rootfstype=ext4 rw rootwait consoleblank=0 devtmpfs.mount=1
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 756540K/1048576K available (8192K kernel code, 627K rwdata, 3192K rodata, 1024K init, 563K bss, 29892K reserved, 262144K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xc0800000 - 0xff800000   (1008 MB)
[    0.000000]     lowmem  : 0x80000000 - 0xc0000000   (1024 MB)
[    0.000000]     pkmap   : 0x7fe00000 - 0x80000000   (   2 MB)
[    0.000000]     modules : 0x7f000000 - 0x7fe00000   (  14 MB)
[    0.000000]       .text : 0x80008000 - 0x80900000   (9184 kB)
[    0.000000]       .init : 0x80d00000 - 0x80e00000   (1024 kB)
[    0.000000]       .data : 0x80e00000 - 0x80e9cda0   ( 628 kB)
[    0.000000]        .bss : 0x80e9cda0 - 0x80f299b4   ( 564 kB)
[    0.000000] Preemptible hierarchical RCU implementation.
[    0.000000] Build-time adjustment of leaf fanout to 32.
[    0.000000] RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=2
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] efuse mapped to c0800000
[    0.000000] slcr mapped to c0802000
[    0.000000] L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
[    0.000000] L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
[    0.000000] L2C-310 erratum 769419 enabled
[    0.000000] L2C-310 enabling early BRESP for Cortex-A9
[    0.000000] L2C-310 full line of zeros enabled for Cortex-A9
[    0.000000] L2C-310 ID prefetch enabled, offset 1 lines
[    0.000000] L2C-310 dynamic clock gating enabled, standby mode enabled
[    0.000000] L2C-310 cache controller enabled, 8 ways, 512 kB
[    0.000000] L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
[    0.000000] zynq_clock_init: clkc starts at c0802100
[    0.000000] Zynq clock init
[    0.000010] sched_clock: 64 bits at 433MHz, resolution 2ns, wraps every 4398046511103ns
[    0.000032] clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x63f09fe9cc, max_idle_ns: 440795221256 ns
[    0.000064] Switching to timer-based delay loop, resolution 2ns
[    0.000177] clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff, max_idle_ns: 413490550 ns
[    0.000212] timer #0 at c080a000, irq=17
[    0.000701] Console: colour dummy device 80x30
[    0.000726] Calibrating delay loop (skipped), value calculated using timer frequency.. 866.66 BogoMIPS (lpj=4333333)
[    0.000743] pid_max: default: 32768 minimum: 301
[    0.001022] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.001038] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.001829] CPU: Testing write buffer coherency: ok
[    0.001867] ftrace: allocating 28009 entries in 83 pages
[    0.050430] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.050500] Setting up static identity map for 0x100000 - 0x100058
[    0.250789] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[    0.250907] Brought up 2 CPUs
[    0.250929] SMP: Total of 2 processors activated (1733.33 BogoMIPS).
[    0.250939] CPU: All CPU(s) started in SVC mode.
[    0.251916] devtmpfs: initialized
[    0.255515] VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
[    0.255838] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.269052] pinctrl core: initialized pinctrl subsystem
[    0.270770] NET: Registered protocol family 16
[    0.272924] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.300865] cpuidle: using governor ladder
[    0.340806] cpuidle: using governor menu
[    0.348752] hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
[    0.348771] hw-breakpoint: maximum watchpoint size is 4 bytes.
[    0.348914] zynq-ocm f800c000.ocmc: ZYNQ OCM pool: 256 KiB @ 0xc0840000
[    0.349350] zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
[    0.399920] OF: /fixedregulator@1: could not get #gpio-cells for /amba/interrupt-controller@f8f01000
[    0.399957] reg-fixed-voltage: probe of fixedregulator@1 failed with error -22
[    0.400446] vgaarb: loaded
[    0.401173] SCSI subsystem initialized
[    0.401978] usbcore: registered new interface driver usbfs
[    0.402161] usbcore: registered new interface driver hub
[    0.402357] usbcore: registered new device driver usb
[    0.402693] media: Linux media interface: v0.10
[    0.402835] Linux video capture interface: v2.00
[    0.403022] pps_core: LinuxPPS API ver. 1 registered
[    0.403036] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.403135] PTP clock support registered
[    0.403322] EDAC MC: Ver: 3.0.0
[    0.404612] FPGA manager framework
[    0.407211] clocksource: Switched to clocksource arm_global_timer
[    0.497724] VFS: Disk quotas dquot_6.6.0
[    0.497828] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.498079] FS-Cache: Loaded
[    0.498460] CacheFiles: Loaded
[    0.515997] NET: Registered protocol family 2
[    0.516749] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
[    0.516838] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
[    0.516955] TCP: Hash tables configured (established 8192 bind 8192)
[    0.517038] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.517091] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.517396] NET: Registered protocol family 1
[    0.517932] RPC: Registered named UNIX socket transport module.
[    0.517947] RPC: Registered udp transport module.
[    0.517956] RPC: Registered tcp transport module.
[    0.517965] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.518238] Trying to unpack rootfs image as initramfs...
[    0.518916] rootfs image is not initramfs (no cpio magic); looks like an initrd
[    0.548458] Freeing initrd memory: 5188K (9faef000 - a0000000)
[    0.548924] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
[    0.550913] futex hash table entries: 512 (order: 3, 32768 bytes)
[    0.552318] workingset: timestamp_bits=30 max_order=18 bucket_order=0
[    0.552960] FS-Cache: Netfs 'nfs' registered for caching
[    0.553204] NFS: Registering the id_resolver key type
[    0.553237] Key type id_resolver registered
[    0.553248] Key type id_legacy registered
[    0.553301] jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
[    0.557987] io scheduler noop registered
[    0.558005] io scheduler deadline registered
[    0.558038] io scheduler cfq registered (default)
[    0.561511] dma-pl330 f8003000.dmac: Loaded driver for PL330 DMAC-241330
[    0.561535] dma-pl330 f8003000.dmac: DBUFF-128x8bytes Num_Chans-8 Num_Peri-4 Num_Events-16
[    0.562694] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.565078] e0000000.serial: ttyPS0 at MMIO 0xe0000000 (irq = 25, base_baud = 6249999) is a xuartps
[    1.282619] console [ttyPS0] enabled
[    1.286959] xdevcfg f8007000.devcfg: ioremap 0xf8007000 to c0828000
[    1.309537] brd: module loaded
[    1.321535] loop: module loaded
[    1.329210] libphy: Fixed MDIO Bus: probed
[    1.333233] CAN device driver interface
[    1.338791] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.345242] ehci-pci: EHCI PCI platform driver
[    1.350044] usbcore: registered new interface driver cdc_acm
[    1.355624] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[    1.363909] usbcore: registered new interface driver usbserial
[    1.369818] usbcore: registered new interface driver usbserial_generic
[    1.376380] usbserial: USB Serial support registered for generic
[    1.382896] mousedev: PS/2 mouse device common for all mice
[    1.389014] i2c /dev entries driver
[    1.394238] EDAC MC: ECC not enabled
[    1.399084] Xilinx Zynq CpuIdle Driver started
[    1.404255] sdhci: Secure Digital Host Controller Interface driver
[    1.410383] sdhci: Copyright(c) Pierre Ossman
[    1.414690] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.487286] mmc0: SDHCI controller on e0100000.sdhci [e0100000.sdhci] using DMA
[    1.497920] ledtrig-cpu: registered to indicate activity on CPUs
[    1.504153] usbcore: registered new interface driver usbhid
[    1.509698] usbhid: USB HID core driver
[    1.523898] ip_tables: (C) 2000-2006 Netfilter Core Team
[    1.529322] NET: Registered protocol family 10
[    1.534682] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    1.540032] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    1.546766] NET: Registered protocol family 17
[    1.551196] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    1.564107] can: controller area network core (rev 20120528 abi 9)
[    1.570337] NET: Registered protocol family 29
[    1.574720] can: raw protocol (rev 20120528)
[    1.578986] can: broadcast manager protocol (rev 20161123 t)
[    1.584618] can: netlink gateway (rev 20130117) max_hops=1
[    1.590243] sctp: Hash tables configured (bind 512/512)
[    1.595714] Key type dns_resolver registered
[    1.600265] Registering SWP/SWPB emulation handler
[    1.607034] Key type encrypted registered
[    1.614370] mmc0: new high speed SDHC card at address aaaa
[    1.617247] hctosys: unable to open rtc device (rtc0)
[    1.617251] of_cfs_init
[    1.617337] of_cfs_init: OK
[    1.630705] RAMDISK: gzip image found at block 0
[    1.641234] mmcblk0: mmc0:aaaa SS08G 7.40 GiB
[    1.647457]  mmcblk0: p1 p2
[    2.140459] EXT4-fs warning (device ram0): ext4_update_dynamic_rev:749: updating to rev 1 because of new feature flag, running e2fsck is recommended
[    2.154297] EXT4-fs (ram0): mounted filesystem without journal. Opts: (null)
[    2.161361] VFS: Mounted root (ext4 filesystem) on device 1:0.
Starting rcS...
++ Mounting filesystem
mount: mounting /dev/mmcblk0p1 on /mnt failed: No such file or directory
mount: mounting /dev/mmcblk0 on /mnt failed: No such file or directory
++ Setting up mdev
++ Starting telnet daemon
++ Starting http daemon
++ Starting ftp daemon
++ Starting ssh daemon
[    2.379431] random: sshd: uninitialized urandom read (32 bytes read)
rcS Complete
zynq>

It does indicate that it failed to mount /dev/mmcblk0p1 but it doesn't seem to do anything with it.

/etc/init.d/rcS

When the ramdisk does start it runs the script '/etc/init.d/rcS' this is the contents of that script:

zynq> cat etc/init.d/rcS
#!/bin/sh


echo "Starting rcS..."


echo "++ Mounting filesystem"
mkdir -p /dev/pts
mkdir -p /dev/i2c
mount -a


echo "++ Setting up mdev"


echo /sbin/mdev > /proc/sys/kernel/hotplug
mdev -s


# the device driver is not creating this node, so for now
# create the devcfg device node manually


if [ ! -e /dev/xdevcfg ]
then
 mknod /dev/xdevcfg c 259 0
fi


echo "++ Starting telnet daemon"
telnetd -l /bin/sh


echo "++ Starting http daemon"
httpd -h /var/www


echo "++ Starting ftp daemon"
tcpsvd 0:21 ftpd ftpd -w /&


echo "++ Starting ssh daemon"
chmod 600 /etc/ssh_host_*
/usr/sbin/sshd


if [ -f /mnt/init.sh ]
then
 echo "++ Running user script init.sh from SD Card"
      source /mnt/init.sh
fi


echo "rcS Complete"

I'm presuming that the 'init.sh' file is not needed because the default Snickerdoodle boot partition doesn't have it.

How to get to a 'normal' rootfs?

Does anyone have any idea on how to change from ramdisk to rootfs (I'm probably saying it wrong).

It took me a while to get to this point and if there are any mistakes let me know I would love it if all I needed to do to update the kernel and dts would be a simple scp script. I also plan on loading the FPGA bit image from the rootfs but first things first.

Thanks for any help,

Dave

[Markus]

hi David, the link to the blog entry is broken.

[David M]

whoops:

https://cospandesign.github.io/linux,snickerdoodle,kernel,driver,uboot/2017/12/27/snickerdoodle-up-and-running.html

[David M]

A little bit further.

I was reading a post on the Xilinx forums and, in fact, you do not need the initrd. As long as the kernel has the ext4 driver built in and you don't need to do something fancy (like mount an encrypted drive) you can boot directly into the rootfs.

I removed this line from the extlinux.conf file and it started to boot into the normal rootfs

      INITRD /boot/uramdisk.image.gz

... until it went into emergency mode and I needed to log in as root. I've had some issues copying the rootfs onto the rootfs partition.

I'm going to try and burn a new SD card and attempt to fix this.

[David M]

Awesome! It worked! the main problem was the loadable modules were not installed into the new root file system.

[Rob Barris]

This looks handy.  Is the blog post complete with the added steps needed ?

I am still living in baremetal land, anything that makes Linux setup / iteration easier for when I get to that level, I bookmark :)

[David M]

I had been debugging sysboot and fixing up the blog when I found that the '/boot' folder is not the boot folder within the EXT4 root file system at all but rather the BOOT partition mounted in the ROOTFS partition at that location... I tried to unmount it but wasn't able to. this means that area of file system is inaccessible! Oh man I just spent a lot of time learning about sysboot but it's practically useless for this application.

<sigh>

Well, if all I need to do to update the kernel is copy uImage and the DTB to that location then it fulfills the same purpose as sysboot.