Added RPi4-64 support

[🇨🇿 Jakub Malý]

Empty Message

[Stefano Babic]

From: Jakub Maly <mal...@proton.me>

Signed-off-by: Jakub Maly <mal...@proton.me>
---
.../rpi-uboot-scr/files/boot.cmd.in | 2 +-
.../rpi-uboot-scr/rpi-u-boot-scr.bbappend | 2 +-
.../files/raspberrypi4-64/fw_env.config | 30 +++++
.../base-files/raspberrypi4-64/fstab | 10 ++
.../raspberrypi4-64/emmcsetup.lua | 85 ++++++++++++++
.../raspberrypi4-64/sw-description | 66 +++++++++++
.../swupdate/raspberrypi4-64/09-swupdate-args | 26 +++++
.../swupdate/raspberrypi4-64/defconfig | 107 ++++++++++++++++++
.../swupdate/raspberrypi4-64/swupdate.cfg | 41 +++++++
9 files changed, 367 insertions(+), 2 deletions(-)
create mode 100644 recipes-bsp/libubootenv/files/raspberrypi4-64/fw_env.config
create mode 100644 recipes-core/base-files/base-files/raspberrypi4-64/fstab
create mode 100644 recipes-extended/images/update-image/raspberrypi4-64/emmcsetup.lua
create mode 100644 recipes-extended/images/update-image/raspberrypi4-64/sw-description
create mode 100644 recipes-support/swupdate/swupdate/raspberrypi4-64/09-swupdate-args
create mode 100644 recipes-support/swupdate/swupdate/raspberrypi4-64/defconfig
create mode 100644 recipes-support/swupdate/swupdate/raspberrypi4-64/swupdate.cfg

diff --git a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
index 04cbd7b..8855f43 100644
--- a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
+++ b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
@@ -2,5 +2,5 @@ saveenv
fdt addr ${fdt_addr} && fdt get value bootargs /chosen bootargs
if env exists rpipart;then echo Booting from mmcblk0p${rpipart};else setenv rpipart 2;echo rpipart not set, default to ${rpipart};fi
load mmc 0:${rpipart} ${kernel_addr_r} boot/@@KERNEL_IMAGETYPE@@
-setenv bootargs ${bootargs} root=/dev/mmcblk0p${rpipart}
+setenv bootargs "${bootargs} root=/dev/mmcblk0p${rpipart}"
@@KERNEL_BOOTCMD@@ ${kernel_addr_r} - ${fdt_addr}
diff --git a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend
index ff7b487..c8aca89 100644
--- a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend
+++ b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend
@@ -1,3 +1,3 @@
FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

-COMPATIBLE = "raspberrypi3"
+COMPATIBLE = "raspberrypi3 raspberrypi4-64"
diff --git a/recipes-bsp/libubootenv/files/raspberrypi4-64/fw_env.config b/recipes-bsp/libubootenv/files/raspberrypi4-64/fw_env.config
new file mode 100644
index 0000000..f6eac0e
--- /dev/null
+++ b/recipes-bsp/libubootenv/files/raspberrypi4-64/fw_env.config
@@ -0,0 +1,30 @@
+# Configuration file for fw_(printenv/setenv) utility.
+# Up to two entries are valid, in this case the redundant
+# environment sector is assumed present.
+# Notice, that the "Number of sectors" is not required on NOR and SPI-dataflash.
+# Futhermore, if the Flash sector size is omitted, this value is assumed to
+# be the same as the Environment size, which is valid for NOR and SPI-dataflash
+# Device offset must be prefixed with 0x to be parsed as a hexadecimal value.
+
+# NOR example
+# MTD device name Device offset Env. size Flash sector size Number of sectors
+#/dev/mtd1 0x0000 0x4000 0x4000
+#/dev/mtd2 0x0000 0x4000 0x4000
+
+# MTD SPI-dataflash example
+# MTD device name Device offset Env. size Flash sector size Number of sectors
+#/dev/mtd5 0x4200 0x4200
+#/dev/mtd6 0x4200 0x4200
+
+# NAND example
+#/dev/mtd0 0x4000 0x4000 0x20000 2
+
+# On a block device a negative offset is treated as a backwards offset from the
+# end of the device/partition, rather than a forwards offset from the start.
+
+# Block device example
+#/dev/mmcblk0 0xc0000 0x20000
+#/dev/mmcblk0 -0x20000 0x20000
+
+# VFAT example
+/boot/uboot.env 0x0000 0x4000
diff --git a/recipes-core/base-files/base-files/raspberrypi4-64/fstab b/recipes-core/base-files/base-files/raspberrypi4-64/fstab
new file mode 100644
index 0000000..4f23935
--- /dev/null
+++ b/recipes-core/base-files/base-files/raspberrypi4-64/fstab
@@ -0,0 +1,10 @@
+# stock fstab - you probably want to override this with a machine specific one
+
+/dev/root / auto defaults 1 1
+proc /proc proc defaults 0 0
+devpts /dev/pts devpts mode=0620,gid=5 0 0
+tmpfs /run tmpfs mode=0755,nodev,nosuid,strictatime 0 0
+tmpfs /var/volatile tmpfs defaults 0 0
+
+/dev/mmcblk0p1 /boot auto defaults,sync 0 0
+/dev/mmcblk0p4 /media auto defaults,sync 0 0
diff --git a/recipes-extended/images/update-image/raspberrypi4-64/emmcsetup.lua b/recipes-extended/images/update-image/raspberrypi4-64/emmcsetup.lua
new file mode 100644
index 0000000..c7ef3db
--- /dev/null
+++ b/recipes-extended/images/update-image/raspberrypi4-64/emmcsetup.lua
@@ -0,0 +1,85 @@
+function os.capture(cmd)
+ local f = assert(io.popen(cmd, 'r'))
+ local s = assert(f:read('*a'))
+ f:close()
+ return s
+end
+
+function file_exists(name)
+ local f=io.open(name,"r")
+ if f~=nil then io.close(f) return true else return false end
+end
+
+function cmdexec(cmd)
+ local ret, s, status = os.execute(cmd)
+ if (status ~= 0) then
+ return false, cmd .. " return with error"
+ end
+
+ return true,""
+end
+
+function preinst()
+ local out
+ local s1
+ local ret
+
+ local log = os.tmpname()
+
+ local eMMC = "/dev/mmcblk0"
+ ret = file_exists("/dev/mmcblk0")
+
+ if (ret == false) then
+ return false, "Cannot fine eMMC"
+ end
+
+ cmdexec("/usr/sbin/sfdisk -d " .. eMMC .. "> /tmp/dumppartitions")
+
+ -- check if there are two identical partitions
+ -- and create the second one if no available
+ f = io.input("/tmp/dumppartitions")
+ fo = io.output("/tmp/partitions")
+ t = f:read()
+ found = false
+ while (t ~= nil) do
+ j=0
+ j=string.find(t, "/dev/mmcblk0p3")
+ fo:write(t .. "\n")
+ if (j == 1) then
+ found=true
+ break
+ end
+ j=string.find(t, "/dev/mmcblk0p2")
+ if (j == 1) then
+ start, size = string.match(t, "%a+%s*=%s*(%d+), size=%s*(%d+)")
+ end
+ t = f:read()
+ end
+
+ if (found) then
+ f:close()
+ fo:close()
+ return true, out
+ end
+
+ start=start+size
+ partitions = eMMC .. "p3 : start= " .. string.format("%d", start) .. ", size= " .. size .. ", type=83\n"
+
+ fo:write(partitions)
+ fo:close()
+ f:close()
+
+ out = os.capture("/usr/sbin/sfdisk --force " .. eMMC .. " < /tmp/partitions")
+
+ -- use partprobe to inform the kernel of the new partitions
+
+ cmdexec("/usr/sbin/partprobe " .. eMMC)
+
+ return true, out
+end
+
+function postinst()
+ local out = "Post installed script called"
+
+ return true, out
+end
diff --git a/recipes-extended/images/update-image/raspberrypi4-64/sw-description b/recipes-extended/images/update-image/raspberrypi4-64/sw-description
new file mode 100644
index 0000000..8a07161
--- /dev/null
+++ b/recipes-extended/images/update-image/raspberrypi4-64/sw-description
@@ -0,0 +1,66 @@
+software =
+{
+ version = "0.1.0";
+
+ raspberrypi4-64 = {
+ hardware-compatibility: [ "1.0"];
+ stable : {
+ copy1 : {
+ images: (
+ {
+ filename = "core-image-full-cmdline-raspberrypi4-64.ext4.gz";
+ type = "raw";
+ compressed = "zlib";
+ device = "/dev/mmcblk0p2";
+ sha256 = "$swupdate_get_sha256(core-image-full-cmdline-raspberrypi4-64.ext4.gz)";
+ }
+ );
+ scripts: (
+ {
+ filename = "emmcsetup.lua";
+ type = "lua";
+ sha256 = "$swupdate_get_sha256(emmcsetup.lua)";
+ }
+ );
+ bootenv: (
+ {
+ name = "rpipart";
+ value = "2";
+ },
+ {
+ name = "ustate";
+ value = "1";
+ }
+ );
+ };
+ copy2 : {
+ images: (
+ {
+ filename = "core-image-full-cmdline-raspberrypi4-64.ext4.gz";
+ type = "raw";
+ compressed = "zlib";
+ device = "/dev/mmcblk0p3";
+ sha256 = "$swupdate_get_sha256(core-image-full-cmdline-raspberrypi4-64.ext4.gz)";
+ }
+ );
+ scripts: (
+ {
+ filename = "emmcsetup.lua";
+ type = "lua";
+ sha256 = "$swupdate_get_sha256(emmcsetup.lua)";
+ }
+ );
+ bootenv: (
+ {
+ name = "rpipart";
+ value = "3";
+ },
+ {
+ name = "ustate";
+ value = "1";
+ }
+ );
+ };
+ };
+ }
+}
diff --git a/recipes-support/swupdate/swupdate/raspberrypi4-64/09-swupdate-args b/recipes-support/swupdate/swupdate/raspberrypi4-64/09-swupdate-args
new file mode 100644
index 0000000..283e667
--- /dev/null
+++ b/recipes-support/swupdate/swupdate/raspberrypi4-64/09-swupdate-args
@@ -0,0 +1,26 @@
+rootfs=`swupdate -g`
+
+if [ $rootfs == '/dev/mmcblk0p2' ];then
+ selection="-e stable,copy2"
+else
+ selection="-e stable,copy1"
+fi
+
+state=`fw_printenv ustate | cut -f 2 -d'='`
+if [ -z "$state" ];then
+ fw_setenv ustate 1
+ state=`fw_printenv ustate | cut -f 2 -d'='`
+fi
+if [ $state == 1 ];then
+ SWUPDATE_SURICATTA_ARGS="-c 2"
+else
+ SWUPDATE_SURICATTA_ARGS=" "
+fi
+
+if [ -e /media/etc/swupdate.cfg ];then
+ CFGFILE="/media/etc/swupdate.cfg"
+else
+ CFGFILE="/etc/swupdate.cfg"
+fi
+
+SWUPDATE_ARGS="-H raspberrypi4-64:1.0 ${selection} -p 'reboot' -f ${CFGFILE}"
diff --git a/recipes-support/swupdate/swupdate/raspberrypi4-64/defconfig b/recipes-support/swupdate/swupdate/raspberrypi4-64/defconfig
new file mode 100644
index 0000000..03cfd7e
--- /dev/null
+++ b/recipes-support/swupdate/swupdate/raspberrypi4-64/defconfig
@@ -0,0 +1,107 @@
+#
+# Automatically generated file; DO NOT EDIT.
+# Swupdate Configuration
+#
+CONFIG_HAVE_DOT_CONFIG=y
+
+#
+# Swupdate Settings
+#
+
+#
+# General Configuration
+#
+CONFIG_CURL=y
+# CONFIG_CURL_SSL is not set
+CONFIG_SYSTEMD=y
+CONFIG_SCRIPTS=y
+CONFIG_HW_COMPATIBILITY=y
+CONFIG_HW_COMPATIBILITY_FILE="/etc/hwrevision"
+CONFIG_SW_VERSIONS_FILE="/etc/sw-versions"
+
+#
+# Socket Paths
+#
+CONFIG_SOCKET_CTRL_PATH=""
+CONFIG_SOCKET_PROGRESS_PATH=""
+CONFIG_SOCKET_REMOTE_HANDLER_DIRECTORY="/tmp/"
+# CONFIG_MTD is not set
+CONFIG_LUA=y
+CONFIG_LUAPKG="lua"
+# CONFIG_FEATURE_SYSLOG is not set
+
+#
+# Build Options
+#
+CONFIG_CROSS_COMPILE=""
+CONFIG_SYSROOT=""
+CONFIG_EXTRA_CFLAGS=""
+CONFIG_EXTRA_LDFLAGS=""
+CONFIG_EXTRA_LDLIBS=""
+
+#
+# Debugging Options
+#
+# CONFIG_DEBUG is not set
+# CONFIG_WERROR is not set
+# CONFIG_NOCLEANUP is not set
+# CONFIG_BOOTLOADER_EBG is not set
+CONFIG_UBOOT=y
+# CONFIG_BOOTLOADER_NONE is not set
+# CONFIG_BOOTLOADER_GRUB is not set
+CONFIG_UBOOT_FWENV="/etc/fw_env.config"
+CONFIG_UBOOT_NEWAPI=y
+CONFIG_UBOOT_DEFAULTENV="/etc/u-boot-initial-env"
+# CONFIG_SSL_IMPL_NONE is not set
+CONFIG_SSL_IMPL_OPENSSL=y
+# CONFIG_SSL_IMPL_MBEDTLS is not set
+CONFIG_DOWNLOAD=y
+# CONFIG_DOWNLOAD_SSL is not set
+CONFIG_CHANNEL_CURL=y
+CONFIG_HASH_VERIFY=y
+CONFIG_SIGNED_IMAGES=y
+CONFIG_ENCRYPTED_IMAGES=y
+CONFIG_SURICATTA=y
+
+#
+# Features
+#
+CONFIG_SURICATTA_SSL=y
+# CONFIG_SURICATTA_STATE_CHOICE_NONE is not set
+CONFIG_SURICATTA_STATE_CHOICE_BOOTLOADER=y
+CONFIG_SURICATTA_STATE_BOOTLOADER="ustate"
+
+#
+# Server
+#
+CONFIG_SURICATTA_HAWKBIT=y
+# CONFIG_SURICATTA_GENERAL is not set
+CONFIG_WEBSERVER=y
+CONFIG_MONGOOSE=y
+# CONFIG_MONGOOSEIPV6 is not set
+CONFIG_MONGOOSESSL=y
+CONFIG_GUNZIP=y
+# CONFIG_ZSTD is not set
+
+#
+# Parser Features
+#
+CONFIG_LIBCONFIG=y
+CONFIG_PARSERROOT=""
+CONFIG_JSON=y
+# CONFIG_LUAEXTERNAL is not set
+# CONFIG_SETSWDESCRIPTION is not set
+
+#
+# Image Handlers
+#
+CONFIG_RAW=y
+# CONFIG_RDIFFHANDLER is not set
+CONFIG_LUASCRIPTHANDLER=y
+CONFIG_SHELLSCRIPTHANDLER=y
+# CONFIG_HANDLER_IN_LUA is not set
+CONFIG_ARCHIVE=y
+CONFIG_REMOTE_HANDLER=y
+# CONFIG_SWUFORWARDER_HANDLER is not set
+CONFIG_BOOTLOADERHANDLER=y
+# CONFIG_UCFWHANDLER is not set
diff --git a/recipes-support/swupdate/swupdate/raspberrypi4-64/swupdate.cfg b/recipes-support/swupdate/swupdate/raspberrypi4-64/swupdate.cfg
new file mode 100644
index 0000000..329d086
--- /dev/null
+++ b/recipes-support/swupdate/swupdate/raspberrypi4-64/swupdate.cfg
@@ -0,0 +1,41 @@
+globals :
+{
+ verbose = true;
+ loglevel = 5;
+ syslog = true;
+ public-key-file = "@PUBKEY@";
+};
+
+download :
+{
+ retries = 3;
+ timeout = 1800;
+};
+
+identify : (
+ { name = "raspberryPI4-64"; value = "1.0"; }
+);
+
+suricatta :
+{
+ enable = true;
+ tenant = "default";
+ id = "@TARGETID@";
+ url = "@PROTOCOL@://@HOSTNAME@:@PORT@";
+ polldelay = 60;
+ nocheckcert = true;
+ retry = 4;
+ retrywait = 200;
+ loglevel = 10;
+ userid = 0;
+ groupid = 0;
+ targettoken = "@TARGETTOKEN@";
+};
+
+webserver :
+{
+ document_root = "/www";
+ userid = 0;
+ groupid = 0;
+ timeout = 20;
+};
--
2.40.1

[Stefano Babic]

Hi Jakub,

This is ok, but there was no issue in the past - is it required ?

These and other files are simply copied from raspberrypi3. However, it
is possible just to overrride raspberrypi instead of copying the
configurations.

If files are put into libubootenv/files/raspberrypi, they should be
taken by any raspberry[3|4].

> diff --git a/recipes-core/base-files/base-files/raspberrypi4-64/fstab b/recipes-core/base-files/base-files/raspberrypi4-64/fstab
> new file mode 100644
> index 0000000..4f23935
> --- /dev/null
> +++ b/recipes-core/base-files/base-files/raspberrypi4-64/fstab
> @@ -0,0 +1,10 @@
> +# stock fstab - you probably want to override this with a machine specific one
> +
> +/dev/root / auto defaults 1 1
> +proc /proc proc defaults 0 0
> +devpts /dev/pts devpts mode=0620,gid=5 0 0
> +tmpfs /run tmpfs mode=0755,nodev,nosuid,strictatime 0 0
> +tmpfs /var/volatile tmpfs defaults 0 0
> +
> +/dev/mmcblk0p1 /boot auto defaults,sync 0 0
> +/dev/mmcblk0p4 /media auto defaults,sync 0 0

Ditto.

This is coming because in the past SWUpdate was not able to partition
the disk itself, and it needed external tool. However, by refreshing
this layer, it should be switched to the diskpart handler.

It is also identical to the one for raspi3, so please factorize.

Ditto.

Ditto.

Best regards,
Stefano Babic

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, 82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[🇨🇿 Jakub Malý]

Hello Stefano,

Yes, basically all files are just copies from rpi3 folders with minor adjustments. I didn't find any general folders called `raspberrypi` at https://github.com/sbabic/meta-swupdate-boards which would be taken by any raspberry[3|4] and therefore I created few new folders and copied all necessary files. The new `emmcsetup.lua` file using `diskpart` is also nowhere to be found. It seems like the GitHub repository is several commits behind any repository that you are referring to.

I should probably get the newest code somehow and try to run it on my 64bit rpi, to confirm that this patch is redundant. However, the `setenv bootargs "${bootargs} root=/dev/mmcblk0p${rpipart}"` is truly needed. Without it, it was never set properly. Might be the new version of U-Boot. Others also encountered this problem but sadly as me, they just created a pull request on GitHub https://github.com/sbabic/meta-swupdate-boards/pull/3 and then closed it without sending any patch to the mailing list.



Best wishes,

Jakub Maly

Sent with Proton Mail secure email.

------- Original Message -------

> > + local s = assert(f:read('a'))

> > + start, size = string.match(t, "%a+%s=%s*(%d+), size=%s*(%d+)")

[Stefano Babic]

Hi Jakub,

On 19.05.23 10:44, 🇨🇿 Jakub Malý wrote:
> Hello Stefano,
>
> Yes, basically all files are just copies from rpi3 folders with minor adjustments. I didn't find any general folders called `raspberrypi` at https://github.com/sbabic/meta-swupdate-boards which would be taken by any raspberry[3|4] and therefore I created few new folders and copied all necessary files.

Because at the time to create meta-swupdate-boards, only Raspberrypi 3
was supported. However, introducing a variant like Raspi 4, files should
be factorized and overriding per board should be done only when needed
without duplicating files.

> The new `emmcsetup.lua` file using `diskpart`

Ther eis no new emmcpart - this file should be dropped and replaced with
builtin handler in SWUpdate, adjusting the sw-description.

> is also nowhere to be found. It seems like the GitHub repository is several commits behind any repository that you are referring to.

meta-swupdate-boards was introduced because some users asking to have
reference with eval board. Goal was that this layer was then mostly
maintained by community, I do not use it - I work directly on my
projects' layers. But this mainly happened only sporadically, because
everyone (after a first check) is really interested to integrate
SWUpdate in own project, and not on a demo using Raspi &Co. So yes,
meta-swupdate-boards is behind current development.

>
> I should probably get the newest code somehow and try to run it on my 64bit rpi, to confirm that this patch is redundant. However, the `setenv bootargs "${bootargs} root=/dev/mmcblk0p${rpipart}"` is truly needed. Without it, it was never set properly. Might be the new version of U-Boot.

I cannot say - the quotes do no disturb, but it should work even
without. Anyway, quotes can be added, there is no drawback.

>Others also encountered this problem but sadly as me, they just created a pull request on GitHub https://github.com/sbabic/meta-swupdate-boards/pull/3 and then closed it without sending any patch to the mailing list.

Best regards,
Stefano Babic

[🇨🇿 Jakub Malý]

Hi Stefano,

> Because at the time to create meta-swupdate-boards, only Raspberrypi 3
> was supported. However, introducing a variant like Raspi 4, files should
> be factorized and overriding per board should be done only when needed
> without duplicating files.

I completely understand your idea but regarding Yocto (and mainly the OE workflow) this isn't possible as easy as it sounds. From my knowledge, by default the OE fetcher looks for folders inside the files directory and compares them to specified MACHINE name. This is full-world, case-sensitive string match evaluation as I noted many times. Therefore, having just one common folder called raspberrypi would require some tweaking through for example a python script, to match it to raspberrypi3, raspberrypi4, or raspberrypi4-64:

# OpenEmbedded fetch helper.
# Since the SWUpdate files does not change for different versions of the RPI
# it is more convenient to tell the fetcher that these files are in one
# common directory than to create several directories with the same content
# just to satisfy its search.
SRC_URI_APPEND = " \
file://09-swupdate-args \
file://swupdate.cfg \
"
python () {
machine = d.getVar("MACHINE", True)
if "raspberrypi" in machine:
d.setVar("SRC_URI_APPEND", "\
file://raspberrypi/09-swupdate-args \
file://raspberrypi/swupdate.cfg \
")
}
SRC_URI += "${SRC_URI_APPEND}"

This script would need to be in every single recipe / recipe append, so basically in four files regarding this layer (libubootenv_%.bbappend, base-files_%.bbappend, update-image.bb, swupdate_%.bbappend). From my perspective it would be much more nicer to just keep the structure OE uses and create a separate folder for each board. We can solve similar files issue by creating symlinks inside these folders.

Changing MACHINE to just raspberrypi is out of question, because it would then build wrong bsp in meta-raspberrypi layer.

> Ther eis no new emmcpart - this file should be dropped and replaced with
> builtin handler in SWUpdate, adjusting the sw-description.

Can you please navigate me to a valid description file with this handler so I can copy & paste it? Thanks

> meta-swupdate-boards was introduced because some users asking to have
> reference with eval board. Goal was that this layer was then mostly
> maintained by community, I do not use it - I work directly on my
> projects' layers. But this mainly happened only sporadically, because
> everyone (after a first check) is really interested to integrate
> SWUpdate in own project, and not on a demo using Raspi &Co. So yes,
> meta-swupdate-boards is behind current development.

I can help with maintaining if you wish. I bet there are and will be many users like me wanting to have 64bit yocto builds on their rpi4.

> I cannot say - the quotes do no disturb, but it should work even
> without. Anyway, quotes can be added, there is no drawback.

Thanks! Let's keep them then.

Best,

Jakub

[Stefano Babic]

Hi Jakub,

On 22.05.23 16:10, '🇨🇿 Jakub Malý' via swupdate wrote:
> Hi Stefano,
>
>> Because at the time to create meta-swupdate-boards, only Raspberrypi 3
>> was supported. However, introducing a variant like Raspi 4, files should
>> be factorized and overriding per board should be done only when needed
>> without duplicating files.
>
> I completely understand your idea but regarding Yocto (and mainly the OE workflow) this isn't possible as easy as it sounds. From my knowledge, by default the OE fetcher looks for folders inside the files directory and compares them to specified MACHINE name. This is full-world, case-sensitive string match evaluation as I noted many times.

Not enough: bitbake looks for files according to the overrides that are
set. For raspberry, you can see that MACHINEOVERRIDES i set to :

MACHINEOVERRIDES="rpi:armv7ve:<MACHINE NAME>", for example

MACHINEOVERRIDES="rpi:armv7ve:raspberrypi3"

Bitbake uses then the FILESOVERRIDES to extend the paths where files are
searched. Just look into the manual.

The search order privileges the MACHINE, that means if you put files
into a "rpit" directory, they are picked up if the same files are not in
a MACHINE directory.

> Therefore, having just one common folder called raspberrypi

s/raspberrypi/rpi/

Lokk for example in meta-raspbberry to
/recipes-bsp/formfactor/formfactor/rpi/.


>would require some tweaking

No

> through for example a python script, to match it to raspberrypi3, raspberrypi4, or raspberrypi4-64:

No

>
> # OpenEmbedded fetch helper.
> # Since the SWUpdate files does not change for different versions of the RPI
> # it is more convenient to tell the fetcher that these files are in one
> # common directory than to create several directories with the same content
> # just to satisfy its search.
> SRC_URI_APPEND = " \
> file://09-swupdate-args \
> file://swupdate.cfg \
> "
> python () {
> machine = d.getVar("MACHINE", True)
> if "raspberrypi" in machine:
> d.setVar("SRC_URI_APPEND", "\
> file://raspberrypi/09-swupdate-args \
> file://raspberrypi/swupdate.cfg \
> ")
> }
> SRC_URI += "${SRC_URI_APPEND}"
>
> This script would need to be in every single recipe / recipe append

It is wrong, it does not work as you expect.

> so basically in four files regarding this layer (libubootenv_%.bbappend, base-files_%.bbappend, update-image.bb, swupdate_%.bbappend). From my perspective it would be much more nicer to just keep the structure OE uses and create a separate folder for each board. We can solve similar files issue by creating symlinks inside these folders.
>
> Changing MACHINE to just raspberrypi

Of course, MACHIONE is MACHINE, this is wrong, we cnnot change the name
as we want.

> is out of question, because it would then build wrong bsp in meta-raspberrypi layer.
>
>
>> Ther eis no new emmcpart - this file should be dropped and replaced with
>> builtin handler in SWUpdate, adjusting the sw-description.
>
> Can you please navigate me to a valid description file with this handler so I can copy & paste it? Thanks
>

http://sbabic.github.io/swupdate/handlers.html#disk-partitioner

>> meta-swupdate-boards was introduced because some users asking to have
>> reference with eval board. Goal was that this layer was then mostly
>> maintained by community, I do not use it - I work directly on my
>> projects' layers. But this mainly happened only sporadically, because
>> everyone (after a first check) is really interested to integrate
>> SWUpdate in own project, and not on a demo using Raspi &Co. So yes,
>> meta-swupdate-boards is behind current development.
>
> I can help with maintaining if you wish. I bet there are and will be many users like me wanting to have 64bit yocto builds on their rpi4.
>
>
>> I cannot say - the quotes do no disturb, but it should work even
>> without. Anyway, quotes can be added, there is no drawback.
>
> Thanks! Let's keep them then.
>

Best regards,
Stefano

[🇨🇿 Jakub Malý]

Hello Stefano,

Thank you for clarifying how machine can be overridden. I'm not that yet experienced as I'd wish with the Yocto project. Please see my updated patch file bellow. Check the do_swuimage:prepend part mainly. This is how I'd solve the search for machine specific image name. But you're sure more experienced and will think about a better solution.

Regarding the partitions part inside the description file, I'll need more guidance. Right now the A and B partitions are being dynamically created via ts-raspberrypi.wks, depending on the size of the rootfs. Why is there need for swupdate to change these partitions? Ususally user will add IMAGE_ROOTFS_EXTRA_SPACE so there there will be more space in case some new binaries/files will come with the update. How does the backup system works, if you reformat the old partition? For example when someone updates form A to B and B proves to be faulty, how can they switch back to A, when the A was changed?


commit 7c07ff30934997832f1859310957f5cec19d679d
Author: Jakub Maly <mal...@proton.me>
Date: Wed May 24 12:32:50 2023 +0200

Switched to one common machine for rpi

diff --git a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
index 04cbd7b..8855f43 100644
--- a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
+++ b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/files/boot.cmd.in
@@ -2,5 +2,5 @@ saveenv
fdt addr ${fdt_addr} && fdt get value bootargs /chosen bootargs
if env exists rpipart;then echo Booting from mmcblk0p${rpipart};else setenv rpipart 2;echo rpipart not set, default to ${rpipart};fi
load mmc 0:${rpipart} ${kernel_addr_r} boot/@@KERNEL_IMAGETYPE@@
-setenv bootargs ${bootargs} root=/dev/mmcblk0p${rpipart}
+setenv bootargs "${bootargs} root=/dev/mmcblk0p${rpipart}"

@@KERNEL_BOOTCMD@@ ${kernel_addr_r} - ${fdt_addr}
diff --git a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend

index ff7b487..cd1615f 100644

--- a/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend
+++ b/dynamic-layers/raspberrypi/recipes-bsp/rpi-uboot-scr/rpi-u-boot-scr.bbappend
@@ -1,3 +1,3 @@
FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

-COMPATIBLE = "raspberrypi3"

+COMPATIBLE = "raspberrypi3 raspberrypi4 raspberrypi4-64"
diff --git a/recipes-bsp/libubootenv/files/raspberrypi3/fw_env.config b/recipes-bsp/libubootenv/files/rpi/fw_env.config
similarity index 100%
rename from recipes-bsp/libubootenv/files/raspberrypi3/fw_env.config
rename to recipes-bsp/libubootenv/files/rpi/fw_env.config
diff --git a/recipes-core/base-files/base-files/raspberrypi3/fstab b/recipes-core/base-files/base-files/rpi/fstab
similarity index 87%
rename from recipes-core/base-files/base-files/raspberrypi3/fstab
rename to recipes-core/base-files/base-files/rpi/fstab
index 4f23935..7988617 100644
--- a/recipes-core/base-files/base-files/raspberrypi3/fstab
+++ b/recipes-core/base-files/base-files/rpi/fstab
@@ -1,4 +1,4 @@
-# stock fstab - you probably want to override this with a machine specific one
+# default fstab for rpi boards

/dev/root / auto defaults 1 1

proc /proc proc defaults 0 0

diff --git a/recipes-extended/images/update-image.bb b/recipes-extended/images/update-image.bb
index 1636ba6..d4dd8bd 100644
--- a/recipes-extended/images/update-image.bb
+++ b/recipes-extended/images/update-image.bb
@@ -17,3 +17,9 @@ IMAGE_DEPENDS = "core-image-full-cmdline"
SWUPDATE_IMAGES = "core-image-full-cmdline"

SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz"
+
+do_swuimage:prepend () {
+ machine = d.getVar("MACHINE", True)
+ workdir = d.getVar('WORKDIR', True)
+ os.system(f'sed -i "s#@MACHINE@#{machine}#g" {workdir}/sw-description')
+}
diff --git a/recipes-extended/images/update-image/raspberrypi3/emmcsetup.lua b/recipes-extended/images/update-image/raspberrypi3/emmcsetup.lua
deleted file mode 100644
index 09d1fb7..0000000
--- a/recipes-extended/images/update-image/raspberrypi3/emmcsetup.lua
+++ /dev/null
@@ -1,85 +0,0 @@
-function os.capture(cmd)
- local f = assert(io.popen(cmd, 'r'))
- local s = assert(f:read('*a'))
- f:close()
- return s
-end
-
-function file_exists(name)
- local f=io.open(name,"r")
- if f~=nil then io.close(f) return true else return false end
-end
-
-function cmdexec(cmd)
- local ret, s, status = os.execute(cmd)
- if (status ~= 0) then
- return false, cmd .. " return with error"
- end
-
- return true,""
-end
-
-function preinst()
- local out
- local s1
- local ret
-
- local log = os.tmpname()
-
- local eMMC = "/dev/mmcblk0"
- ret = file_exists("/dev/mmcblk0")
-
- if (ret == false) then
- return false, "Cannot fine eMMC"
- end
-
- cmdexec("/usr/sbin/sfdisk -d " .. eMMC .. "> /tmp/dumppartitions")
-
- -- check if there are two identical partitions
- -- and create the second one if no available
- f = io.input("/tmp/dumppartitions")
- fo = io.output("/tmp/partitions")
- t = f:read()
- found = false
- while (t ~= nil) do
- j=0
- j=string.find(t, "/dev/mmcblk0p3")
- fo:write(t .. "\n")
- if (j == 1) then
- found=true
- break
- end
- j=string.find(t, "/dev/mmcblk0p2")
- if (j == 1) then
- start, size = string.match(t, "%a+%s*=%s*(%d+), size=%s*(%d+)")
- end
- t = f:read()
- end
-
- if (found) then
- f:close()
- fo:close()
- return true, out
- end
-
- start=start+size
- partitions = eMMC .. "p3 : start= " .. string.format("%d", start) .. ", size= " .. size .. ", type=83\n"
-
- fo:write(partitions)
- fo:close()
- f:close()
-
- out = os.capture("/usr/sbin/sfdisk --force " .. eMMC .. " < /tmp/partitions")
-
- -- use partprobe to inform the kernel of the new partitions
-
- cmdexec("/usr/sbin/partprobe " .. eMMC)
-
- return true, out
-end
-
-function postinst()
- local out = "Post installed script called"
-
- return true, out
-end
diff --git a/recipes-extended/images/update-image/raspberrypi3/sw-description b/recipes-extended/images/update-image/raspberrypi3/sw-description
deleted file mode 100644
index f279d1a..0000000
--- a/recipes-extended/images/update-image/raspberrypi3/sw-description
+++ /dev/null
@@ -1,55 +0,0 @@
-software =
-{
- version = "0.1.0";
-
- raspberrypi3 = {
- hardware-compatibility: [ "1.0"];
- stable : {
- copy1 : {
- images: (
- {
- filename = "core-image-full-cmdline-raspberrypi3.ext4.gz";
- type = "raw";
- compressed = "zlib";
- device = "/dev/mmcblk0p2";
- }
- );
- scripts: (
- {
- filename = "emmcsetup.lua";
- type = "lua";
- }
- );
- uboot: (
- {
- name = "rpipart";
- value = "2";
- }
- );
-
- };
- copy2 : {
- images: (
- {
- filename = "core-image-full-cmdline-raspberrypi3.ext4.gz";
- type = "raw";
- compressed = "zlib";
- device = "/dev/mmcblk0p3";
- }
- );
- scripts: (
- {
- filename = "emmcsetup.lua";
- type = "lua";
- }
- );
- uboot: (
- {
- name = "rpipart";
- value = "3";
- }
- );
- };
- };
- }
-}
diff --git a/recipes-extended/images/update-image/raspberrypi3/sw-description.embscript b/recipes-extended/images/update-image/raspberrypi3/sw-description.embscript
deleted file mode 100644
index 7cf781f..0000000
--- a/recipes-extended/images/update-image/raspberrypi3/sw-description.embscript
+++ /dev/null
@@ -1,145 +0,0 @@
-software =
-{
- version = "0.1.0";
-
- raspberrypi3 = {
-
- embedded-script ="
---[[
- SWUpdate Embedded Script Example
- Copyright (C) 2017, Stefano Babic
- Description:
- This is just an example how to integrate an embedded script in sw-description
- The script is first parsed, be sure to escape double quotes
- and use double quotes just when they are strictly required.
- The script simply reads from a GPIO, and returns false if this GPIO
- is set. That means:
- GPIO(x)=0 ==> SWUpdate installs the SWU
- GPIO(x)=1 ==> SWUpdate stops returning an error
-]]
--- Include swupdate library to use TRACE / ERROR
-require (\"swupdate\")
-
-INPUT = 1
-OUTPUT = 0
-
---[[
- Some functions are simply retrieved from LUA examples
- see book 'Programming in Lua' or some snipset from internet.
-]]
-
-function file_exists(filename)
- local file=io.open(filename,'r')
- if file~=nil then io.close(file) return true else return false end
-end
-
-function writeToFile (filename, data)
- local file=io.open(filename, 'w')
- file:write(data)
- file:close()
-end
-
-function readFromFile (filename)
- if file_exists(filename) then
- local file=io.open(filename, 'r')
- local data = file:read(1)
- file:close()
- return data
- else
- return \"\"
- end
-end
-
-function pinMode(pin, mode)
- local gpio_path = '/sys/class/gpio/'
- local gpio_direction = gpio_path..'gpio'..pin..'/direction'
- local gpio_export = gpio_path..'export'
-
- if not file_exists(gpio_direction) then
- writeToFile(gpio_export,pin)
- end
-
- if mode==INPUT then
- writeToFile(gpio_direction, 'in')
- else
- writeToFile(gpio_direction, 'out')
- end
-end
-
-function digitalRead(pin)
- value = readFromFile('/sys/class/gpio/gpio'..pin..'/value')
- return tonumber(value)
-end
-
-function allowInstall(image)
-
- -- Completely unuseful, but it is to show how to get values from image
-
- for k,l in pairs(image) do
- swupdate.trace(\"image[\" .. tostring(k) .. \"] = \" .. tostring(l))
- end
-
- -- Read GPIO21, that means pin 40 on the Header
- pinMode(21,INPUT)
- val=digitalRead(21)
- if (val == 0) then
- swupdate.trace(\"Updated allowed: go on !\")
- return true, image
- else
- swupdate.trace(\"Updated forbidden: STOP !\")
- return false, image
- end
-end ";
-
- hardware-compatibility: [ "1.0"];
- stable : {
- copy1 : {
- images: (
- {
- filename = "core-image-full-cmdline-raspberrypi3.ext4.gz";
- type = "raw";
- compressed = "zlib";
- device = "/dev/mmcblk0p2";
- hook = "allowInstall";
- }
- );
- scripts: (
- {
- filename = "emmcsetup.lua";
- type = "lua";
- }
- );
- uboot: (
- {
- name = "rpipart";
- value = "2";
- }
- );
-
- };
- copy2 : {
- images: (
- {
- filename = "core-image-full-cmdline-raspberrypi3.ext4.gz";
- type = "raw";
- compressed = "zlib";
- device = "/dev/mmcblk0p3";
- hook = "allowInstall";
- }
- );
- scripts: (
- {
- filename = "emmcsetup.lua";
- type = "lua";
- }
- );
- uboot: (
- {
- name = "rpipart";
- value = "3";
- }
- );
- };
- };
- }
-}
diff --git a/recipes-extended/images/update-image/rpi/emmcsetup.lua b/recipes-extended/images/update-image/rpi/emmcsetup.lua
new file mode 100644
index 0000000..e69de29
diff --git a/recipes-extended/images/update-image/rpi/sw-description b/recipes-extended/images/update-image/rpi/sw-description
new file mode 100644
index 0000000..c28ed3d
--- /dev/null
+++ b/recipes-extended/images/update-image/rpi/sw-description
@@ -0,0 +1,52 @@

+software =
+{
+ version = "0.1.0";
+

+ raspberrypi = {

+ hardware-compatibility: [ "1.0"];
+ stable : {
+ copy1 : {
+ images: (
+ {

+ filename = "core-image-full-cmdline-@MACHINE@.ext4.gz";

+ type = "raw";
+ compressed = "zlib";
+ device = "/dev/mmcblk0p2";

+ sha256 = "$swupdate_get_sha256(core-image-full-cmdline-@MACHINE@.ext4.gz)";

+ }
+ );
+ bootenv: (
+ {
+ name = "rpipart";
+ value = "2";
+ },
+ {
+ name = "ustate";
+ value = "1";
+ }
+ );
+ };
+ copy2 : {
+ images: (
+ {

+ filename = "core-image-full-cmdline-@MACHINE@.ext4.gz";

+ type = "raw";
+ compressed = "zlib";
+ device = "/dev/mmcblk0p3";

+ sha256 = "$swupdate_get_sha256(core-image-full-cmdline-@MACHINE@.ext4.gz)";

+ }
+ );
+ bootenv: (
+ {
+ name = "rpipart";
+ value = "3";
+ },
+ {
+ name = "ustate";
+ value = "1";
+ }
+ );
+ };
+ };
+ }
+}

diff --git a/recipes-support/swupdate/swupdate/raspberrypi3/swupdate.cfg b/recipes-support/swupdate/swupdate/raspberrypi3/swupdate.cfg
deleted file mode 100644
index 5671fe3..0000000
--- a/recipes-support/swupdate/swupdate/raspberrypi3/swupdate.cfg
+++ /dev/null
@@ -1,47 +0,0 @@
-globals :
-{
-
- verbose = true;
- loglevel = 5;
- syslog = true;
- /* public-key-file = "test.pem";*/
-};
-
-download :
-{
- retries = 3;
- timeout = 1800;
-};
-
-identify : (
- { name = "raspberryPI3"; value = "1.0"; }
-);
-
-suricatta :
-{
-
- tenant = "default";
- id = "raspberrypi3";
- confirm = 0;
- url = "http://paperina2:8280";
- polldelay = 60;
- nocheckcert = true;
- retry = 4;
- retrywait = 200;
- loglevel = 10;
- userid = 0;
- groupid = 0;
- max_artifacts = 1;
-/*
- cafile = "/etc/ssl/cafile";
- sslkey = "/etc/ssl/sslkey";
- sslcert = "/etc/ssl/sslcert";
-*/
-};
-
-webserver :
-{
- document_root = "/www";
- userid = 0;
- groupid = 0;
-};
diff --git a/recipes-support/swupdate/swupdate/raspberrypi3/09-swupdate-args b/recipes-support/swupdate/swupdate/rpi/09-swupdate-args
similarity index 69%
rename from recipes-support/swupdate/swupdate/raspberrypi3/09-swupdate-args
rename to recipes-support/swupdate/swupdate/rpi/09-swupdate-args
index 243aded..2f9c2ef 100644
--- a/recipes-support/swupdate/swupdate/raspberrypi3/09-swupdate-args
+++ b/recipes-support/swupdate/swupdate/rpi/09-swupdate-args
@@ -7,6 +7,10 @@ else
fi

state=`fw_printenv ustate | cut -f 2 -d'='`
+if [ -z "$state" ];then
+ fw_setenv ustate 1
+ state=`fw_printenv ustate | cut -f 2 -d'='`
+fi

if [ $state == 1 ];then
SWUPDATE_SURICATTA_ARGS="-c 2"
else
@@ -19,4 +23,4 @@ else
CFGFILE="/etc/swupdate.cfg"
fi

-SWUPDATE_ARGS="-H raspberrypi3:1.0 ${selection} -f ${CFGFILE}"
+SWUPDATE_ARGS="-H raspberrypi:1.0 ${selection} -p 'reboot' -f ${CFGFILE}"
diff --git a/recipes-support/swupdate/swupdate/raspberrypi3/defconfig b/recipes-support/swupdate/swupdate/rpi/defconfig
similarity index 98%
rename from recipes-support/swupdate/swupdate/raspberrypi3/defconfig
rename to recipes-support/swupdate/swupdate/rpi/defconfig
index fd8ce63..03cfd7e 100644
--- a/recipes-support/swupdate/swupdate/raspberrypi3/defconfig
+++ b/recipes-support/swupdate/swupdate/rpi/defconfig
@@ -59,7 +59,7 @@ CONFIG_DOWNLOAD=y
# CONFIG_DOWNLOAD_SSL is not set
CONFIG_CHANNEL_CURL=y
CONFIG_HASH_VERIFY=y
-# CONFIG_SIGNED_IMAGES is not set
+CONFIG_SIGNED_IMAGES=y
CONFIG_ENCRYPTED_IMAGES=y
CONFIG_SURICATTA=y

diff --git a/recipes-support/swupdate/swupdate/rpi/swupdate.cfg b/recipes-support/swupdate/swupdate/rpi/swupdate.cfg
new file mode 100644
index 0000000..4b8a582
--- /dev/null
+++ b/recipes-support/swupdate/swupdate/rpi/swupdate.cfg

@@ -0,0 +1,41 @@
+globals :
+{
+ verbose = true;
+ loglevel = 5;
+ syslog = true;
+ public-key-file = "@PUBKEY@";
+};
+
+download :
+{
+ retries = 3;
+ timeout = 1800;
+};
+
+identify : (

+ { name = "raspberryPI"; value = "1.0"; }

Best,

Jakub

Sent with Proton Mail secure email.

------- Original Message -------

[Stefano Babic]

Hi Jakub,

On 24.05.23 12:56, '🇨🇿 Jakub Malý' via swupdate wrote:
> Hello Stefano,
>
> Thank you for clarifying how machine can be overridden. I'm not that yet experienced as I'd wish with the Yocto project. Please see my updated patch file bellow.

I am glad I could teach this to you ;-)

> Check the do_swuimage:prepend part mainly. This is how I'd solve the search for machine specific image name. But you're sure more experienced and will think about a better solution.
>

It does not work in this way.

meta-swupdate supports out-of-the-box python variable substitiution. Any
variable known to bitbake can be inserted into sw-description (that
becomes a template) and the generated sw-descripütion will have the
right value. This prepend is not needed at all.

See:

http://sbabic.github.io/swupdate/building-with-yocto.html#bitbake-variable-expansion-in-sw-description

This is because this file is old and before variable substitution was
introduced.

Use this instead:

filename = "core-image-full-cmdline-@@MACHINE@@.ext4.gz";

The rest of the patch looks good to me - any chance to use git
send-email to post it ? Else patchworks cannot collect it, see :

https://patchwork.ozlabs.org/user/todo/swupdate/


Best regards,
Stefano Babic

[Stefano Babic]

This should also replaced with @@MACHINE@@, else the hardware
compatibility is not checked by SWUpdate. That means I could have a
Raspi 2 or even an older, and the SWU is for Raspi 4. SWUpdate cannot
detect this, and will install a software that cannot run.

This requires to set here:

@@MACHINE@@ = {

and adjust 09-swupdate-args to pass the correct machine to SWUpdate.

Best regards,
Stefano

[Jakub Maly]

Nice catch! I used gmail's smtp to send the patch via git send-email. Check the new conversation :)