[meta-swupdate][PATCH 1/4] swupdate_class: prepare to use SRC_URI in image recipes

[Anatolij Gustschin]

Enable swuimage task if building 'update-image' recipe of the
meta-swupdate-boards layer or if building usual image recipes
with appended SRC_URI and enabled 'swu' image fstype. Fetch
SRC_URI files if 'swu' image fstype was selected.

This is in preparation of support for 'swu' image type class
(for building .swu images without meta-swupdate-boards layer).

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---
classes/swupdate.bbclass | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
index 2c2430b..774de01 100644
--- a/classes/swupdate.bbclass
+++ b/classes/swupdate.bbclass
@@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
do_package_write_deb[noexec] = "1"
do_package_write_rpm[noexec] = "1"

+USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"
+
python () {
deps = " " + swupdate_getdepends(d)
d.appendVarFlag('do_swuimage', 'depends', deps)
+
+ # add swuimage task if building with swu image type or if
+ # building via 'update-image' recipe in meta-swupdate-boards
+ if d.getVar('USING_SWU') or d.getVar('PN') == 'update-image':
+ bb.build.addtask('do_swuimage', 'do_build', 'do_image_complete', d)
}

python do_swuimage () {
import shutil

+ if d.getVar('USING_SWU'):
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if len(src_uri) == 0:
+ bb.fatal("SRC_URI (sw-description) required with 'swu' in IMAGE_FSTYPES")
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.download()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+ try:
+ fetcher.unpack(d.getVar('WORKDIR'))
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+
workdir = d.getVar('WORKDIR', True)
images = (d.getVar('SWUPDATE_IMAGES', True) or "").split()
s = d.getVar('S', True)
@@ -168,5 +190,3 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"

INHIBIT_DEFAULT_DEPS = "1"
EXCLUDE_FROM_WORLD = "1"
-
-addtask do_swuimage after do_unpack do_prepare_recipe_sysroot before do_build
--
2.17.1

[Anatolij Gustschin]

When building with 'swu' image fstype enabled, there is an error:
...
core-image-full-cmdline.bb:do_build has circular dependency on \
core-image-full-cmdline.bb:do_swuimage

Depend on do_image_complete instead of do_build (do_build is an
empty task here).

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

classes/swupdate.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
index 774de01..70b855f 100644
--- a/classes/swupdate.bbclass
+++ b/classes/swupdate.bbclass
@@ -51,7 +51,7 @@ def swupdate_getdepends(d):

depstr = ""
for dep in deps:
- depstr += " " + dep + ":do_build"
+ depstr += " " + dep + ":do_image_complete"
return depstr

IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-swuimage"
--
2.17.1

[Anatolij Gustschin]

Building .swu images with meta-swupdate-boards does not work
when building them by devtool in installed eSDK:

$ devtool build-image update-image
...
ERROR: Specified recipe update-image is not an image recipe

This series adds swu image fstype to swupdate class, so that it is
possible to generate .swu images via normal image recipes extended
by appended options. With these patches applied, the swu image building
works in eSDK by "devtool build-image core-image-full-cmdline".
Patch 4 is an example how an image recipe can be extended to
enable .swu image generation (tested for 'wandboard' machine).

Anatolij Gustschin (4):
swupdate_class: prepare to use SRC_URI in image recipes
swupdate_class: fix do_swuimage circular dependency
swupdate_class: enable building .swu images via swu image fstype
core-image: add example for .swu image generation

classes/swupdate.bbclass | 44 ++++++++++++++++++-
.../images/core-image-full-cmdline.bbappend | 26 +++++++++++
.../wandboard/sw-description | 43 ++++++++++++++++++
3 files changed, 111 insertions(+), 2 deletions(-)
create mode 100644 recipes-extended/images/core-image-full-cmdline.bbappend
create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/sw-description

--
2.17.1

[Anatolij Gustschin]

Extend swupdate class to support swu image type. To use this in
an image recipe following changes should be appended in it:

- inherit swupdate class
- add SRC_URI with sw-description (and more required files)
- add IMAGE_FSTYPES, i.e. "ext4.gz swu" or "ext4.gz.enc swu"
- SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz"
- IMAGE_DEPENDS = "core-image-full-cmdline"
- SWUPDATE_IMAGES = "core-image-full-cmdline"

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

classes/swupdate.bbclass | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
index 70b855f..4112970 100644
--- a/classes/swupdate.bbclass
+++ b/classes/swupdate.bbclass
@@ -32,6 +32,8 @@
# image file with exactly the name as specified in SWUPDATE_IMAGES is searched for.

inherit swupdate-common.bbclass
+inherit swupdate-enc
+inherit image_types

S = "${WORKDIR}/${PN}"

@@ -190,3 +192,21 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"

INHIBIT_DEFAULT_DEPS = "1"
EXCLUDE_FROM_WORLD = "1"

+
+IMAGE_CMD_swu () {
+ # We cannot use existing python swupdate class code here,
+ # so this is a dummy command to make image_types class work.
+ # The .swu image generation is in separate do_swuimage task.
+}
+
+clean_swuimages () {
+ cd ${DEPLOY_DIR_IMAGE}
+ rm -rf ${IMAGE_LINK_NAME}*.swu
+}
+
+do_clean_swuimage[doc] = "Removes .swu output files from image deploy dir"
+do_clean_swuimage[nostamp] = "1"
+python do_clean_swuimage () {
+ bb.build.exec_func('clean_swuimages', d)
+}
+addtask clean_swuimage after do_rootfs before do_image
--
2.17.1

[Anatolij Gustschin]

Extend core-image-full-cmdline with options for building
.swu images.

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

.../images/core-image-full-cmdline.bbappend | 26 +++++++++++
.../wandboard/sw-description | 43 +++++++++++++++++++
2 files changed, 69 insertions(+)

create mode 100644 recipes-extended/images/core-image-full-cmdline.bbappend
create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/sw-description

diff --git a/recipes-extended/images/core-image-full-cmdline.bbappend b/recipes-extended/images/core-image-full-cmdline.bbappend
new file mode 100644
index 0000000..cbc59a6
--- /dev/null
+++ b/recipes-extended/images/core-image-full-cmdline.bbappend
@@ -0,0 +1,26 @@
+# Extensions for core-image-full-cmdline image recipe
+# to generate .swu image using swupdate class
+
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
+
+inherit swupdate
+
+SRC_URI = " \
+ file://sw-description \
+"
+
+IMAGE_FSTYPES = "ext4.gz swu"
+
+# remove comment when configuring to generate ext4.gz.enc or
+# ext4.gz.enc.swu images and add a key to the build directory
+# as "conf/enc.key"
+#SWUPDATE_AES_FILE = "${TOPDIR}/conf/enc.key"
+
+#SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz.enc"
+SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz"
+
+# images to build before building swupdate image
+IMAGE_DEPENDS = "core-image-full-cmdline"
+
+# images and files that will be included in the .swu image
+SWUPDATE_IMAGES = "core-image-full-cmdline"
diff --git a/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description b/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description
new file mode 100644
index 0000000..3a372d9
--- /dev/null
+++ b/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description
@@ -0,0 +1,43 @@
+software =
+{
+ version = "@@DISTRO_VERSION@@";
+
+ @@MACHINE@@ = {
+ hardware-compatibility: [ "revB", "revC", "revD"];
+ stable : {
+ copy1 : {
+ images: (
+ {
+ filename = "@@IMAGE_LINK_NAME@@.ext4.gz";
+ type = "raw";
+ compressed = true;
+ device = "/dev/mmcblk0p1";
+ }
+ );
+ uboot: (
+ {
+ name = "rootpart";
+ value = "1";
+ }
+ );
+
+ };
+ copy2 : {
+ images: (
+ {
+ filename = "@@IMAGE_LINK_NAME@@.ext4.gz";
+ type = "raw";
+ compressed = true;
+ device = "/dev/mmcblk0p2";
+ }
+ );
+ uboot: (
+ {
+ name = "rootpart";
+ value = "2";
+ }
+ );
+ };
+ };
+ }
+}
--
2.17.1

[Stefano Babic]

Hi Anatolji,

On 11.02.21 11:38, Anatolij Gustschin wrote:
> Building .swu images with meta-swupdate-boards does not work
> when building them by devtool in installed eSDK:
>
> $ devtool build-image update-image
> ...
> ERROR: Specified recipe update-image is not an image recipe

Well, in fact it is *NOT* an update recipe. It inherits swupdate, not
image. The error is consistent.

>
> This series adds swu image fstype to swupdate class, so that it is
> possible to generate .swu images via normal image recipes extended

Ok, so you want for the very simple case (just rootfs in SWU) to
generate quite automatically a SWU. But if we have more artifacts in
SWU, I do not see many advantages for this, I still need to add the
right DEPENDS and all required SWUPDATE_ flags. Is it just for lazy people ?

> by appended options. With these patches applied, the swu image building
> works in eSDK by "devtool build-image core-image-full-cmdline".> Patch 4 is an example how an image recipe can be extended to
> enable .swu image generation (tested for 'wandboard' machine).
>
> Anatolij Gustschin (4):
> swupdate_class: prepare to use SRC_URI in image recipes
> swupdate_class: fix do_swuimage circular dependency
> swupdate_class: enable building .swu images via swu image fstype
> core-image: add example for .swu image generation
>
> classes/swupdate.bbclass | 44 ++++++++++++++++++-
> .../images/core-image-full-cmdline.bbappend | 26 +++++++++++
> .../wandboard/sw-description | 43 ++++++++++++++++++
> 3 files changed, 111 insertions(+), 2 deletions(-)
> create mode 100644 recipes-extended/images/core-image-full-cmdline.bbappend
> create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/sw-description
>

Regards,
Stefano

--
=====================================================================
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sba...@denx.de
=====================================================================

[Stefano Babic]

Hi Anatolji,

On 11.02.21 11:38, Anatolij Gustschin wrote:

> Enable swuimage task if building 'update-image' recipe of the
> meta-swupdate-boards layer or if building usual image recipes
> with appended SRC_URI and enabled 'swu' image fstype. Fetch
> SRC_URI files if 'swu' image fstype was selected.
>
> This is in preparation of support for 'swu' image type class
> (for building .swu images without meta-swupdate-boards layer).
>
> Signed-off-by: Anatolij Gustschin <ag...@denx.de>
> ---
> classes/swupdate.bbclass | 24 ++++++++++++++++++++++--
> 1 file changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 2c2430b..774de01 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
> do_package_write_deb[noexec] = "1"
> do_package_write_rpm[noexec] = "1"
>
> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"

Name is quite misleading, SWUpdate always "uses" a SWU - another name ?
GEN_SWU, what else ?

> +
> python () {
> deps = " " + swupdate_getdepends(d)
> d.appendVarFlag('do_swuimage', 'depends', deps)
> +
> + # add swuimage task if building with swu image type or if
> + # building via 'update-image' recipe in meta-swupdate-boards
> + if d.getVar('USING_SWU') or d.getVar('PN') == 'update-image':

I gave the name "update-image", maybe a bad choice. But I do not like
that the name of a recipe is hard-coded here. Is it not enought to check
for IMAGE_FSTYPES, that is with USING_SWU ?

> + bb.build.addtask('do_swuimage', 'do_build', 'do_image_complete', d)

Why is it added at runtime and not statically as before ?

Do we always habe do_image_complete ? In most cases, I have not...

[Stefano Babic]

On 11.02.21 11:38, Anatolij Gustschin wrote:

But I do not always have do_image_complete. For example, if I add
"virtual/kernel", this should fail - does it work in any condition ?

> return depstr
>
> IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-swuimage"
>

[Anatolij Gustschin]

Hi Stefano,

On Thu, 11 Feb 2021 13:25:54 +0100
Stefano Babic sba...@denx.de wrote:

>Hi Anatolji,
>
>On 11.02.21 11:38, Anatolij Gustschin wrote:
>> Building .swu images with meta-swupdate-boards does not work
>> when building them by devtool in installed eSDK:
>>
>> $ devtool build-image update-image
>> ...
>> ERROR: Specified recipe update-image is not an image recipe
>
>Well, in fact it is *NOT* an update recipe. It inherits swupdate, not
>image. The error is consistent.

Yes, the error is expected here since this is not an image recipe.
devtool in eSDK has two separate commands for building images and
regular recipes:

devtool build-image core-image-full-cmdline
devtool build recipe-name

Therefore building an .swu image does not work in eSDK environment.
But a customer needs this and wants to build .swu via own image
recipes using eSDK workflow.

...

>> This series adds swu image fstype to swupdate class, so that it is
>> possible to generate .swu images via normal image recipes extended
>
>Ok, so you want for the very simple case (just rootfs in SWU) to
>generate quite automatically a SWU. But if we have more artifacts in
>SWU, I do not see many advantages for this, I still need to add the
>right DEPENDS and all required SWUPDATE_ flags. Is it just for lazy people ?

This is mainly for a customer who wants to build .swu images in eSDK
using different image recipes. This should be possible with mainline
meta-swupdate.

Thanks,
Anatolij

[Stefano Babic]

Hi Anatolji,

On 11.02.21 14:38, Anatolij Gustschin wrote:
> Hi Stefano,
>
> On Thu, 11 Feb 2021 13:25:54 +0100
> Stefano Babic sba...@denx.de wrote:
>
>> Hi Anatolji,
>>
>> On 11.02.21 11:38, Anatolij Gustschin wrote:
>>> Building .swu images with meta-swupdate-boards does not work
>>> when building them by devtool in installed eSDK:
>>>
>>> $ devtool build-image update-image
>>> ...
>>> ERROR: Specified recipe update-image is not an image recipe
>>
>> Well, in fact it is *NOT* an update recipe. It inherits swupdate, not
>> image. The error is consistent.
>
> Yes, the error is expected here since this is not an image recipe.
> devtool in eSDK has two separate commands for building images and
> regular recipes:

Ok

>
> devtool build-image core-image-full-cmdline

clear

> devtool build recipe-name

but even devtool build update-image fails ? I have supposed this works.

>
> Therefore building an .swu image does not work in eSDK environment.
> But a customer needs this and wants to build .swu via own image
> recipes using eSDK workflow.
>
> ...
>>> This series adds swu image fstype to swupdate class, so that it is
>>> possible to generate .swu images via normal image recipes extended
>>
>> Ok, so you want for the very simple case (just rootfs in SWU) to
>> generate quite automatically a SWU. But if we have more artifacts in
>> SWU, I do not see many advantages for this, I still need to add the
>> right DEPENDS and all required SWUPDATE_ flags. Is it just for lazy people ?
>
> This is mainly for a customer who wants to build .swu images in eSDK
> using different image recipes. This should be possible with mainline
> meta-swupdate.
>
> Thanks,
> Anatolij
>

[Anatolij Gustschin]

Hi Stefano,

On Thu, 11 Feb 2021 13:30:32 +0100
Stefano Babic sba...@denx.de wrote:

>Hi Anatolji,
>
>On 11.02.21 11:38, Anatolij Gustschin wrote:
>> Enable swuimage task if building 'update-image' recipe of the
>> meta-swupdate-boards layer or if building usual image recipes
>> with appended SRC_URI and enabled 'swu' image fstype. Fetch
>> SRC_URI files if 'swu' image fstype was selected.
>>
>> This is in preparation of support for 'swu' image type class
>> (for building .swu images without meta-swupdate-boards layer).
>>
>> Signed-off-by: Anatolij Gustschin <ag...@denx.de>
>> ---
>> classes/swupdate.bbclass | 24 ++++++++++++++++++++++--
>> 1 file changed, 22 insertions(+), 2 deletions(-)
>>
>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>> index 2c2430b..774de01 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
>> do_package_write_deb[noexec] = "1"
>> do_package_write_rpm[noexec] = "1"
>>
>> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"
>
>Name is quite misleading, SWUpdate always "uses" a SWU - another name ?
>GEN_SWU, what else ?

This variable indicates that building via swu FSTYPES was enabled,
so I though this name should be okay.

>> python () {
>> deps = " " + swupdate_getdepends(d)
>> d.appendVarFlag('do_swuimage', 'depends', deps)
>> +
>> + # add swuimage task if building with swu image type or if
>> + # building via 'update-image' recipe in meta-swupdate-boards
>> + if d.getVar('USING_SWU') or d.getVar('PN') == 'update-image':
>
>I gave the name "update-image", maybe a bad choice. But I do not like
>that the name of a recipe is hard-coded here. Is it not enought to check
>for IMAGE_FSTYPES, that is with USING_SWU ?

If both possibilities for generating .swu should be supported (old via
meta-swupdate-boards update-image and new via IMAGE_FSTYPES += "swu"),
then this is not enough, in case of adding the task at runtime.

>> + bb.build.addtask('do_swuimage', 'do_build', 'do_image_complete', d)
>
>Why is it added at runtime and not statically as before ?

I wanted to skip running this task if an image recipe inherited
swupdate class, but did not enable 'swu' fstype for some reason.
If the task must always be statically enabled, then I need to add
a check for 'swu' fstype at the beginning of the task and return
early if swu fstype is disabled. Therefore I preferred to add the
task at runtime.

>Do we always habe do_image_complete ? In most cases, I have not...

For image recipes, yes. For usual package recipes there is no
such task. Maybe we need to stay with do_build dependency in case
of building via update-image. I also tested building via update-image
and it worked. I'm not sure yet.

Thanks,
Anatolij

[Anatolij Gustschin]

Hi Stefano,

On Thu, 11 Feb 2021 15:07:00 +0100
Stefano Babic sba...@denx.de wrote:
...

>> devtool build-image core-image-full-cmdline
>
>clear
>
>> devtool build recipe-name
>
>but even devtool build update-image fails ? I have supposed this works.

Yes, it fails:

$ devtool build update-image
NOTE: Starting bitbake server...
ERROR: No recipe named 'update-image' in your workspace

When I prepare the recipe workspace by

$ devtool modify update-image

and try a build, I get:

$ devtool build update-image
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:00
Loaded 2539 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:00
Parsing of 1736 .bb files complete (1733 cached, 3 parsed). 2539 targets, 463 skipped, 0 masked, 0 errors.
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:01
Loaded 2538 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:00
Parsing of 1736 .bb files complete (1735 cached, 1 parsed). 2539 targets, 463 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
ERROR: Task do_populate_sysroot does not exist for target update-image (/esdk/fslc-framebuffer_sdk/layers/poky/meta-swupdate-boards/recipes-extended/images/update-image.bb:do_populate_sysroot). Close matches:
do_populate_lic
ERROR: Command execution failed: 1

And if I comment out "deltask do_populate_sysroot" line in meta-swupdate/classes/swupdate.bbclass,
the build command seems to complete, but there is no update-image*.swu in deploy directory:

$ devtool build update-image
NOTE: Starting bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)...
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:00
Loaded 2538 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:00
Parsing of 1736 .bb files complete (1733 cached, 3 parsed). 2539 targets, 463 skipped, 0 masked, 0 errors.
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:01
Loaded 2538 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:00
Parsing of 1736 .bb files complete (1735 cached, 1 parsed). 2539 targets, 463 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Found 0 Missed 0 Current 84 (0% match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 388 tasks of which 388 didn't need to be rerun and all succeeded.

$ ls tmp/deploy/images/wandboard/update-image*
ls: cannot access 'tmp/deploy/images/wandboard/update-image*': No such file or directory

Thanks,
Anatolij

[Stefan Herbrechtsmeier]

Hi Anatolij,

Am 11.02.21 um 11:38 schrieb Anatolij Gustschin:

> Building .swu images with meta-swupdate-boards does not work
> when building them by devtool in installed eSDK:
>
> $ devtool build-image update-image
> ...
> ERROR: Specified recipe update-image is not an image recipe
>
> This series adds swu image fstype to swupdate class, so that it is
> possible to generate .swu images via normal image recipes extended
> by appended options.

Is this a common approach? Is the content of the bbappend image or
machine specific? The WIC image use dependency and configuration
variables from the machine and thereby supports different images.

Regards
Stefan

[Stefan Herbrechtsmeier]

Hi Anatolij,

Am 11.02.21 um 11:38 schrieb Anatolij Gustschin:

> Enable swuimage task if building 'update-image' recipe of the
> meta-swupdate-boards layer or if building usual image recipes
> with appended SRC_URI and enabled 'swu' image fstype. Fetch
> SRC_URI files if 'swu' image fstype was selected.
>
> This is in preparation of support for 'swu' image type class
> (for building .swu images without meta-swupdate-boards layer).
>
> Signed-off-by: Anatolij Gustschin <ag...@denx.de>
> ---
> classes/swupdate.bbclass | 24 ++++++++++++++++++++++--
> 1 file changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 2c2430b..774de01 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
> do_package_write_deb[noexec] = "1"
> do_package_write_rpm[noexec] = "1"
>
> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"

Why do you use one file for different use cases instead of one file per
use case (swupdate.bbclass and image_types_swu.bbclass) and a extra file
with shared content (swupdate-common.bbclass)?

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 11.02.21 um 13:25 schrieb Stefano Babic:

> Hi Anatolji,
>
> On 11.02.21 11:38, Anatolij Gustschin wrote:
>> This series adds swu image fstype to swupdate class, so that it is
>> possible to generate .swu images via normal image recipes extended
>
> Ok, so you want for the very simple case (just rootfs in SWU) to
> generate quite automatically a SWU. But if we have more artifacts in
> SWU, I do not see many advantages for this, I still need to add the
> right DEPENDS and all required SWUPDATE_ flags. Is it just for lazy people ?

I think an image type for swupdate is reasonable to support different
images. The WIC image type supports different artifacts and could create
file systems on the fly. At the moment it is possible to create an SD
card with arbitrary deployed artifacts in different partitions with
various filesystems for any image. Doing the same to create an swu
update file is much more complicated.

Regards
Stefan

[Stefano Babic]

Hi Stefan,

Right.

> Doing the same to create an swu
> update file is much more complicated.

Yes, I am just bothering how this can be done in the right way. There
are sure a lot of projects where the SWU just contains the rootfs, and
then a .swu is quite straightforward, but there are a raising number of
projects (at least, this is my experience) where the SWU contains update
for separate entities on the same device (i.e. microcontroller
firmwares) or even to forward software to other devices. At the end, I
come with a very specialized and project specific sw-description that
fits that project, but it cannot be reused on other projects. I do not
know how a general .swu image type can be factorized as Anatolji is
trying to do.

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 14.02.21 um 10:34 schrieb Stefano Babic:

But is this really a problem for an swu image class? The main difference
is how you define you configuration. In most cases an image class use
global configurations from the machine configuration and in case of a
wic image a shared wks configuration file from a common directory. At
the moment swupdate use a special recipe on top of an other main image
recipe.

I think that the main problems are the scatter information in the
sw-description and the recipe. Other classes generate the configuration
file on the fly (ex. kernel-fitimage) or use a separate configuration
file (ex. WIC).

Regards
Stefan

[Stefano Babic]

Hi Stefan,

A global (MACHINE or DISTRO) configuration covers just the simple cases,
but there are many projects creating multiple SWUs. So a MACHINE
configuration does not work in many cases.

The difference with a WKS is that we need here a configuration for the
device (sw-description), that is also interpreted by the device, and a
configuration for the build system, that should not go on the device
(that means, the recipe), and the build system can require some part
from sw-description (or in any cases, it fills automatically some
parts). I cannot say that SWUpdate use a recipe on top of others, it is
a recipe that depends on one (simple rootfs) or many other recipes, and
in some cases even from images generated for other OE machines.

>
> I think that the main problems are the scatter information in the
> sw-description and the recipe.

Yes, but they contain information for different actors, the device
(sw-description) and the buildsystem (the recipe). Information in recipe
are useless for the device, but the buildsystem can use some information
from sw-description (and it extends it).

> Other classes generate the configuration
> file on the fly (ex. kernel-fitimage)

well, let me say that I am not so fond of it. It creates a fitImage for
most cases with a hard-coded and generated .its, that is
kernel+DT+Ramdisk, but there are some other cases where additional
images must be packed into a fitImage and the kernel class is not
configurable for such things (i.e. fpga bitstreams, etc.). Anyway, in
most cases a fitImage consists of kernel+DT, and then at least 90% of
cases are covered - I do not see the same in SWUpdate.

> or use a separate configuration
> file (ex. WIC).

Right, but wks is a configuration file for the buildsystem and the
device does not need to bother with it. We have a use case here more
similar to fitImage, but where there is quite no "standard" case like
fitImage to allow the generation of a sw-description.

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 14.02.21 um 18:15 schrieb Stefano Babic:

But a global configuration could be override in recipes or you could
share it between multiple recipes. It should work for the simple and the
complex solutions.

> The difference with a WKS is that we need here a configuration for the
> device (sw-description), that is also interpreted by the device, and a
> configuration for the build system, that should not go on the device
> (that means, the recipe), and the build system can require some part
> from sw-description (or in any cases, it fills automatically some
> parts). I cannot say that SWUpdate use a recipe on top of others, it is
> a recipe that depends on one (simple rootfs) or many other recipes, and
> in some cases even from images generated for other OE machines.

The WKS also need some information for the device but they are generated
/ extracted from the configuration and the WKS could also depends on
other recipes and there artifacts.

At the moment I need an image recipe and an extra vfat image type class
to create a boot partition. Furthermore I need a update recipe per each
image recipe.

>> I think that the main problems are the scatter information in the
>> sw-description and the recipe.
>
> Yes, but they contain information for different actors, the device
> (sw-description) and the buildsystem (the recipe). Information in recipe
> are useless for the device, but the buildsystem can use some information
> from sw-description (and it extends it).
>
>> Other classes generate the configuration file on the fly (ex.
>> kernel-fitimage)
>
> well, let me say that I am not so fond of it. It creates a fitImage for
> most cases with a hard-coded and generated .its, that is
> kernel+DT+Ramdisk, but there are some other cases where additional
> images must be packed into a fitImage and the kernel class is not
> configurable for such things (i.e. fpga bitstreams, etc.). Anyway, in
> most cases a fitImage consists of kernel+DT, and then at least 90% of
> cases are covered - I do not see the same in SWUpdate.

You are right a sw-description generator will become very complex.

>> or use a separate configuration file (ex. WIC).
>
> Right, but wks is a configuration file for the buildsystem and the
> device does not need to bother with it. We have a use case here more
> similar to fitImage, but where there is quite no "standard" case like
> fitImage to allow the generation of a sw-description.

The problem is not the missing standard case but the flexibility of our
configuration.

Maybe we could define a swu.in or sw-description.in configuration which
extends the sw-description with build information? Something like a
source entry for the image, file and script entries. We could reuse the
filename to rename the files and the compressed and encrypted entry to
prepare the files. Furthermore we could add a filesystem and name entry
to an image to create a file system on the fly from IMAGE_BOOT_FILES.

Regards
Stefan

[Anatolij Gustschin]

Hi Stefano,

On Thu, 11 Feb 2021 13:33:10 +0100
Stefano Babic sba...@denx.de wrote:

>On 11.02.21 11:38, Anatolij Gustschin wrote:
>> When building with 'swu' image fstype enabled, there is an error:
>> ...
>> core-image-full-cmdline.bb:do_build has circular dependency on \
>> core-image-full-cmdline.bb:do_swuimage
>>
>> Depend on do_image_complete instead of do_build (do_build is an
>> empty task here).
>>
>> Signed-off-by: Anatolij Gustschin <ag...@denx.de>
>> ---
>> classes/swupdate.bbclass | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>> index 774de01..70b855f 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -51,7 +51,7 @@ def swupdate_getdepends(d):
>>
>> depstr = ""
>> for dep in deps:
>> - depstr += " " + dep + ":do_build"
>> + depstr += " " + dep + ":do_image_complete"
>
>But I do not always have do_image_complete. For example, if I add
>"virtual/kernel", this should fail - does it work in any condition ?

you are right, as is it will fail when IMAGE_DEPENDS contains
non image recipes like "virtual/kernel". I'll change to add it
conditionally.

Thanks,
Anatolij

[Anatolij Gustschin]

Hi Stefan,

On Sat, 13 Feb 2021 16:44:03 +0100
Stefan Herbrechtsmeier ste...@herbrechtsmeier.net wrote:
...

>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>> index 2c2430b..774de01 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
>> do_package_write_deb[noexec] = "1"
>> do_package_write_rpm[noexec] = "1"
>>
>> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"
>
>Why do you use one file for different use cases instead of one file per
>use case (swupdate.bbclass and image_types_swu.bbclass) and a extra file
>with shared content (swupdate-common.bbclass)?

other layer (meta-swupdate-boards) uses this swupdate.bbclass, I have
to ensure that using this layer still works after applying this series.
But I think I can add separate image_types_swu.bbclass which inherits
swupdate.bbclass. Will fix in v2 series.

Thanks,
Anatolij

[Anatolij Gustschin]

Hi Stefan,

The content is image and machine specific. bbappend is just an
example here to demonstrate the content for .swu generation.
For custom images I expect that the swu specific options will be
added to the usual image files directly. I think it is better
to add bbappend in patch 4 to meta-swupdate-boards, so for only
meta-swupdate users the core-image-full-cmdline image will not
change. Will to this in v2 series.

Thanks,
Anatolij

[Stefano Babic]

Hi Anatolji,

On 15.02.21 23:13, Anatolij Gustschin wrote:
> Hi Stefan,
>
> On Sat, 13 Feb 2021 16:44:03 +0100
> Stefan Herbrechtsmeier ste...@herbrechtsmeier.net wrote:
> ...
>>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>>> index 2c2430b..774de01 100644
>>> --- a/classes/swupdate.bbclass
>>> +++ b/classes/swupdate.bbclass
>>> @@ -76,14 +76,36 @@ do_package_write_ipk[noexec] = "1"
>>> do_package_write_deb[noexec] = "1"
>>> do_package_write_rpm[noexec] = "1"
>>>
>>> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"
>>
>> Why do you use one file for different use cases instead of one file per
>> use case (swupdate.bbclass and image_types_swu.bbclass)

+1

This is the common way in OE and it is preferable instead of putting
everything in a single class.

>> and a extra file
>> with shared content (swupdate-common.bbclass)? >
> other layer (meta-swupdate-boards) uses this swupdate.bbclass, I have
> to ensure that using this layer still works after applying this series.
> But I think I can add separate image_types_swu.bbclass which inherits
> swupdate.bbclass. Will fix in v2 series.
>
> Thanks,
> Anatolij
>

[Stefano Babic]

Hi Anatolji,

On 15.02.21 23:49, Anatolij Gustschin wrote:
> Hi Stefan,
>
> On Sat, 13 Feb 2021 16:39:03 +0100
> Stefan Herbrechtsmeier ste...@herbrechtsmeier.net wrote:
>
>> Hi Anatolij,
>>
>> Am 11.02.21 um 11:38 schrieb Anatolij Gustschin:
>>> Building .swu images with meta-swupdate-boards does not work
>>> when building them by devtool in installed eSDK:
>>>
>>> $ devtool build-image update-image
>>> ...
>>> ERROR: Specified recipe update-image is not an image recipe
>>>
>>> This series adds swu image fstype to swupdate class, so that it is
>>> possible to generate .swu images via normal image recipes extended
>>> by appended options.
>>
>> Is this a common approach? Is the content of the bbappend image or
>> machine specific? The WIC image use dependency and configuration
>> variables from the machine and thereby supports different images.
>
> The content is image and machine specific. bbappend is just an
> example here to demonstrate the content for .swu generation.
> For custom images I expect that the swu specific options will be
> added to the usual image files directly.

Right, else this extension has no meaning.

> I think it is better
> to add bbappend in patch 4 to meta-swupdate-boards, so for only
> meta-swupdate users the core-image-full-cmdline image will not
> change. Will to this in v2 series.

But I am expecting that boards and more important customers who do not
use IMAGE_FSTYPES += "swu" wont be touched by this series, else we can
have compatibility problems.

[Stefano Babic]

Hi Stefan,

Then it is a .inc file that we can include in multiple recipes, but for
its nature, a SWU configuration is related to an image, and not to a
MACHINE / DISTRO.

> It should work for the simple and the
> complex solutions.

.inc file, yes, but SWU variables in a MACHINE config is quite
misleading, because this should mean it is machine specific, and it is
really more image specific.

>
>
>> The difference with a WKS is that we need here a configuration for the
>> device (sw-description), that is also interpreted by the device, and a
>> configuration for the build system, that should not go on the device
>> (that means, the recipe), and the build system can require some part
>> from sw-description (or in any cases, it fills automatically some
>> parts). I cannot say that SWUpdate use a recipe on top of others, it is
>> a recipe that depends on one (simple rootfs) or many other recipes,
>> and in some cases even from images generated for other OE machines.
>
> The WKS also need some information for the device but they are generated
> / extracted from the configuration and the WKS could also depends on
> other recipes and there artifacts.

But the device has nothing to do with it, while in case of SWU,
sw-description is the description of the release and must understand it.

>
> At the moment I need an image recipe and an extra vfat image type class
> to create a boot partition.

Ok, this is to generate the .wic for the first setup of the device. I
guess this cannot be avoided.

> Furthermore I need a update recipe per each
> image recipe.

What I do is to have a .inc file shared by all update recipes, so that
the update recipes are very simple. But yes, an update recipe per image
is necessary, even if quite empty.

>
>>> I think that the main problems are the scatter information in the
>>> sw-description and the recipe.
>>
>> Yes, but they contain information for different actors, the device
>> (sw-description) and the buildsystem (the recipe). Information in
>> recipe are useless for the device, but the buildsystem can use some
>> information from sw-description (and it extends it).
>>
>>> Other classes generate the configuration file on the fly (ex.
>>> kernel-fitimage)
>>
>> well, let me say that I am not so fond of it. It creates a fitImage
>> for most cases with a hard-coded and generated .its, that is
>> kernel+DT+Ramdisk, but there are some other cases where additional
>> images must be packed into a fitImage and the kernel class is not
>> configurable for such things (i.e. fpga bitstreams, etc.). Anyway, in
>> most cases a fitImage consists of kernel+DT, and then at least 90% of
>> cases are covered - I do not see the same in SWUpdate.
>
> You are right a sw-description generator will become very complex.
>

IMHO sw-description is code instead of simple configuration, and it is
written by developers / integrators, because requirements and use case
strongly differ. For example, a customer of mine wants to group some
artifacts (U-Boot, firmware, a blob) together and update all of them if
one of the artifacts belonging to the group needs to be updated due to
version mismatch. Some requirements can be satisfied using hooks in
sw-description and embedded Lua code. An automatic generation of
sw-description is then impossible but very simple cases (aka update of
just rootfs).

>>> or use a separate configuration file (ex. WIC).
>>
>> Right, but wks is a configuration file for the buildsystem and the
>> device does not need to bother with it. We have a use case here more
>> similar to fitImage, but where there is quite no "standard" case like
>> fitImage to allow the generation of a sw-description.
>
> The problem is not the missing standard case but the flexibility of our
> configuration.
>
> Maybe we could define a swu.in or sw-description.in configuration which
> extends the sw-description with build information?

I do not know how, information must be removed then because they should
not reach the device. And there is not just OE, but also Buildroot and
Debian.

I am also careful to change sw-description: it is ok to add new features
/ attributes, but they should not break update for devices already in
field if they do not use the new features.

> Something like a
> source entry for the image, file and script entries.

With just a hook, it does not work, I think. And handler must be
specified (what about to update a UBIFS instead of eMMC ?), at the end
it is a sw-description written in another format.

> We could reuse the
> filename to rename the files and the compressed and encrypted entry to
> prepare the files. Furthermore we could add a filesystem and name entry
> to an image to create a file system on the fly from IMAGE_BOOT_FILES.
>

[Anatolij Gustschin]

Building .swu images with meta-swupdate-boards does not work
when building them by devtool in installed eSDK:

$ devtool build-image update-image
...
ERROR: Specified recipe update-image is not an image recipe

This series adds swu image fstype to swupdate class, so that it is
possible to generate .swu images via normal image recipes extended

by appended options. With these patches applied, the swu image building
works in eSDK by "devtool build-image core-image-full-cmdline".

Patch 3 is an example how an image recipe can be extended to

enable .swu image generation (tested for 'wandboard' machine).

Changes v2:
- conditionally add do_image_complete dependency, drop patch 2/4
- add image_types_swu.bbclass
- prepare core-image-full-cmdline bbappend for meta-swupdate-boards

Anatolij Gustschin (3):

swupdate_class: prepare to use SRC_URI in image recipes

Enable building .swu images via swu image fstype

core-image: add example for .swu image generation

classes/image_types_swu.bbclass | 21 ++++++
classes/swupdate.bbclass | 31 +++++++-
.../images/core-image-full-cmdline.bbappend | 28 ++++++++
.../wandboard/emmcsetup.lua | 12 ++++
.../wandboard/sw-description | 71 +++++++++++++++++++
5 files changed, 160 insertions(+), 3 deletions(-)
create mode 100644 classes/image_types_swu.bbclass
create mode 100644 recipes-extended/images/core-image-full-cmdline.bbappend
create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/emmcsetup.lua

[Anatolij Gustschin]

Enable swuimage task if building 'update-image' recipe of the
meta-swupdate-boards layer or if building usual image recipes
with appended SRC_URI and enabled 'swu' image fstype. Fetch
SRC_URI files if 'swu' image fstype was selected.

This is in preparation of support for 'swu' image type class
(for building .swu images without meta-swupdate-boards layer).

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

classes/swupdate.bbclass | 31 ++++++++++++++++++++++++++++---
1 file changed, 28 insertions(+), 3 deletions(-)

diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
index 2c2430b..c2899be 100644
--- a/classes/swupdate.bbclass
+++ b/classes/swupdate.bbclass
@@ -51,7 +51,10 @@ def swupdate_getdepends(d):

depstr = ""
for dep in deps:
- depstr += " " + dep + ":do_build"

+ if dep == d.getVar('PN'):

+ depstr += " " + dep + ":do_image_complete"

+ else:
+ depstr += " " + dep + ":do_build"

return depstr

IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-swuimage"

@@ -76,14 +79,38 @@ do_package_write_ipk[noexec] = "1"

do_package_write_deb[noexec] = "1"
do_package_write_rpm[noexec] = "1"

+USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"

+

python () {
deps = " " + swupdate_getdepends(d)
d.appendVarFlag('do_swuimage', 'depends', deps)
+
+ # add swuimage task if building with swu image type or if
+ # building via 'update-image' recipe in meta-swupdate-boards

+ if d.getVar('PN') == 'update-image':
+ bb.build.addtask('do_swuimage', 'do_build', 'do_unpack do_prepare_recipe_sysroot', d)
+ elif d.getVar('USING_SWU'):

+ bb.build.addtask('do_swuimage', 'do_build', 'do_image_complete', d)
}

python do_swuimage () {
import shutil

+ if d.getVar('USING_SWU'):
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ if len(src_uri) == 0:
+ bb.fatal("SRC_URI (sw-description) required with 'swu' in IMAGE_FSTYPES")
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.download()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+ try:
+ fetcher.unpack(d.getVar('WORKDIR'))
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+
workdir = d.getVar('WORKDIR', True)
images = (d.getVar('SWUPDATE_IMAGES', True) or "").split()
s = d.getVar('S', True)

@@ -168,5 +195,3 @@ PACKAGE_ARCH = "${MACHINE_ARCH}"

INHIBIT_DEFAULT_DEPS = "1"
EXCLUDE_FROM_WORLD = "1"
-
-addtask do_swuimage after do_unpack do_prepare_recipe_sysroot before do_build

--
2.17.1

[Anatolij Gustschin]

Add image_types_swu class to support swu image type. To use this
in an image recipe following changes should be added in it:

- inherit image_types_swu class

- add SRC_URI with sw-description (and more required files)
- add IMAGE_FSTYPES, i.e. "ext4.gz swu" or "ext4.gz.enc swu"
- SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz"
- IMAGE_DEPENDS = "core-image-full-cmdline"
- SWUPDATE_IMAGES = "core-image-full-cmdline"

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

classes/image_types_swu.bbclass | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
create mode 100644 classes/image_types_swu.bbclass

diff --git a/classes/image_types_swu.bbclass b/classes/image_types_swu.bbclass
new file mode 100644
index 0000000..6b5c6b5
--- /dev/null
+++ b/classes/image_types_swu.bbclass
@@ -0,0 +1,21 @@
+inherit swupdate
+inherit swupdate-enc
+inherit image_types

[Anatolij Gustschin]

Extend core-image-full-cmdline with options for building
.swu images.

Signed-off-by: Anatolij Gustschin <ag...@denx.de>
---

.../images/core-image-full-cmdline.bbappend | 28 ++++++++
.../wandboard/emmcsetup.lua | 12 ++++
.../wandboard/sw-description | 71 +++++++++++++++++++

3 files changed, 111 insertions(+)

create mode 100644 recipes-extended/images/core-image-full-cmdline.bbappend
create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/emmcsetup.lua
create mode 100644 recipes-extended/images/core-image-full-cmdline/wandboard/sw-description

diff --git a/recipes-extended/images/core-image-full-cmdline.bbappend b/recipes-extended/images/core-image-full-cmdline.bbappend
new file mode 100644
index 0000000..dcca4ef
--- /dev/null
+++ b/recipes-extended/images/core-image-full-cmdline.bbappend
@@ -0,0 +1,28 @@

+# Extensions for core-image-full-cmdline image recipe
+# to generate .swu image using swupdate class
+
+FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
+

+inherit image_types_swu
+
+SRC_URI_wandboard = " \
+ file://emmcsetup.lua \

+ file://sw-description \
+"
+

+# Uncomment below line to enable .swu image generation
+#IMAGE_FSTYPES_wandboard += "ext4.gz swu"

+
+# remove comment when configuring to generate ext4.gz.enc or
+# ext4.gz.enc.swu images and add a key to the build directory
+# as "conf/enc.key"
+#SWUPDATE_AES_FILE = "${TOPDIR}/conf/enc.key"
+
+#SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz.enc"
+SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ext4.gz"
+
+# images to build before building swupdate image
+IMAGE_DEPENDS = "core-image-full-cmdline"
+
+# images and files that will be included in the .swu image
+SWUPDATE_IMAGES = "core-image-full-cmdline"

diff --git a/recipes-extended/images/core-image-full-cmdline/wandboard/emmcsetup.lua b/recipes-extended/images/core-image-full-cmdline/wandboard/emmcsetup.lua
new file mode 100644
index 0000000..58a7832
--- /dev/null
+++ b/recipes-extended/images/core-image-full-cmdline/wandboard/emmcsetup.lua
@@ -0,0 +1,12 @@
+function preinst()
+ local out = "Post installed script called"
+
+ return true, out
+
+end
+
+function postinst()
+ local out = "Post installed script called"
+
+ return true, out
+end
diff --git a/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description b/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description
new file mode 100644
index 0000000..b3598fc
--- /dev/null
+++ b/recipes-extended/images/core-image-full-cmdline/wandboard/sw-description
@@ -0,0 +1,71 @@

+software =
+{
+ version = "@@DISTRO_VERSION@@";
+
+ @@MACHINE@@ = {
+ hardware-compatibility: [ "revB", "revC", "revD"];
+ stable : {
+ copy1 : {
+ images: (
+ {
+ filename = "@@IMAGE_LINK_NAME@@.ext4.gz";
+ type = "raw";
+ compressed = true;

+ device = "/dev/mmcblk2p1";
+ }
+ );
+ scripts: (
+ {
+ filename = "emmcsetup.lua";
+ type = "lua";

+ }
+ );
+ uboot: (
+ {
+ name = "rootpart";
+ value = "1";

+ },
+ {
+ name = "finduuid";
+ value = "part uuid mmc 0:${rootpart} uuid";
+ },
+ {
+ name = "scan_dev_for_boot_part";
+ value = "setenv devplist ${rootpart};for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done";

+ }
+ );
+
+ };
+ copy2 : {
+ images: (
+ {
+ filename = "@@IMAGE_LINK_NAME@@.ext4.gz";
+ type = "raw";
+ compressed = true;

+ device = "/dev/mmcblk2p2";
+ }
+ );
+ scripts: (
+ {
+ filename = "emmcsetup.lua";
+ type = "lua";

+ }
+ );
+ uboot: (
+ {
+ name = "rootpart";
+ value = "2";

+ },
+ {
+ name = "finduuid";
+ value = "part uuid mmc 0:${rootpart} uuid";
+ },
+ {
+ name = "scan_dev_for_boot_part";
+ value = "setenv devplist ${rootpart};for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done";

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 16.02.21 um 09:02 schrieb Stefano Babic:

Does newer versions of OE support files relative to an include? Older
versions doesn't set the THISDIR for a inc file.

Why is a swu related to an image? What is the difference between a sd
image and an update in terms of it content? Both contain a rootfs and
some additional images like bootloader or firmware.

Why is an update related to an image? Does the content of the rootfs
really matter?

>> It should work for the simple and the complex solutions.
>
> .inc file, yes, but SWU variables in a MACHINE config is quite
> misleading, because this should mean it is machine specific, and it is
> really more image specific.

An .inc file is hard to override. A configuration file could be reused
between images and it is possible to provide a default configuration for
a machine or distro.

>>> The difference with a WKS is that we need here a configuration for
>>> the device (sw-description), that is also interpreted by the device,
>>> and a configuration for the build system, that should not go on the
>>> device (that means, the recipe), and the build system can require
>>> some part from sw-description (or in any cases, it fills
>>> automatically some parts). I cannot say that SWUpdate use a recipe on
>>> top of others, it is
>>> a recipe that depends on one (simple rootfs) or many other recipes,
>>> and in some cases even from images generated for other OE machines.
>>
>> The WKS also need some information for the device but they are
>> generated / extracted from the configuration and the WKS could also
>> depends on other recipes and there artifacts.
>
> But the device has nothing to do with it, while in case of SWU,
> sw-description is the description of the release and must understand it.

What do you mean?

>> At the moment I need an image recipe and an extra vfat image type
>> class to create a boot partition.
>
> Ok, this is to generate the .wic for the first setup of the device. I
> guess this cannot be avoided.

WIC avoid this problem and generates the file system on the fly.

Furthermore the update of a full filesystem instead of its content have
advantages and this leads to images for every partition.

>> Furthermore I need a update recipe per each image recipe.
>
> What I do is to have a .inc file shared by all update recipes, so that
> the update recipes are very simple. But yes, an update recipe per image
> is necessary, even if quite empty.

But you could avoid this with a swu image class and a configuration file.

So we should keep the sw-description. The question is how we could
define the build steps.

>>>> or use a separate configuration file (ex. WIC).
>>>
>>> Right, but wks is a configuration file for the buildsystem and the
>>> device does not need to bother with it. We have a use case here more
>>> similar to fitImage, but where there is quite no "standard" case like
>>> fitImage to allow the generation of a sw-description.
>>
>> The problem is not the missing standard case but the flexibility of
>> our configuration.
>>
>> Maybe we could define a swu.in or sw-description.in configuration
>> which extends the sw-description with build information?
>
> I do not know how, information must be removed then because they should
> not reach the device. And there is not just OE, but also Buildroot and
> Debian.

So you prefare a common solution like an swu build tool?

> I am also careful to change sw-description: it is ok to add new features
> / attributes, but they should not break update for devices already in
> field if they do not use the new features.

The build information shouldn't influence the update and should be
removed from the shipped sw-description.

>> Something like a source entry for the image, file and script entries.
>
> With just a hook, it does not work, I think. And handler must be
> specified (what about to update a UBIFS instead of eMMC ?), at the end
> it is a sw-description written in another format.

I don't speak about an other file format in front of sw-description.

>> We could reuse the filename to rename the files and the compressed and
>> encrypted entry to prepare the files. Furthermore we could add a
>> filesystem and name entry to an image to create a file system on the
>> fly from IMAGE_BOOT_FILES.
>>

My idea is to add build information into the sw-description. I have
rethink the solution and would use specific comments as decorations /
annotation inside the sw-description. This makes it easy to extract and
remove the build information from the sw-description. Each decorations
include a name for the file and an optional compression and encryption.
We could support raw files from the deploy directory, on the fly
filesystems via IMAGE_BOOT_FILES and the rootfs:

//! boot.bin.gz --source=rawcopy
--sourceparams="file=boot-@MACHINE@.bin" --compression=zlib
//! boot.vfat.gz --source=bootimg --fstype=vfat --compression=zlib
//! rootfs.ext4.gz --source=rootfs --fstype=ext4 --compression=zlib

Regards
Stefan

[Stefano Babic]

Hi Stefan,

I never need it - .inc files are searched across the layers, so I can
put a .inc somewhere and then add a require even in another layer with
the path from the TOPDIR of the layer with the .inc, like "require
recipes-bsp/u-boot/u-boot.inc" to make an example.

>
> Why is a  swu related to an image?

In fact, it is not...the relation is introduced now by this patchset. If
we introduce a .swu in IMAGE_FSTYPES, *it* is an image. This is also why
I am saying that adding support for FSTYPES just simplifies the very
simple use case, that is update=rootfs. In most my use casesm it is not.

A SWU is a package with multiple artifacts and with instructions how the
new software should be installed on the device. Not always it is derived
from an image recipe, I have also normal packages (that could be
installed as part of rootfs by bitbake) that generate a SWU, too,
because it is required to deliver in this way. Or even no OE package at
all, the SWU contains configuration / calibration / data with
instructions (in sw-description) so that SWUpdate knows what it has to
do. Completely unrelated.

But if the topic of the series is "swu image fstype", this binds a swu
with an image recipe.

> What is the difference between a sd
> image and an update in terms of it content? Both contain a rootfs and
> some additional images like bootloader or firmware.

Well, differences are in the usage. With SD, the device is passive: you
go in front of the device, you turn the device off, you insert the SD.
Device boots or not, but there is no instructions / code to update. In
OTA and in SWUpdate, the content (the SWU), apart of the artifacts that
we can consider like the SD, has also instructions and code to perform a
smooth update. This can help to fix hardware problems (change the
behavior before update), provide a migration with repartitioning,
whatever. And the behavior can be detected at runtime, so that
buildsystem cannot provide everything.

>
> Why is an update related to an image?

IMHO it is not, but adding swu fstype binds it.

> Does the content of the rootfs
> really matter?

No

>
>>> It should work for the simple and the complex solutions.
>>
>> .inc file, yes, but SWU variables in a MACHINE config is quite
>> misleading, because this should mean it is machine specific, and it is
>> really more image specific.
>
> An .inc file is hard to override.

Mmmhhh...I do not think so - a .inc is just a common part of a recipe,
and like all resipes, it is overridden by later set (after the require
statement) or by a .bbappend.

> A configuration file could be reused
> between images and it is possible to provide a default configuration for
> a machine or distro.

ok, let's say it is just another place to set SWUPDATE_ variables (or to
set the behavior of the class).

>
>
>>>> The difference with a WKS is that we need here a configuration for
>>>> the device (sw-description), that is also interpreted by the device,
>>>> and a configuration for the build system, that should not go on the
>>>> device (that means, the recipe), and the build system can require
>>>> some part from sw-description (or in any cases, it fills
>>>> automatically some parts). I cannot say that SWUpdate use a recipe
>>>> on top of others, it is
>>>> a recipe that depends on one (simple rootfs) or many other recipes,
>>>> and in some cases even from images generated for other OE machines.
>>>
>>> The WKS also need some information for the device but they are
>>> generated / extracted from the configuration and the WKS could also
>>> depends on other recipes and there artifacts.
>>
>> But the device has nothing to do with it, while in case of SWU,
>> sw-description is the description of the release and must understand it.
>
> What do you mean?

See above - WKS is 100% completed by the buildsystem, behavior of an
update via SWU is determined by the device at runtime. WKS just need to
pack all stuff together, you do not need to inform the device what to do
with the packed data.

>
>>> At the moment I need an image recipe and an extra vfat image type
>>> class to create a boot partition.
>>
>> Ok, this is to generate the .wic for the first setup of the device. I
>> guess this cannot be avoided.
>
> WIC avoid this problem and generates the file system on the fly.

I miss what you want to say here or what you find as problem. Is it the
generation of filesystem, maybe empty ?

>
> Furthermore the update of a full filesystem instead of its content have
> advantages and this leads to images for every partition.

Also I do not get, the "images" section in sw-description provides
exactly what you mean, and this feature was from the early beginning in
SWUpdate. SWUpdate does not know the contents.

>
>>> Furthermore I need a update recipe per each image recipe.
>>
>> What I do is to have a .inc file shared by all update recipes, so that
>> the update recipes are very simple. But yes, an update recipe per
>> image is necessary, even if quite empty.
>
> But you could avoid this with a swu image class and a configuration file.

So I need an image class, sw-description and a configuration file.
Currently we have the image class to create the SWU, sw-description and
a recipe for the SWU. Apart that configuration is put into a .conf
instead of a recipe (and yes, I factorize this with .inc currently), I
am missing the real difference and advantages..

I do not see any way to generate it but in the very simple cases (that I
have not since a long time, anyway).

> The question is how we could
> define the build steps.

Ok

>
>
>>>>> or use a separate configuration file (ex. WIC).
>>>>
>>>> Right, but wks is a configuration file for the buildsystem and the
>>>> device does not need to bother with it. We have a use case here more
>>>> similar to fitImage, but where there is quite no "standard" case
>>>> like fitImage to allow the generation of a sw-description.
>>>
>>> The problem is not the missing standard case but the flexibility of
>>> our configuration.
>>>
>>> Maybe we could define a swu.in or sw-description.in configuration
>>> which extends the sw-description with build information?
>>
>> I do not know how, information must be removed then because they
>> should not reach the device. And there is not just OE, but also
>> Buildroot and Debian.
>
> So you prefare a common solution like an swu build tool?

This is an option, but it is much more interesting for projects based on
Debian. But as sw-description is part of SWUpdate, and the project is
unaware of the buildsystem, is quite confusing if sw-description
contains parts that are very specific to a buildsystem.

>
>> I am also careful to change sw-description: it is ok to add new
>> features / attributes, but they should not break update for devices
>> already in field if they do not use the new features.
>
> The build information shouldn't influence the update and should be
> removed from the shipped sw-description.

But if I retrieve the info from sw-description, I process it and then I
remove from sw-description again, what is the main difference as letting
this build information outside sw-description ?

>
>>> Something like a source entry for the image, file and script entries.
>>
>> With just a hook, it does not work, I think. And handler must be
>> specified (what about to update a UBIFS instead of eMMC ?), at the end
>> it is a sw-description written in another format.
>
> I don't speak about an other file format in front of sw-description.

Ok

>
>>> We could reuse the filename to rename the files and the compressed
>>> and encrypted entry to prepare the files. Furthermore we could add a
>>> filesystem and name entry to an image to create a file system on the
>>> fly from IMAGE_BOOT_FILES.
>>>
>
> My idea is to add build information into the sw-description. I have
> rethink the solution and would use specific comments as decorations /
> annotation inside the sw-description. This makes it easy to extract and
> remove the build information from the sw-description. Each decorations
> include a name for the file and an optional compression and encryption.
> We could support raw files from the deploy directory, on the fly
> filesystems via IMAGE_BOOT_FILES and the rootfs:
>
> //! boot.bin.gz --source=rawcopy
> --sourceparams="file=boot-@MACHINE@.bin" --compression=zlib
> //! boot.vfat.gz --source=bootimg --fstype=vfat --compression=zlib
> //! rootfs.ext4.gz --source=rootfs --fstype=ext4 --compression=zlib
>

Which is your main goal ? So it is more important to drop the recipe for
the SWU or to drop the configuration via variables and use WKS like
instructions ? Or both ?

Best regards,

[Stefano Babic]

Hi Anatolji,

On 16.02.21 10:52, Anatolij Gustschin wrote:
> Enable swuimage task if building 'update-image' recipe of the
> meta-swupdate-boards layer or if building usual image recipes
> with appended SRC_URI and enabled 'swu' image fstype. Fetch
> SRC_URI files if 'swu' image fstype was selected.
>
> This is in preparation of support for 'swu' image type class
> (for building .swu images without meta-swupdate-boards layer).
>
> Signed-off-by: Anatolij Gustschin <ag...@denx.de>
> ---
> classes/swupdate.bbclass | 31 ++++++++++++++++++++++++++++---
> 1 file changed, 28 insertions(+), 3 deletions(-)
>
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 2c2430b..c2899be 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -51,7 +51,10 @@ def swupdate_getdepends(d):
>
> depstr = ""
> for dep in deps:
> - depstr += " " + dep + ":do_build"
> + if dep == d.getVar('PN'):
> + depstr += " " + dep + ":do_image_complete"
> + else:
> + depstr += " " + dep + ":do_build"
> return depstr

mmhhhh..

also, you are checking if 'PN' exists and if it is the same as the name
of your recipe. If this is true, you decide that this is an image, and
it depends on do_image_complete. It looks to me a very indirect way.
Really (I have not checked if and how this can be reached), you want to
know for the dependency "dep" if it has a task called "do_image_complete".

What about if I build a SWU from a package ? This is a use case on my
site. I have to package a file (a license file) into a SWU, and this
will be installed by SWUpdate. It is something like:

SRC_URI = "\
file://file_to_beput_in_swu \
file://sw-description \
file://something_else.lua \
"

FILES_${PN} = ....
inherit swupdate

....re-enable do_install, do_package,... ---

...


The result is a package (for rootfs) + a SWU (SWU is like another
packaging) to install this alone. Your change above should break this,
because it will add a dep to do_image_complete.

>
> IMGDEPLOYDIR = "${WORKDIR}/deploy-${PN}-swuimage"
> @@ -76,14 +79,38 @@ do_package_write_ipk[noexec] = "1"
> do_package_write_deb[noexec] = "1"
> do_package_write_rpm[noexec] = "1"
>
> +USING_SWU = "${@bb.utils.contains('IMAGE_FSTYPES', 'swu', '1', '', d)}"
> +
> python () {
> deps = " " + swupdate_getdepends(d)
> d.appendVarFlag('do_swuimage', 'depends', deps)
> +
> + # add swuimage task if building with swu image type or if
> + # building via 'update-image' recipe in meta-swupdate-boards
> + if d.getVar('PN') == 'update-image':

meta-swupdate-boards is just a bunch of examples, and yes the name for
SWU is "update-image", but this class should be unaware of it. There
should be no constrain how to name the recipes for SWUs.

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 17.02.21 um 10:02 schrieb Stefano Babic:

How do you share a sw-description file between different layers?

>> Why is a  swu related to an image?
>
> In fact, it is not...the relation is introduced now by this patchset. If
> we introduce a .swu in IMAGE_FSTYPES, *it* is an image. This is also why
> I am saying that adding support for FSTYPES just simplifies the very
> simple use case, that is update=rootfs. In most my use casesm it is not.

But isn't it a valid use case? In my case many images consists of a
rootfs, boot loader, firmware and recovery images. The updates differ
only in the rootfs image.

> A SWU is a package with multiple artifacts and with instructions how the
> new software should be installed on the device. Not always it is derived
> from an image recipe, I have also normal packages (that could be
> installed as part of rootfs by bitbake) that generate a SWU, too,
> because it is required to deliver in this way.

Why do you have to combine both in one? The same would be possible with
a recipe which installs or deploys the file and a generalized
baremetal-image.

> Or even no OE package at
> all, the SWU contains configuration / calibration / data with
> instructions (in sw-description) so that SWUpdate knows what it has to
> do. Completely unrelated.

Does it matter if it is handled by an empty package or an empty image?

> But if the topic of the series is "swu image fstype", this binds a swu
> with an image recipe.

Only if the topic imply to remove the old way.

>> What is the difference between a sd
>> image and an update in terms of it content? Both contain a rootfs and
>> some additional images like bootloader or firmware.
>
> Well, differences are in the usage. With SD, the device is passive: you
> go in front of the device, you turn the device off, you insert the SD.
> Device boots or not, but there is no instructions / code to update. In
> OTA and in SWUpdate, the content (the SWU), apart of the artifacts that
> we can consider like the SD, has also instructions and code to perform a
> smooth update. This can help to fix hardware problems (change the
> behavior before update), provide a migration with repartitioning,
> whatever. And the behavior can be detected at runtime, so that
> buildsystem cannot provide everything.

In both cases the build systems take some artifacts and extend it with
additional information. In case of the wks it doesn't matter which image
the user build. Why does it matters for an swu?

>> Why is an update related to an image?
>
> IMHO it is not, but adding swu fstype binds it.

Why?

Does it really matter which image (rootfs) is packed inside the swu?

The image class only specify how an image should be packed.

>> Does the content of the rootfs
>> really matter?
>
> No

But why we need a recipe per image.

>>>> It should work for the simple and the complex solutions.
>>>
>>> .inc file, yes, but SWU variables in a MACHINE config is quite
>>> misleading, because this should mean it is machine specific, and it is
>>> really more image specific.
>>
>> An .inc file is hard to override.
>
> Mmmhhh...I do not think so - a .inc is just a common part of a recipe,
> and like all resipes, it is overridden by later set (after the require
> statement) or by a .bbappend.

The main difference is that an .inc file is immediately parsed. The
WKS_FILE variable could be overridden by an recipe.

>> A configuration file could be reused
>> between images and it is possible to provide a default configuration for
>> a machine or distro.
>
> ok, let's say it is just another place to set SWUPDATE_ variables (or to
> set the behavior of the class).

It depends on the implementation. Maybe it is independent of the class
and a configuration for an swu build tool.

The problem is that WIC could generate filesystems on the fly. You
doesn't need multiple recipe to create different filesystems.

>> Furthermore the update of a full filesystem instead of its content have
>> advantages and this leads to images for every partition.
>
> Also I do not get, the "images" section in sw-description provides
> exactly what you mean, and this feature was from the early beginning in
> SWUpdate. SWUpdate does not know the contents.
>
>>
>>>> Furthermore I need a update recipe per each image recipe.
>>>
>>> What I do is to have a .inc file shared by all update recipes, so that
>>> the update recipes are very simple. But yes, an update recipe per
>>> image is necessary, even if quite empty.
>>
>> But you could avoid this with a swu image class and a configuration file.
>
> So I need an image class, sw-description and a configuration file.
> Currently we have the image class to create the SWU, sw-description and
> a recipe for the SWU. Apart that configuration is put into a .conf
> instead of a recipe (and yes, I factorize this with .inc currently), I
> am missing the real difference and advantages..

An image class could create an swu for arbitrary images. The machine
could define the dependencies and content of the swu beside the rootfs.
An image could override the dependencies and content. The image class
could reuse exists variables to create filesystems.

But we could do semi generation via variable expansion. This allows the
image class to define the artifact name, compression type, encryption
and filesystem type.

>> The question is how we could
>> define the build steps.
>
> Ok
>
>>
>>
>>>>>> or use a separate configuration file (ex. WIC).
>>>>>
>>>>> Right, but wks is a configuration file for the buildsystem and the
>>>>> device does not need to bother with it. We have a use case here more
>>>>> similar to fitImage, but where there is quite no "standard" case
>>>>> like fitImage to allow the generation of a sw-description.
>>>>
>>>> The problem is not the missing standard case but the flexibility of
>>>> our configuration.
>>>>
>>>> Maybe we could define a swu.in or sw-description.in configuration
>>>> which extends the sw-description with build information?
>>>
>>> I do not know how, information must be removed then because they
>>> should not reach the device. And there is not just OE, but also
>>> Buildroot and Debian.
>>
>> So you prefare a common solution like an swu build tool?
>
> This is an option, but it is much more interesting for projects based on
> Debian. But as sw-description is part of SWUpdate, and the project is
> unaware of the buildsystem, is quite confusing if sw-description
> contains parts that are very specific to a buildsystem.

Okay.

>>> I am also careful to change sw-description: it is ok to add new
>>> features / attributes, but they should not break update for devices
>>> already in field if they do not use the new features.
>>
>> The build information shouldn't influence the update and should be
>> removed from the shipped sw-description.
>
> But if I retrieve the info from sw-description, I process it and then I
> remove from sw-description again, what is the main difference as letting
> this build information outside sw-description ?

What is the advantage of two files? Does it make sense to use the same
sw-description for different build instructions if it excluded the
source name of the artifacts.

>>>> Something like a source entry for the image, file and script entries.
>>>
>>> With just a hook, it does not work, I think. And handler must be
>>> specified (what about to update a UBIFS instead of eMMC ?), at the end
>>> it is a sw-description written in another format.
>>
>> I don't speak about an other file format in front of sw-description.
>
> Ok
>
>>
>>>> We could reuse the filename to rename the files and the compressed
>>>> and encrypted entry to prepare the files. Furthermore we could add a
>>>> filesystem and name entry to an image to create a file system on the
>>>> fly from IMAGE_BOOT_FILES.
>>>>
>>
>> My idea is to add build information into the sw-description. I have
>> rethink the solution and would use specific comments as decorations /
>> annotation inside the sw-description. This makes it easy to extract and
>> remove the build information from the sw-description. Each decorations
>> include a name for the file and an optional compression and encryption.
>> We could support raw files from the deploy directory, on the fly
>> filesystems via IMAGE_BOOT_FILES and the rootfs:
>>
>> //! boot.bin.gz --source=rawcopy
>> --sourceparams="file=boot-@MACHINE@.bin" --compression=zlib
>> //! boot.vfat.gz --source=bootimg --fstype=vfat --compression=zlib
>> //! rootfs.ext4.gz --source=rootfs --fstype=ext4 --compression=zlib
>>
>
> Which is your main goal ? So it is more important to drop the recipe for
> the SWU or to drop the configuration via variables and use WKS like
> instructions ? Or both ?

The main goal is to drop the duplicated recipe for the swu and to reuse
the configurations of WIC. I would like to build a wks and swu image of
an arbitrary image.

Regards
Stefan

[Stefano Babic]

Hi Stefan,

I just put it into a repo and then I fetch it from recipe.

>
>
>>> Why is a  swu related to an image?
>>
>> In fact, it is not...the relation is introduced now by this patchset. If
>> we introduce a .swu in IMAGE_FSTYPES, *it* is an image. This is also why
>> I am saying that adding support for FSTYPES just simplifies the very
>> simple use case, that is update=rootfs. In most my use casesm it is not.
>
> But isn't it a valid use case?

Of course it is. But simplifying this use case should not discard
flexibility and cause other use cases, even if they are rare, impossible
to be implemented.

> In my case many images consists of a
> rootfs, boot loader, firmware and recovery images. The updates differ
> only in the rootfs image.

Ok - what I do in this case is to have a sw-description with dynamic
rootfs and yes, one recipe for each rootfs image to set this variable.
Let's see what is going on.

>
>
>> A SWU is a package with multiple artifacts and with instructions how the
>> new software should be installed on the device. Not always it is derived
>> from an image recipe, I have also normal packages (that could be
>> installed as part of rootfs by bitbake) that generate a SWU, too,
>> because it is required to deliver in this way.
>
> Why do you have to combine both in one? The same would be possible with
> a recipe which installs or deploys the file and a generalized
> baremetal-image.

No.

This is for SWUpdate, that is this package *must* be a SWU because
SWUpdate have to accept it and process it.

One use case is a license file with keys, and sw-description contains
instructions how to process it to allow to install further SWUs. This
license file can be part of a rootfs or not, and it is required in any
case for migration. But I have several other use case where what is
delivered does not belong to the classic schema (rootfs and just
software to be installed).

>
>
>> Or even no OE package at
>> all, the SWU contains configuration / calibration / data with
>> instructions (in sw-description) so that SWUpdate knows what it has to
>> do. Completely unrelated.
>
> Does it matter if it is handled by an empty package or an empty image?
>
>
>> But if the topic of the series is "swu image fstype", this binds a swu
>> with an image recipe.
>
> Only if the topic imply to remove the old way.

Right, but this is what I discovered with Anatolji's patches, because
they break some use cases of mine.

>
>
>>> What is the difference between a sd
>>> image and an update in terms of it content? Both contain a rootfs and
>>> some additional images like bootloader or firmware.
>>
>> Well, differences are in the usage. With SD, the device is passive: you
>> go in front of the device, you turn the device off, you insert the SD.
>> Device boots or not, but there is no instructions / code to update. In
>> OTA and in SWUpdate, the content (the SWU), apart of the artifacts that
>> we can consider like the SD, has also instructions and code to perform a
>> smooth update. This can help to fix hardware problems (change the
>> behavior before update), provide a migration with repartitioning,
>> whatever. And the behavior can be detected at runtime, so that
>> buildsystem cannot provide everything.
>
> In both cases the build systems take some artifacts and extend it with
> additional information. In case of the wks it doesn't matter which image
> the user build. Why does it matters for an swu?
>
>
>>> Why is an update related to an image?
>>
>> IMHO it is not, but adding swu fstype binds it.
>
> Why?

Because in OE a fstype is strict bound to an image (I need a recipe
image to use it), while a SWU is *not* bound to an image, it just
collects a list of artifacts. If we want to add a fstype, that is a
class to for .swu to be used with IMAGE_FSTYPES, we bind with an image.
But as you also say, the SWU is unrelated to an image or a rootfs.

>
> Does it really matter which image (rootfs) is packed inside the swu?

It does not matter : in fact, it is currently unaware. The recipe just
gets a list of artifacts, they can be a rootfs or not, it does not
matter at all. But this is the current state, what this series is
willing to change and then bind to a rootfs.

>
> The image class only specify how an image should be packed.

Right, but we need an image for it, and there are simply *no images* in
some use cases.

>
>
>>> Does the content of the rootfs
>>> really matter?
>>
>> No
>
> But why we need a recipe per image.

Exactly because it is unrelated, and we need to describe which
components are part of a SWU.

>
>
>>>>> It should work for the simple and the complex solutions.
>>>>
>>>> .inc file, yes, but SWU variables in a MACHINE config is quite
>>>> misleading, because this should mean it is machine specific, and it is
>>>> really more image specific.
>>>
>>> An .inc file is hard to override.
>>
>> Mmmhhh...I do not think so - a .inc is just a common part of a recipe,
>> and like all resipes, it is overridden by later set (after the require
>> statement) or by a .bbappend.
>
> The main difference is that an .inc file is immediately parsed. The
> WKS_FILE variable could be overridden by an recipe.

Right - but then we need a WKS file for each image. To maintain, it is
not quite different as having a recipe per image for each SWU.

I am quite losing which are the goals here or what we want to address. I
try to sumarize, maybe I misunderstand something:

- we need currently a recipe per image in case of different rootfs, and
adding a .swu fstype lets to avoid the recipe. However, sw-description
cannot be generated as I said previously, it can be factorized among
images.

- there are some cases not covered with current generation, related to
filesystem generation. I am not sure I have understood this. SWUpdate
delivers artifacts. If a filesystem must be created, the device can
generally do this itself, first eample at first boot after update. Or
this can be executed inside the update process as part of a preinstall
part. Why this should be done when SWU is generated, is something I am
missing: if the filesystem contains files, well, then it is an "image"
and can be built in OE, and then added to the list of artifacts. If the
filesystem is empty, if it is generated at build time it must be
delivered, too, increasing the SWU size (but an empty filesystem can be
well compressed), and it looks to me more suitable that the device does it.

- is there another goal ?

>
>
>>> A configuration file could be reused
>>> between images and it is possible to provide a default configuration for
>>> a machine or distro.
>>
>> ok, let's say it is just another place to set SWUPDATE_ variables (or to
>> set the behavior of the class).
>
> It depends on the implementation. Maybe it is independent of the class
> and a configuration for an swu build tool.

Ok

So is this the issue you want to address ? Anyway, the WIC is also quite
limited. It works fine for the rootfs, and I can use the rawcopy plugin
with --source rootfs. For some parts (see IMAGE_BOOTFILES), we can add
also some files to a generated filesystem (EFI is the clear example).
But I cannot combine partitions from different images, like a rootfs and
a partition just containing the application files, that is generated
with a different image recipe.

>
>
>>> Furthermore the update of a full filesystem instead of its content have
>>> advantages and this leads to images for every partition.
>>
>> Also I do not get, the "images" section in sw-description provides
>> exactly what you mean, and this feature was from the early beginning in
>> SWUpdate. SWUpdate does not know the contents.
>>
>>>
>>>>> Furthermore I need a update recipe per each image recipe.
>>>>
>>>> What I do is to have a .inc file shared by all update recipes, so that
>>>> the update recipes are very simple. But yes, an update recipe per
>>>> image is necessary, even if quite empty.
>>>
>>> But you could avoid this with a swu image class and a configuration
>>> file.
>>
>> So I need an image class, sw-description and a configuration file.
>> Currently we have the image class to create the SWU, sw-description and
>> a recipe for the SWU. Apart that configuration is put into a .conf
>> instead of a recipe (and yes, I factorize this with .inc currently), I
>> am missing the real difference and advantages..
>
> An image class could create an swu for arbitrary images. The machine
> could define the dependencies and content of the swu beside the rootfs.
> An image could override the dependencies and content. The image class
> could reuse exists variables to create filesystems.

Ok, I am quite sure now that the big issue on your use case is that
filesystems on empty partitions are not generated when SWU is built.

They are unrelated and no dependencies are created.

> Does it make sense to use the same
> sw-description for different build instructions if it excluded the
> source name of the artifacts.

This is what I often do with variable substitution (for rootfs), having
just one sw-description, but multiple recipes for the different use
cases. I am trying to understand if the proposed is just an alternative
implementation or there is really some added values.

Ok, this is clear.

> and to reuse
> the configurations of WIC.

You mean same configuration ? So are you thinking to a WIC plugin that
generates a SWU ?

> I would like to build a wks and swu image of
> an arbitrary image.

Regards,

[Stefan Herbrechtsmeier]

Hi Stefano,

apologize the long delay.

Am 01.03.21 um 10:02 schrieb Stefano Babic:

> Hi Stefan,
>
> On 28.02.21 21:06, Stefan Herbrechtsmeier wrote:
>> Hi Stefano,
>>
>> Am 17.02.21 um 10:02 schrieb Stefano Babic:
>>> Hi Stefan,
>>>
>>> On 16.02.21 21:34, Stefan Herbrechtsmeier wrote:
>>>> Hi Stefano,
>>>>
>>>> Am 16.02.21 um 09:02 schrieb Stefano Babic:
>>>>> Hi Stefan,
>>>>>
>>>>> On 15.02.21 20:21, Stefan Herbrechtsmeier wrote:
>>>>>> Hi Stefano,
>>>>>>
>>>>>> Am 14.02.21 um 18:15 schrieb Stefano Babic:
>>>>>>> Hi Stefan,
>>>>>>>
>>>>>>> On 14.02.21 16:11, Stefan Herbrechtsmeier wrote:
>>>>>>>> Hi Stefano,
>>>>>>>>
>>>>>>>> Am 14.02.21 um 10:34 schrieb Stefano Babic:
>>>>>>>>> Hi Stefan,
>>>>>>>>>
>>>>>>>>> On 13.02.21 17:05, Stefan Herbrechtsmeier wrote:
>>>>>>>>>> Hi Stefano,
>>>>>>>>>>
>>>>>>>>>> Am 11.02.21 um 13:25 schrieb Stefano Babic:
>>>>>>>>>>> Hi Anatolji,
>>>>>>>>>>>
>>>>>>>>>>> On 11.02.21 11:38, Anatolij Gustschin wrote:

[snip]

For me the rootfs is only a specific image. An image consists of
different deploy artifacts and / or packages. I distinguish between
normal recipes which produce packages and deploy binary artifacts on one
side and images which create special deploy artifacts only.

I think the normal system update swu should be independent of the
concrete rootfs image. At the moment I could create a wic image for an
arbitrary rootfs image and I don't understand why this shouldn't be
possible for system update swu. In any case you could override the
common configuration and use image specific configurations for specific
images.

> I am quite losing which are the goals here or what we want to address. I
> try to sumarize, maybe I misunderstand something:
>
> - we need currently a recipe per image in case of different rootfs, and
> adding a .swu fstype lets to avoid the recipe. However, sw-description
> cannot be generated as I said previously, it can be factorized among
> images.

A swu fstype allows a generic default sw-description per distro or
machine and support an arbitrary rootfs.

> - there are some cases not covered with current generation, related to
> filesystem generation. I am not sure I have understood this. SWUpdate
> delivers artifacts. If a filesystem must be created, the device can
> generally do this itself, first eample at first boot after update. Or
> this can be executed inside the update process as part of a preinstall
> part. Why this should be done when SWU is generated, is something I am
> missing: if the filesystem contains files, well, then it is an "image"
> and can be built in OE, and then added to the list of artifacts.

Yes, but you need a recipe per image. The WIC image can do this without
a special recipe per image / partition for arbitrary deploy artifacts.
An common example is the boot partition or a partition with a filesystem
and the recovery FIT image.

> If the
> filesystem is empty, if it is generated at build time it must be
> delivered, too, increasing the SWU size (but an empty filesystem can be
> well compressed), and it looks to me more suitable that the device does it.
>
> - is there another goal ?

No.


[snip]

>> The problem is that WIC could generate filesystems on the fly. You
>> doesn't need multiple recipe to create different filesystems.
>
> So is this the issue you want to address ? Anyway, the WIC is also quite
> limited. It works fine for the rootfs, and I can use the rawcopy plugin
> with --source rootfs. For some parts (see IMAGE_BOOTFILES), we can add
> also some files to a generated filesystem (EFI is the clear example).
> But I cannot combine partitions from different images, like a rootfs and
> a partition just containing the application files, that is generated
> with a different image recipe.

This should be possible with the rawcopy plugin or you could split the
rootfs on the fly with --exclude-path and --change-directory. And again
you could avoid an additional recipe if you use the later option.


[snip]

> Ok, I am quite sure now that the big issue on your use case is that
> filesystems on empty partitions are not generated when SWU is built.

No, my issue is the needed extra recipes. WIC could create images with
renamed deployed artifacts on the fly and I could build arbitrary images.


[snip]

>> Does it make sense to use the same sw-description for different build
>> instructions if it excluded the source name of the artifacts.
>
> This is what I often do with variable substitution (for rootfs), having
> just one sw-description, but multiple recipes for the different use
> cases. I am trying to understand if the proposed is just an alternative
> implementation or there is really some added values.

The added value is no needed extra recipes and that it works out of the box.


[snip]

>>> Which is your main goal ? So it is more important to drop the recipe for
>>> the SWU or to drop the configuration via variables and use WKS like
>>> instructions ? Or both ?
>>
>> The main goal is to drop the duplicated recipe for the swu
>
> Ok, this is clear.
>
>> and to reuse the configurations of WIC.
>
> You mean same configuration ? So are you thinking to a WIC plugin that
> generates a SWU ?

This is one possible solution. An other solution could be an extra
boot-image recipe. In any case we should reuse matching WIC variables.

Regards
Stefan

[Stefano Babic]

Hi Stefan,

On 05.04.21 15:36, Stefan Herbrechtsmeier wrote:
> Hi Stefano,
>

> apologize the long delay.
>

No problem - it looks like that Eastern's holiday give us some time to
check again older topics ;-)

Right, I see the same0

> I distinguish between
> normal recipes which produce packages and deploy binary artifacts on one
> side and images which create special deploy artifacts only.
>
> I think the normal system update swu should be independent of the
> concrete rootfs image.

IMHO it is independent now. The SWU does not know if one of the artifact
is an image. We can have a SWU with any kind of artifacts, even without
a rootfs.

> At the moment I could create a wic image for an
> arbitrary rootfs image and I don't understand why this shouldn't be
> possible for system update swu.

Well, right, but a WIC depends on an image that is a rootfs, while SWU
not. In fact, in a wic I set a --source rootfs if I want to put the
filesystem in one of the partition. I cannot (at the moment) add a
different image, like for example another filesystem. The only exception
is the generation of EFI images via plugin.

> In any case you could override the
> common configuration and use image specific configurations for specific
> images.

That's right, but there are always a 1:1 relationship with a rootfs /
image (or how we name it). But I do not see how to build a single WIC
where each partitition is generated with a different recipe, and I put
each image in a different partition. Just as simple case: two images
with OS and application in separate partitions.

On the other side, I do not see how generated a wic from SWU can help. A
SWU is understood just from SWUpdate, it is not a filesystem (then
putting in a partition makes less sense).

>
>> I am quite losing which are the goals here or what we want to address.
>> I try to sumarize, maybe I misunderstand something:
>>
>> - we need currently a recipe per image in case of different rootfs,
>> and adding a .swu fstype lets to avoid the recipe. However,
>> sw-description cannot be generated as I said previously, it can be
>> factorized among images.
>
> A swu fstype allows a generic default sw-description per distro or
> machine and support an arbitrary rootfs.

Yes, but this contradicts your first statement, that an image is just a
set of artifacts to be deplowed. A SWU fstype means a rootfs.

>
>
>> - there are some cases not covered with current generation, related to
>> filesystem generation. I am not sure I have understood this. SWUpdate
>> delivers artifacts. If a filesystem must be created, the device can
>> generally do this itself, first eample at first boot after update. Or
>> this can be executed inside the update process as part of a preinstall
>> part. Why this should be done when SWU is generated, is something I am
>> missing: if the filesystem contains files, well, then it is an "image"
>> and can be built in OE, and then added to the list of artifacts.
>
> Yes, but you need a recipe per image.

That's right - but I am not understanding why this is a problem when I
can get as benefit much flexibility.

> The WIC image can do this without
> a special recipe per image / partition for arbitrary deploy artifacts.

How ? It is not clear to me, maybe I haven't solved this. Let's say I
need to create several filesystem for a device, and then I put all
together with a WIC. But the WIC is specified for each filesystem, that
is the deployable of each recipe having a "inherit image". How can I get
a WIC to deploy all images at once ? Is it possible ? (I am not aware of
it).

> An common example is the boot partition or a partition with a filesystem
> and the recovery FIT image.

IMHO they are not so good examples - the boot (EFI) partition cannot be
any kind of generated image, that means I cannot create an image as
usual, set the filesystem to VFAT and put it as first partition via WIC.
WIC generates the partition following a schema, and this matches for
most cases (but not in all of them).

>
>
>> If the filesystem is empty, if it is generated at build time it must
>> be delivered, too, increasing the SWU size (but an empty filesystem
>> can be well compressed), and it looks to me more suitable that the
>> device does it.
>>
>> - is there another goal ?
>
> No.
>
>
> [snip]
>
>>> The problem is that WIC could generate filesystems on the fly. You
>>> doesn't need multiple recipe to create different filesystems.
>>
>> So is this the issue you want to address ? Anyway, the WIC is also
>> quite limited. It works fine for the rootfs, and I can use the rawcopy
>> plugin with --source rootfs. For some parts (see IMAGE_BOOTFILES), we
>> can add also some files to a generated filesystem (EFI is the clear
>> example).
>> But I cannot combine partitions from different images, like a rootfs
>> and a partition just containing the application files, that is
>> generated with a different image recipe.
>
> This should be possible with the rawcopy plugin or you could split the
> rootfs on the fly with --exclude-path and --change-directory. And again
> you could avoid an additional recipe if you use the later option.

Ok, so at the end the discussion is to avoid to have a second recipe.
This is fine with me if it does not break any use case. This was not
with Anatolji's patches, they are breaking some use cases of mine.

>
>
> [snip]
>
>> Ok, I am quite sure now that the big issue on your use case is that
>> filesystems on empty partitions are not generated when SWU is built.
>
> No, my issue is the needed extra recipes.

Ok, understood.

> WIC could create images with
> renamed deployed artifacts on the fly and I could build arbitrary images.

I know they can build artifacts on the fly, my intention is to have a
SWU that is not aware of all about what is deployed.

Anyway, I get your goal and if we find a suiztable way where the two
ways can coexist, it is fine.

>
>
> [snip]
>
>>> Does it make sense to use the same sw-description for different build
>>> instructions if it excluded the source name of the artifacts.
>>
>> This is what I often do with variable substitution (for rootfs),
>> having just one sw-description, but multiple recipes for the different
>> use cases. I am trying to understand if the proposed is just an
>> alternative implementation or there is really some added values.
>
> The added value is no needed extra recipes and that it works out of the
> box.

Ok, got it.

>
>
> [snip]
>
>>>> Which is your main goal ? So it is more important to drop the recipe
>>>> for
>>>> the SWU or to drop the configuration via variables and use WKS like
>>>> instructions ? Or both ?
>>>
>>> The main goal is to drop the duplicated recipe for the swu
>>
>> Ok, this is clear.
>>
>>> and to reuse the configurations of WIC.
>>
>> You mean same configuration ? So are you thinking to a WIC plugin that
>> generates a SWU ?
>
> This is one possible solution. An other solution could be an extra
> boot-image recipe. In any case we should reuse matching WIC variables.
>
> Regards
>   Stefan

[Stefan Herbrechtsmeier]

Hi Stefano,

Am 05.04.21 um 16:20 schrieb Stefano Babic:

Is the --source rootfs mandatory? Couldn't you create an empty rootfs
and use rawcopy to reuse other image artifacts?

>> In any case you could override the common configuration and use image
>> specific configurations for specific images.
>
> That's right, but there are always a 1:1 relationship with a rootfs /
> image (or how we name it). But I do not see how to build a single WIC
> where each partitition is generated with a different recipe, and I put
> each image in a different partition. Just as simple case: two images
> with OS and application in separate partitions.

This is possible. You could use rawcopy if you already have a recipe for
the application image or you could install the application in a specific
directory and use --exclude-path and --change-directory to split the
rootfs into two partitions.

> On the other side, I do not see how generated a wic from SWU can help. A
> SWU is understood just from SWUpdate, it is not a filesystem (then
> putting in a partition makes less sense).

What do you mean?

>>> I am quite losing which are the goals here or what we want to
>>> address. I try to sumarize, maybe I misunderstand something:
>>>
>>> - we need currently a recipe per image in case of different rootfs,
>>> and adding a .swu fstype lets to avoid the recipe. However,
>>> sw-description cannot be generated as I said previously, it can be
>>> factorized among images.
>>
>> A swu fstype allows a generic default sw-description per distro or
>> machine and support an arbitrary rootfs.
>
> Yes, but this contradicts your first statement, that an image is just a
> set of artifacts to be deplowed. A SWU fstype means a rootfs.

You are right. It must be deployed artifacts and packages.

>>> - there are some cases not covered with current generation, related
>>> to filesystem generation. I am not sure I have understood this.
>>> SWUpdate delivers artifacts. If a filesystem must be created, the
>>> device can generally do this itself, first eample at first boot after
>>> update. Or this can be executed inside the update process as part of
>>> a preinstall part. Why this should be done when SWU is generated, is
>>> something I am missing: if the filesystem contains files, well, then
>>> it is an "image" and can be built in OE, and then added to the list
>>> of artifacts.
>>
>> Yes, but you need a recipe per image.
>
> That's right - but I am not understanding why this is a problem when I
> can get as benefit much flexibility.

Because everybody have to pay the price even if he doesn't need the
flexibility. The complexity should scale with the needed flexibility.

>
>> The WIC image can do this without a special recipe per image /
>> partition for arbitrary deploy artifacts.
>
> How ? It is not clear to me, maybe I haven't solved this. Let's say I
> need to create several filesystem for a device, and then I put all
> together with a WIC. But the WIC is specified for each filesystem, that
> is the deployable of each recipe having a "inherit image". How can I get
> a WIC to deploy all images at once ? Is it possible ? (I am not aware of
> it).

What is the content of the filesystem? You could use rawcopy for
filesystem images (created by other image recipes), bootimg-partition to
create a filesystem from deployed images and split the rootfs into
different partitions. It is possible to create partitions for boot
loader, kernel, recovery, system and data without any additional recipe.

>> An common example is the boot partition or a partition with a
>> filesystem and the recovery FIT image.

This is possible. We use a dual copy for boot loader, firmware,
recovery, kernel and system. We create a wic image without any
additional recipe.

> IMHO they are not so good examples - the boot (EFI) partition cannot be
> any kind of generated image, that means I cannot create an image as
> usual, set the filesystem to VFAT and put it as first partition via WIC.

Again this should be possible with rawcopy and an fstype class for vfat.
But why do you need a special recipe for this image? bootimg-partition
could create a vfat partition for you and copy and rename images from
deploy into the filesystem.

> WIC generates the partition following a schema, and this matches for
> most cases (but not in all of them).

What case isn't supported by wic?

>>> If the filesystem is empty, if it is generated at build time it must
>>> be delivered, too, increasing the SWU size (but an empty filesystem
>>> can be well compressed), and it looks to me more suitable that the
>>> device does it.
>>>
>>> - is there another goal ?
>>
>> No.
>>
>>
>> [snip]
>>
>>>> The problem is that WIC could generate filesystems on the fly. You
>>>> doesn't need multiple recipe to create different filesystems.
>>>
>>> So is this the issue you want to address ? Anyway, the WIC is also
>>> quite limited. It works fine for the rootfs, and I can use the
>>> rawcopy plugin with --source rootfs. For some parts (see
>>> IMAGE_BOOTFILES), we can add also some files to a generated
>>> filesystem (EFI is the clear example).
>>> But I cannot combine partitions from different images, like a rootfs
>>> and a partition just containing the application files, that is
>>> generated with a different image recipe.
>>
>> This should be possible with the rawcopy plugin or you could split the
>> rootfs on the fly with --exclude-path and --change-directory. And
>> again you could avoid an additional recipe if you use the later option.
>
> Ok, so at the end the discussion is to avoid to have a second recipe.
> This is fine with me if it does not break any use case. This was not
> with Anatolji's patches, they are breaking some use cases of mine.

Okay

>> [snip]
>>
>>> Ok, I am quite sure now that the big issue on your use case is that
>>> filesystems on empty partitions are not generated when SWU is built.
>>
>> No, my issue is the needed extra recipes.
>
> Ok, understood.
>
>> WIC could create images with renamed deployed artifacts on the fly and
>> I could build arbitrary images.
>
> I know they can build artifacts on the fly, my intention is to have a
> SWU that is not aware of all about what is deployed.

What do you mean by this? Do you mean the license install example?

> Anyway, I get your goal and if we find a suiztable way where the two
> ways can coexist, it is fine.

Okay. This should be possible as the two could be different classes.

Regards
Stefan

[Stefano Babic]

Hi Stefan,

No, it is not - yes, an empty fs + rawcopy will work, too.

>
>
>>> In any case you could override the common configuration and use image
>>> specific configurations for specific images.
>>
>> That's right, but there are always a 1:1 relationship with a rootfs /
>> image (or how we name it). But I do not see how to build a single WIC
>> where each partitition is generated with a different recipe, and I put
>> each image in a different partition. Just as simple case: two images
>> with OS and application in separate partitions.
>
> This is possible. You could use rawcopy if you already have a recipe for
> the application image or you could install the application in a specific
> directory and use --exclude-path and --change-directory to split the
> rootfs into two partitions.

Ok, nice

>
>
>> On the other side, I do not see how generated a wic from SWU can help.
>> A SWU is understood just from SWUpdate, it is not a filesystem (then
>> putting in a partition makes less sense).
>
> What do you mean?

I guess I have mixed up, ignore this sentence...

>
>
>>>> I am quite losing which are the goals here or what we want to
>>>> address. I try to sumarize, maybe I misunderstand something:
>>>>
>>>> - we need currently a recipe per image in case of different rootfs,
>>>> and adding a .swu fstype lets to avoid the recipe. However,
>>>> sw-description cannot be generated as I said previously, it can be
>>>> factorized among images.
>>>
>>> A swu fstype allows a generic default sw-description per distro or
>>> machine and support an arbitrary rootfs.
>>
>> Yes, but this contradicts your first statement, that an image is just
>> a set of artifacts to be deplowed. A SWU fstype means a rootfs.
>
> You are right. It must be deployed artifacts and packages.

Ok

>
>
>>>> - there are some cases not covered with current generation, related
>>>> to filesystem generation. I am not sure I have understood this.
>>>> SWUpdate delivers artifacts. If a filesystem must be created, the
>>>> device can generally do this itself, first eample at first boot
>>>> after update. Or this can be executed inside the update process as
>>>> part of a preinstall part. Why this should be done when SWU is
>>>> generated, is something I am missing: if the filesystem contains
>>>> files, well, then it is an "image" and can be built in OE, and then
>>>> added to the list of artifacts.
>>>
>>> Yes, but you need a recipe per image.
>>
>> That's right - but I am not understanding why this is a problem when I
>> can get as benefit much flexibility.
>
> Because everybody have to pay the price even if he doesn't need the
> flexibility. The complexity should scale with the needed flexibility.

That's right - we need a way to have both without one of them that
forbids the other one.

>
>>
>>> The WIC image can do this without a special recipe per image /
>>> partition for arbitrary deploy artifacts.
>>
>> How ? It is not clear to me, maybe I haven't solved this. Let's say I
>> need to create several filesystem for a device, and then I put all
>> together with a WIC. But the WIC is specified for each filesystem,
>> that is the deployable of each recipe having a "inherit image". How
>> can I get a WIC to deploy all images at once ? Is it possible ? (I am
>> not aware of it).
>
> What is the content of the filesystem? You could use rawcopy for
> filesystem images (created by other image recipes),

mmmhh...yes, --source rawcopy --sourceparams=... should be enough for that.

> bootimg-partition to
> create a filesystem from deployed images and split the rootfs into
> different partitions. It is possible to create partitions for boot
> loader, kernel, recovery, system and data without any additional recipe.
>
>
>>> An common example is the boot partition or a partition with a
>>> filesystem and the recovery FIT image.
>
> This is possible. We use a dual copy for boot loader, firmware,
> recovery, kernel  and system. We create a wic image without any
> additional recipe.

Ok

>
>
>> IMHO they are not so good examples - the boot (EFI) partition cannot
>> be any kind of generated image, that means I cannot create an image as
>> usual, set the filesystem to VFAT and put it as first partition via WIC.
>
> Again this should be possible with rawcopy and an fstype class for vfat.
> But why do you need a special recipe for this image?

No, I wanted just to find examples.

> bootimg-partition
> could create a vfat partition for you and copy and rename images from
> deploy into the filesystem.

Yes, that is fine.

>
>
>> WIC generates the partition following a schema, and this matches for
>> most cases (but not in all of them).
>
> What case isn't supported by wic?

Well, the most frequent use case for me to use wic is to deploy a first
image or if the device boots from a SD device. In all cases, where the
boot device is different from the main storage (SPI vs eMMC or NAND, for
example), it is useless.

+1

We can maybe factorize some part (common functions), but without
dependencies that block one usage or the other one.

Best regards,