[PATCH v5 0/3] Deploy DTBs with separate recipe

[Anton Mikanovich]

Image task "do_copy_boot_files" is used to deploy kernel, initrd and
devicetree binaries to deploy directory.

When different images for the same target are built (e.g., "-base" and
"-debug") in parallel, this causes issues with DTB deployment since
they have no any image_name-specific stuff in the path, unlike kernel
and initrd.

Here we make a separate recipe responsible for the DTB deployment.
DTBs are extracted from the kernel package (provided by distro or
self-built).

The recipe 'dtb-files' is MACHINE-dependent, so different DTB_FILES
may be deployed for different machines. At the same time, all these
'dtb-files-${MACHINE}' targets depend on the same kernel package, so
it is built only once.

The case when different distros (e.g., bullseye/bookworm) are using the
same name for DTB binaries is covered by the patch 1. It also covers
the case when image uses different kernel suffix (e.g., "realtime"
option for the kernel in some downstreams).

Related topics on maillist:

https://groups.google.com/g/isar-users/c/ht3YmKZcJlM
https://groups.google.com/g/isar-users/c/4BRDM02xC40
https://groups.google.com/g/isar-users/c/qAnjahjjfsw
https://groups.google.com/g/isar-users/c/ZMD4XY4dKWQ
https://groups.google.com/g/isar-users/c/PSGU_AcdPZ8
https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA
https://groups.google.com/g/isar-users/c/ixzytHYnTmI

Changes since v4:
- Rebase on next.
- Squash latest patch to first one.
- Fix deploy paths in initrd tests.

Changes since v3:
- Rebase on next.
- Fix target deploy dir path for installer.

Changes since v2:
- Patch 1: Consider also kernel suffix.
- Patch 2: Use separate testcase for possible dtb clashes.
- Minor cosmetic fixes.

Changes since v1:
- Separate `dtb-files` recipe used instead of linux one
- Included older "[PATCH] Fix do_copy_boot_files error" into the
series ( https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA )

Anton Mikanovich (1):
testsuite: Add test to check dtb deployment

Ilia Skochilov (1):
meta: Fix do_copy_boot_files error for different distros of same
machine

Uladzimir Bely (1):
meta: Move DTB deployment to a separate dtb-files recipe

RECIPE-API-CHANGELOG.md | 15 +++++++
.../installer-add-rootfs.bbclass | 4 +-
meta/classes-recipe/image.bbclass | 15 +++----
meta/conf/bitbake.conf | 2 +-
meta/recipes-kernel/dtb-files/dtb-files.bb | 41 +++++++++++++++++++
testsuite/citest.py | 30 ++++++++++++--
6 files changed, 92 insertions(+), 15 deletions(-)
create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

--
2.43.0

[Anton Mikanovich]

From: Ilia Skochilov <iskoc...@ilbers.de>

When building different distros with the same machine (e.g.,
phyboard-mira-bullseye and phyboard-mira-bookworm) it happens that
some files with the same name (e.g, DTB files) are deployed
to the same location and this causes build error.

Use DISTRO- and KERNEL- dependent deploy directory.

Also fix paths in installer and testsuite.

Signed-off-by: Ilia Skochilov <iskoc...@ilbers.de>
Signed-off-by: Uladzimir Bely <ub...@ilbers.de>
Signed-off-by: Badrikesh Prusty <badrikes...@siemens.com>
Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---
RECIPE-API-CHANGELOG.md | 15 +++++++++++++++
.../classes-recipe/installer-add-rootfs.bbclass | 4 +++-
meta/conf/bitbake.conf | 2 +-
testsuite/citest.py | 7 ++++---
4 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 0bad8a44..6d7f12b2 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -962,3 +962,18 @@ INSTALLER_UNATTENDED_ABORT_ENABLE = "1"
# Optional: set countdown timeout in seconds (default 5)
INSTALLER_UNATTENDED_ABORT_TIMEOUT = "5"
```
+
+### Change DEPLOY_DIR_IMAGE path
+
+Change DEPLOY_DIR_IMAGE from ${DEPLOY_DIR}/images/${MACHINE} to
+${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}-${KERNEL_NAME}. When building
+different distros with the same machine the following error occurs:
+
+do_copy_boot_files: The recipe isar-image-base is trying to install files
+into a shared area when those files already exists. It happens when some
+files have the same names (e.g., dtb files) for different distros.
+
+To prevent this new path is separated also by distro and kernel values.
+
+This change will influence on build artifacts location and should be taken
+into account by downstreams.
diff --git a/meta-isar/classes-recipe/installer-add-rootfs.bbclass b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
index 2fa551bd..b79c2357 100644
--- a/meta-isar/classes-recipe/installer-add-rootfs.bbclass
+++ b/meta-isar/classes-recipe/installer-add-rootfs.bbclass
@@ -13,7 +13,9 @@ INSTALLER_TARGET_IMAGES ??= "${INSTALLER_TARGET_IMAGE}"
INSTALLER_TARGET_MC ??= "installer-target"
INSTALLER_TARGET_DISTRO ??= "${DISTRO}"
INSTALLER_TARGET_MACHINE ??= "${MACHINE}"
-INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}"
+INSTALLER_TARGET_KERNEL_NAME ??= "${KERNEL_NAME}"
+INSTALLER_TARGET_DEPLOY_DIR_POSTFIX ??= "${@ ('-'+d.getVar('INSTALLER_TARGET_KERNEL_NAME')) if d.getVar('INSTALLER_TARGET_KERNEL_NAME') else ''}"
+INSTALLER_TARGET_DEPLOY_DIR_IMAGE ??= "${DEPLOY_DIR}/images/${INSTALLER_TARGET_MACHINE}-${INSTALLER_TARGET_DISTRO}${INSTALLER_TARGET_DEPLOY_DIR_POSTFIX}"

IMAGE_DATA_FILE ??= "${INSTALLER_TARGET_IMAGE}-${INSTALLER_TARGET_DISTRO}-${INSTALLER_TARGET_MACHINE}"
IMAGE_DATA_POSTFIX ??= "wic.zst"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0f84e715..ad3d6b53 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -57,7 +57,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
-DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
+DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}-${DISTRO}${@ ('-'+d.getVar('KERNEL_NAME')) if d.getVar('KERNEL_NAME') else ''}"
DL_DIR ?= "${TOPDIR}/downloads"
SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${DISTRO}-${DISTRO_ARCH}"
diff --git a/testsuite/citest.py b/testsuite/citest.py
index f11cd5af..8047c42c 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -496,8 +496,9 @@ class InitRdBaseTest(CIBaseTest):
super().init()
self.deploy_dir = os.path.join(self.build_dir, 'tmp', 'deploy')

- def deploy_dir_image(self, machine):
- return os.path.join(self.deploy_dir, 'images', machine)
+ def deploy_dir_image(self, mc, image):
+ multiconfig = f"{mc}:{image}"
+ return CIUtils.getVars('DEPLOY_DIR_IMAGE', target=multiconfig)

def dracut_in_image(self, targets):
machine = 'qemuamd64'
@@ -526,7 +527,7 @@ class InitRdBaseTest(CIBaseTest):
bb_should_fail=False):
mc = f'mc:{machine}-{distro.removeprefix("debian-")}'
initrd_image = f'{initrd}-{distro}-{machine}-initrd.img'
- initrd_path = os.path.join(self.deploy_dir_image(machine), initrd_image)
+ initrd_path = os.path.join(self.deploy_dir_image(mc, initrd), initrd_image)

# cleansstate if the initrd image was already built/deployed to verify
# that a new build does result in the image being deployed
--
2.43.0

[Anton Mikanovich]

Build several images in parallel to catch all possible conflicts
between deployed DTB files:
- Same distro/machine, different images.
- Same machine, different distros.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

testsuite/citest.py | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index 8047c42c..bed0505b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -678,6 +678,29 @@ class WicTest(CIBaseTest):
)


+class DtbDeployTest(CIBaseTest):
+
+ """
+ Build some targets potentially causing DTB deploy issues.
+ Cover cases:
+ - Same machine, different distros
+ - Same machine/distro, different images
+
+ :avocado: tags=dtbdeploy,full
+ """
+
+ def test_dtb_deploy(self):
+ targets = [
+ 'mc:phyboard-mira-bullseye:isar-image-base',
+ 'mc:phyboard-mira-bullseye:isar-image-ci',
+ 'mc:phyboard-mira-bookworm:isar-image-base',
+ 'mc:phyboard-mira-bookworm:isar-image-ci',
+ ]
+
+ self.init()
+ self.perform_build_test(targets)
+
+
class NoCrossTest(CIBaseTest):

"""
--
2.43.0

[Anton Mikanovich]

From: Uladzimir Bely <ub...@ilbers.de>

Task do_copy_boot_files was deploying DTB files into the same location
for different images (e.g. -base or -debug). This causes build issue.

Introduce `dtb-files` recipe responsible for the extraction of DTBs
from linux-image package (distro or self-built) and their deployment.

Different images for the same machine now depend on this recipe and
shouldn't cause parallel execution issues.

Signed-off-by: Uladzimir Bely <ub...@ilbers.de>

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

meta/classes-recipe/image.bbclass | 15 +++-----
meta/recipes-kernel/dtb-files/dtb-files.bb | 41 ++++++++++++++++++++++
2 files changed, 46 insertions(+), 10 deletions(-)
create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index e605bc80..dc599535 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -372,6 +372,11 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"
# only one dtb file supported, pick the first
DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"

+python() {
+ if d.getVar('DTB_FILES'):
+ d.appendVarFlag("do_copy_boot_files", "depends", "dtb-files-${MACHINE}:do_deploy")
+}
+
do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
do_copy_boot_files[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
@@ -385,16 +390,6 @@ do_copy_boot_files() {
sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
fi

- for file in ${DTB_FILES}; do
- dtb="$(find '${IMAGE_ROOTFS}/usr/lib' -type f \
- -iwholename '*linux-image-*/'${file} | head -1)"
-
- if [ -z "$dtb" -o ! -e "$dtb" ]; then
- die "${file} not found"
- fi
-
- cp -f "$dtb" "${DEPLOYDIR}/"
- done
}
addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install

diff --git a/meta/recipes-kernel/dtb-files/dtb-files.bb b/meta/recipes-kernel/dtb-files/dtb-files.bb
new file mode 100644
index 00000000..f64b4845
--- /dev/null
+++ b/meta/recipes-kernel/dtb-files/dtb-files.bb
@@ -0,0 +1,41 @@
+# This software is a part of Isar.
+# Copyright (C) ilbers GmbH, 2025
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg-raw
+
+PN:append = "-${MACHINE}"
+
+KERNEL_IMAGE_PKG ??= "${@ ("linux-image-" + d.getVar("KERNEL_NAME")) if d.getVar("KERNEL_NAME") else ""}"
+
+DEPENDS = "${KERNEL_IMAGE_PKG}"
+DEBIAN_BUILD_DEPENDS = "${KERNEL_IMAGE_PKG}"
+
+DPKG_ARCH = "${PACKAGE_ARCH}"
+
+do_prepare_build:append() {
+ for dtb in ${DTB_FILES}; do
+ dir=debian/${PN}/usr/lib/${PN}/$(dirname ${dtb})
+ cat <<EOF >> ${S}/debian/rules
+ mkdir -p ${dir}
+ find /usr/lib/linux-image* -path "*${dtb}" -print -exec cp {} ${dir} \;
+EOF
+ done
+}
+
+DTB_PACKAGE ??= "${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb"
+
+do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
+do_deploy[cleandirs] = "${WORKDIR}/deploy"
+do_deploy() {
+ dpkg --fsys-tarfile ${WORKDIR}/${DTB_PACKAGE} | \
+ tar --wildcards --extract --directory ${WORKDIR}/deploy ./usr/lib/${PN}
+ for dtb in ${DTB_FILES}; do
+ mkdir -p ${DEPLOY_DIR_IMAGE}/$(dirname ${dtb})
+ find ${WORKDIR}/deploy/usr/lib/${PN} -path "*${dtb}" -print \
+ -exec cp {} ${DEPLOY_DIR_IMAGE}/${dtb} \;
+ done
+}
+
+addtask deploy before do_deploy_deb after do_dpkg_build
--
2.43.0

[Jan Kiszka]

On 06.02.26 13:27, Anton Mikanovich wrote:
> From: Uladzimir Bely <ub...@ilbers.de>
>
> Task do_copy_boot_files was deploying DTB files into the same location
> for different images (e.g. -base or -debug). This causes build issue.
>

But this still doesn't justify this redundant extraction. Adjust the
deployment paths and be done.

Jan

Siemens AG, Foundational Technologies
Linux Expert Center

[Zhihang Wei]

Hi, from CI side there is at least one confirmed error:
In full, "citest.py:NoCrossTest.test_nocross" failed when building target
'mc:qemuamd64-iso-bookworm:isar-image-ci'.

Log follows:
ERROR: mc:qemuamd64-iso-bookworm:isar-image-ci-1.0-r0 do_image_wic:
ExecutionError('/work/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64-iso/1.0-r0/temp/run.do_image_wic.94122',
1, None, None)
ERROR: Logfile of failure stored in:
/work/build/tmp/work/debian-bookworm-amd64/isar-image-ci-qemuamd64-iso/1.0-r0/temp/log.do_image_wic.94122
ERROR: Task
(mc:qemuamd64-iso-bookworm:/work/build/../meta-test/recipes-core/images/isar-image-ci.bb:do_image_wic)
failed with exit code '1'

And log.do_image_wic.94122:
......
Setting up grub-common (2.06-13+deb12u1) ...
Setting up grub-efi-amd64-bin (2.06-13+deb12u1) ...
Processing triggers for libc-bin (2.36-9+deb12u13) ...
Processing triggers for man-db (2.11.2-2) ...
INFO: Creating image(s)...

ERROR: _exec_cmd: cd
/tmp/isar-image-ci-debian-bookworm-qemuamd64-iso.wic/tmp.wic.27jkr5qc/INITRD
&& find . | cpio -o -H newc -R root:root
>/tmp/isar-image-ci-debian-bookworm-qemuamd64-iso.wic/tmp.wic.27jkr5qc/initrd.cpio
returned '127' instead of 0
output: /bin/sh: 1: cpio: not found

To redo the test using avocado:
1. Have a clean clone of isar, checkout to branch next and apply your
patches:
$ git clone -b next https://github.com/ilbers/isar.git
$ cd isar
$ git am /path-to/0001-my-contribution-to-isar.patch
2. Disable unrelated targets to make error appears faster, by applying
the following diff to "testsuite/citest.py":

diff --git a/testsuite/citest.py b/testsuite/citest.py
index bed0505b..8586f76b 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -711,36 +711,7 @@ class NoCrossTest(CIBaseTest):

     def test_nocross(self):
         targets = [
-            'mc:qemuarm-buster:isar-image-ci',
-            'mc:qemuarm-bullseye:isar-image-base',
-            'mc:qemuarm64-bullseye:isar-image-base',
-            'mc:qemuarm64-bookworm:isar-image-ci',
-            'mc:qemui386-buster:isar-image-base',
-            'mc:qemui386-bullseye:isar-image-base',
-            'mc:qemuamd64-buster:isar-image-ci',
-            'mc:qemuamd64-bullseye:isar-initramfs',
-            'mc:qemumipsel-bullseye:isar-image-base',
-            'mc:imx6-sabrelite-bullseye:isar-image-base',
-            'mc:phyboard-mira-bullseye:isar-image-base',
-            'mc:hikey-bullseye:isar-image-base',
-            'mc:virtualbox-bullseye:isar-image-base',
-            'mc:virtualbox-bookworm:isar-image-base',
-            'mc:bananapi-bullseye:isar-image-base',
-            'mc:bananapi-bookworm:isar-image-base',
-            'mc:nanopi-neo-bullseye:isar-image-base',
-            'mc:nanopi-neo-bookworm:isar-image-base',
-            'mc:qemuamd64-focal:isar-image-ci',
-            'mc:qemuamd64-bookworm:isar-image-ci',
             'mc:qemuamd64-iso-bookworm:isar-image-ci',
-            'mc:qemui386-bookworm:isar-image-base',
-            'mc:qemumipsel-bookworm:isar-image-ci',
-            'mc:hikey-bookworm:isar-image-base',
-            'mc:beagleplay-bookworm:isar-image-base',
-            'mc:qemuarm64-noble:isar-image-base',
-            'mc:qemuamd64-noble:isar-image-base',
-            'mc:qemuamd64-jammy:isar-image-base',
-            'mc:qemuarm64-jammy:isar-image-base',
-            'mc:x86-pc-bookworm:isar-image-base',
         ]

         self.init()

3. Run kas shell, setup CI prerequisites (avocado, qemu) and cleanup:
$ ./kas/kas-container shell kas/isar.yaml --command \
"rm -rf /work/build/conf && /work/scripts/ci_setup.sh"
4. Run the failed test:
$ cd /work/testsuite
$ avocado run citest.py:NoCrossTest.test_nocross$
(took ~6 minutes on my laptop for this error to show up)

There are other errors but I believe they are caused by network
failures. I'm re-testing them
now. Otherwise I'll let you know.

Zhihang

[Baurzhan Ismagulov]

On 2026-02-06 14:39, 'Jan Kiszka' via isar-users wrote:
> > Task do_copy_boot_files was deploying DTB files into the same location
> > for different images (e.g. -base or -debug). This causes build issue.
>
> But this still doesn't justify this redundant extraction. Adjust the
> deployment paths and be done.

Currently, the deployment is done in the image. Two images deploying to the
same dtb path cause conflicts (with sstate-cache, IIRC). Detaching the
deployment from the image and attaching it to the kernel is addressing this.
We'll split the testcases to demonstrate this issue.

With kind regards,
Baurzhan

[Jan Kiszka]

This is not the problem of the image recipes deploying, it's a problem
of the deploying to the same path. That's why I doubt this solution.

Jan

[Anton Mikanovich]

The current implementation tries to obtain machine name from the deploy
directory name, which relies on hardcoded path format. Add MACHINE to
the WIC environment and read it in isoimage-isohybrid-isar.py to fix
that.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

meta/classes-recipe/imagetypes_wic.bbclass | 2 +-
.../scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py | 4 +++-
2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 6b82add3..d50022ce 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -107,7 +107,7 @@ WICVARS += "\
ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"

# Isar specific vars used in our plugins
-WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE"
+WICVARS += "DISTRO DISTRO_ARCH KERNEL_FILE MACHINE"

python do_rootfs_wicenv () {
wicvars = d.getVar('WICVARS')
diff --git a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
index fb6c6b39..0ed61fff 100644
--- a/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/isoimage-isohybrid-isar.py
@@ -191,7 +191,9 @@ class IsoImagePlugin(SourcePlugin):
if not image_type:
raise WicError("Couldn't find INITRAMFS_FSTYPES, exiting.")

- machine = os.path.basename(initrd_dir)
+ machine = get_bitbake_var("MACHINE")
+ if not machine:
+ raise WicError("Couldn't find MACHINE, exiting.")

pattern = '%s/%s*%s.%s' % (initrd_dir, image_name, machine, image_type)
pattern = '%s/%s-%s-initrd.%s' % (initrd_dir, image_name, machine, image_type)
--
2.34.1

[Anton Mikanovich]

Image task "do_copy_boot_files" is used to deploy kernel, initrd and
devicetree binaries to deploy directory.

When different images for the same target are built (e.g., "-base" and
"-debug") in parallel, this causes issues with DTB deployment since
they have no any image_name-specific stuff in the path, unlike kernel
and initrd.

Here we make a separate recipe responsible for the DTB deployment.
DTBs are extracted from the kernel package (provided by distro or
self-built).

The recipe 'dtb-files' is MACHINE-dependent, so different DTB_FILES
may be deployed for different machines. At the same time, all these
'dtb-files-${MACHINE}' targets depend on the same kernel package, so
it is built only once.

The case when different distros (e.g., bullseye/bookworm) are using the

same name for DTB binaries is covered by the patch 3. It also covers

the case when image uses different kernel suffix (e.g., "realtime"
option for the kernel in some downstreams).

Related topics on maillist:

https://groups.google.com/g/isar-users/c/ht3YmKZcJlM
https://groups.google.com/g/isar-users/c/4BRDM02xC40
https://groups.google.com/g/isar-users/c/qAnjahjjfsw
https://groups.google.com/g/isar-users/c/ZMD4XY4dKWQ
https://groups.google.com/g/isar-users/c/PSGU_AcdPZ8
https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA
https://groups.google.com/g/isar-users/c/ixzytHYnTmI

Changes since v6:
- Fix isoimage-isohybrid-isar WIC plugin.
- Split testcases.
- Update wording.

Changes since v4:
- Rebase on next.
- Squash latest patch to first one.
- Fix deploy paths in initrd tests.

Changes since v3:
- Rebase on next.
- Fix target deploy dir path for installer.

Changes since v2:
- Patch 1: Consider also kernel suffix.
- Patch 2: Use separate testcase for possible dtb clashes.
- Minor cosmetic fixes.

Changes since v1:
- Separate `dtb-files` recipe used instead of linux one
- Included older "[PATCH] Fix do_copy_boot_files error" into the
series ( https://groups.google.com/g/isar-users/c/Va0Ue-ISYeA )

Anton Mikanovich (2):
wic: Obtain real machine name in isoimage source plugin
testsuite: Add testcases to check dtb deployment

Ilia Skochilov (1):
meta: Fix do_copy_boot_files error for different distros of same
machine

Uladzimir Bely (1):

meta: Deploy DTBs once per kernel

RECIPE-API-CHANGELOG.md | 15 +++++++
.../installer-add-rootfs.bbclass | 4 +-
meta/classes-recipe/image.bbclass | 15 +++----

meta/classes-recipe/imagetypes_wic.bbclass | 2 +-

meta/conf/bitbake.conf | 2 +-
meta/recipes-kernel/dtb-files/dtb-files.bb | 41 +++++++++++++++++++

.../plugins/source/isoimage-isohybrid-isar.py | 4 +-
testsuite/citest.py | 40 ++++++++++++++++--
8 files changed, 106 insertions(+), 17 deletions(-)
create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

--
2.34.1

[Anton Mikanovich]

Build several images with dtbs deployed to the same path.

- test_dtb_deploy_distros

Same machine, different distros, same image, same kernel, same dtb
name and contents.

- test_dtb_deploy_images

Same machine, same distro, different images, same kernel, same dtb
name and contents.

Both fail with:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
(not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

testsuite/citest.py | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

diff --git a/testsuite/citest.py b/testsuite/citest.py
index f11cd5af..c521f6fe 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py
@@ -677,6 +677,39 @@ class WicTest(CIBaseTest):

)


+class DtbDeployTest(CIBaseTest):
+
+ """
+ Build some targets potentially causing DTB deploy issues.
+

+ :avocado: tags=dtbdeploy,full
+ """
+

+ def test_dtb_deploy_distros(self):
+ """
+ Cover case: Same machine, different distros
+ """

+ targets = [
+ 'mc:phyboard-mira-bullseye:isar-image-base',

+ 'mc:phyboard-mira-bookworm:isar-image-base',

+ ]
+
+ self.init()
+ self.perform_build_test(targets)
+

+ def test_dtb_deploy_images(self):
+ """
+ Cover case: Same machine/distro, different images
+ """
+ targets = [

+ 'mc:phyboard-mira-bookworm:isar-image-base',
+ 'mc:phyboard-mira-bookworm:isar-image-ci',
+ ]
+
+ self.init()
+ self.perform_build_test(targets)
+
+
class NoCrossTest(CIBaseTest):

"""
--

2.34.1

[Anton Mikanovich]

From: Ilia Skochilov <iskoc...@ilbers.de>

When building different distros for the same machine (e.g.,
phyboard-mira-bullseye and phyboard-mira-bookworm), files with the same
name (e.g, DTB files) might be deployed to the same location and this
causes a build error.

Use DISTRO- and KERNEL-dependent deploy directory.

Also fix paths in installer and testsuite.

Fixes test_dtb_deploy_distros testcase:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed
build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
(not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Ilia Skochilov <iskoc...@ilbers.de>
Signed-off-by: Uladzimir Bely <ub...@ilbers.de>
Signed-off-by: Badrikesh Prusty <badrikes...@siemens.com>

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

diff --git a/testsuite/citest.py b/testsuite/citest.py
index c521f6fe..072d8e92 100755
--- a/testsuite/citest.py
+++ b/testsuite/citest.py

@@ -496,8 +496,9 @@ class InitRdBaseTest(CIBaseTest):
super().init()
self.deploy_dir = os.path.join(self.build_dir, 'tmp', 'deploy')

- def deploy_dir_image(self, machine):
- return os.path.join(self.deploy_dir, 'images', machine)
+ def deploy_dir_image(self, mc, image):
+ multiconfig = f"{mc}:{image}"
+ return CIUtils.getVars('DEPLOY_DIR_IMAGE', target=multiconfig)

def dracut_in_image(self, targets):
machine = 'qemuamd64'
@@ -526,7 +527,7 @@ class InitRdBaseTest(CIBaseTest):
bb_should_fail=False):
mc = f'mc:{machine}-{distro.removeprefix("debian-")}'
initrd_image = f'{initrd}-{distro}-{machine}-initrd.img'
- initrd_path = os.path.join(self.deploy_dir_image(machine), initrd_image)
+ initrd_path = os.path.join(self.deploy_dir_image(mc, initrd), initrd_image)

# cleansstate if the initrd image was already built/deployed to verify
# that a new build does result in the image being deployed
--

2.34.1

[Anton Mikanovich]

From: Uladzimir Bely <ub...@ilbers.de>

Task do_copy_boot_files deploys DTB files into the same location for
different images (e.g., isar-image-base and isar-image-ci). This causes
a build error.

do_copy_boot_files is called once for every image recipe while dtb files
belong to the kernel which is the same for both images. Performing dtb
deployment once for the same kernel solves the issue.

Introduce `dtb-files` recipe responsible for extracting the DTBs from
the linux-image package and deploying them. The kernel package is now
installed into sbuild chroot. This is achieved via the respective
dependency of `dtb-files` recipe.

Fixes test_dtb_deploy_images testcase:

ERROR: mc:phyboard-mira-bookworm:isar-image-base-1.0-r0 do_copy_boot_files: The recipe isar-image-base is trying to install files into a shared area when those files already exist. Those files and their manifest location are:

build/tmp/deploy/images/phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb
(not matched to any task)
Please verify which recipe should provide the above files.

Signed-off-by: Uladzimir Bely <ub...@ilbers.de>

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

meta/classes-recipe/image.bbclass | 15 +++-----
meta/recipes-kernel/dtb-files/dtb-files.bb | 41 ++++++++++++++++++++++

2 files changed, 46 insertions(+), 10 deletions(-)
create mode 100644 meta/recipes-kernel/dtb-files/dtb-files.bb

diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index afe6906e..06b72b09 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -381,6 +381,11 @@ INITRD_IMG = "${PP_DEPLOY}/${INITRD_DEPLOY_FILE}"

# only one dtb file supported, pick the first
DTB_IMG = "${PP_DEPLOY}/${@(d.getVar('DTB_FILES').split() or [''])[0]}"

+python() {
+ if d.getVar('DTB_FILES'):
+ d.appendVarFlag("do_copy_boot_files", "depends", "dtb-files-${MACHINE}:do_deploy")
+}
+
do_copy_boot_files[cleandirs] += "${DEPLOYDIR}"
do_copy_boot_files[sstate-inputdirs] = "${DEPLOYDIR}"
do_copy_boot_files[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"

@@ -394,16 +399,6 @@ do_copy_boot_files() {

sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
fi

- for file in ${DTB_FILES}; do
- dtb="$(find '${IMAGE_ROOTFS}/usr/lib' -type f \
- -iwholename '*linux-image-*/'${file} | head -1)"
-
- if [ -z "$dtb" -o ! -e "$dtb" ]; then
- die "${file} not found"
- fi
-
- cp -f "$dtb" "${DEPLOYDIR}/"
- done
}
addtask copy_boot_files before do_rootfs_postprocess after do_rootfs_install

diff --git a/meta/recipes-kernel/dtb-files/dtb-files.bb b/meta/recipes-kernel/dtb-files/dtb-files.bb
new file mode 100644

index 00000000..6f1ef146

--- /dev/null
+++ b/meta/recipes-kernel/dtb-files/dtb-files.bb
@@ -0,0 +1,41 @@
+# This software is a part of Isar.

+# Copyright (C) ilbers GmbH, 2026

2.34.1

[Jan Kiszka]

On 13.02.26 08:40, Anton Mikanovich wrote:
> From: Uladzimir Bely <ub...@ilbers.de>
>
> Task do_copy_boot_files deploys DTB files into the same location for
> different images (e.g., isar-image-base and isar-image-ci). This causes
> a build error.
>
> do_copy_boot_files is called once for every image recipe while dtb files
> belong to the kernel which is the same for both images. Performing dtb
> deployment once for the same kernel solves the issue.
>

This remains inconsistent: DTBs are just another artifact of the kernel.
The kernel image itself is also deployed via the image recipe. Why is
there no conflict is because of image-specific file names being used. We
can't change the DTB file names, but we could put them in image-specific
deployment folders, for the case that multiple images are pushed into
the same deployment folder.

Again: NAK, wrong approach, please finally process my feedback.

[Baurzhan Ismagulov]

On 2026-02-13 09:01, 'Jan Kiszka' via isar-users wrote:
> > Task do_copy_boot_files deploys DTB files into the same location for
> > different images (e.g., isar-image-base and isar-image-ci). This causes
> > a build error.
> >
> > do_copy_boot_files is called once for every image recipe while dtb files
> > belong to the kernel which is the same for both images. Performing dtb
> > deployment once for the same kernel solves the issue.
>
> This remains inconsistent: DTBs are just another artifact of the kernel.
> The kernel image itself is also deployed via the image recipe. Why is
> there no conflict is because of image-specific file names being used. We
> can't change the DTB file names, but we could put them in image-specific
> deployment folders, for the case that multiple images are pushed into
> the same deployment folder.
>
> Again: NAK, wrong approach, please finally process my feedback.

v6 has been sent for documenting the reasons in the archive as previously
announced; I think we can solve this without escalating.

Now that you've shared your proposal of splitting by image -- that's an option
and we can discuss it.

Our use case is:
1. Two images, same dtb name, same dtb contents.
2. Downstream already has many deliverables.

My underlying question: Should we always duplicate, or should we at least try
to identify cases where we can avoid duplication? "Same file -- single
deployment".

Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
producing it, reducing shuffling around in the image recipe. Yes, the kernel is
still deployed in the image which is inconsistent at this step; I'd be fine
doing that later.

We'll try playing with splitting by image. Even if we end up doing it at this
time, I'd still propose to explore the other options step by step.

With kind regards,
Baurzhan

[Jan Kiszka]

On 16.02.26 18:15, Baurzhan Ismagulov wrote:
> On 2026-02-13 09:01, 'Jan Kiszka' via isar-users wrote:
>>> Task do_copy_boot_files deploys DTB files into the same location for
>>> different images (e.g., isar-image-base and isar-image-ci). This causes
>>> a build error.
>>>
>>> do_copy_boot_files is called once for every image recipe while dtb files
>>> belong to the kernel which is the same for both images. Performing dtb
>>> deployment once for the same kernel solves the issue.
>>
>> This remains inconsistent: DTBs are just another artifact of the kernel.
>> The kernel image itself is also deployed via the image recipe. Why is
>> there no conflict is because of image-specific file names being used. We
>> can't change the DTB file names, but we could put them in image-specific
>> deployment folders, for the case that multiple images are pushed into
>> the same deployment folder.
>>
>> Again: NAK, wrong approach, please finally process my feedback.
>
> v6 has been sent for documenting the reasons in the archive as previously
> announced; I think we can solve this without escalating.
>
> Now that you've shared your proposal of splitting by image -- that's an option
> and we can discuss it.
>
> Our use case is:
> 1. Two images, same dtb name, same dtb contents.

Which is just the tip of this issue. Other layers deploy bootloader
files or firmware binaries under the same name, overwriting things right
now if two images are built under different names. And that overwriting
sometimes triggers the infamous bitbake deployment errors.

> 2. Downstream already has many deliverables.
>
> My underlying question: Should we always duplicate, or should we at least try
> to identify cases where we can avoid duplication? "Same file -- single
> deployment".

How would you switch between de-duplicated and different artifacts by
name then? How would imagers or other users find deployed artifacts in
both cases?

>
> Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
> producing it, reducing shuffling around in the image recipe. Yes, the kernel is
> still deployed in the image which is inconsistent at this step; I'd be fine
> doing that later.
>
> We'll try playing with splitting by image. Even if we end up doing it at this
> time, I'd still propose to explore the other options step by step.

Please not so many user-visible changes "step by step". We need a design
that holds now, and we need one step to get there, at least when that
step implies path changes that downstream layers need to adjust to.

I guess we should rather introduce the pattern of creating a subfolder
${IMAGE_FULLNAME} and put all those potentially colliding artifacts there.

[Baurzhan Ismagulov]

On 2026-02-16 18:51, 'Jan Kiszka' via isar-users wrote:
> > Our use case is:
> > 1. Two images, same dtb name, same dtb contents.
>
> Which is just the tip of this issue. Other layers deploy bootloader
> files or firmware binaries under the same name, overwriting things right
> now if two images are built under different names. And that overwriting
> sometimes triggers the infamous bitbake deployment errors.

Thanks for the detailed elaboration. If the contents are the same, v6 patch 4
would handle it. The "same name, different contents" case we couldn't identify
from the past threads and the downstreams we see, so we had left it for later
when it would actually occur (often the real cases are different from what one
anticipates).

If "same name, different contents" is already the case today, duplicating is
the only option.

I'm a bit surprised that the errors occur only sometimes -- we can reproduce
them reliably given the right mix of build targets.

> > 2. Downstream already has many deliverables.
> >
> > My underlying question: Should we always duplicate, or should we at least try
> > to identify cases where we can avoid duplication? "Same file -- single
> > deployment".
>
> How would you switch between de-duplicated and different artifacts by
> name then? How would imagers or other users find deployed artifacts in
> both cases?

Yes, that would depend on the contents and might be counter-intuitive at times.
Maybe duplicating and having the complete sets for every machine would be the
easiest to understand for the end users. We need to see how it works in
practice.

> > Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
> > producing it, reducing shuffling around in the image recipe. Yes, the kernel is
> > still deployed in the image which is inconsistent at this step; I'd be fine
> > doing that later.
> >
> > We'll try playing with splitting by image. Even if we end up doing it at this
> > time, I'd still propose to explore the other options step by step.
>
> Please not so many user-visible changes "step by step". We need a design
> that holds now, and we need one step to get there, at least when that
> step implies path changes that downstream layers need to adjust to.

I agree -- this was the goal of v6 patch 4 -- it doesn't change the path if it
is not necessary, and a similar kernel consolidation also wouldn't change the
path.

> I guess we should rather introduce the pattern of creating a subfolder
> ${IMAGE_FULLNAME} and put all those potentially colliding artifacts there.

We'll send v7 with duplication.


Currently, some artifacts are duplicated via file names. Since we already break
the API, we could touch them as well. Opinions welcome.

1. Before the change:

mc:phyboard-mira-bookworm:isar-image-base

phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb # bitbake conflict
phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img
phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz

mc:phyboard-mira-bookworm:isar-image-ci

phyboard-mira/imx6q-phytec-mira-rdk-nand.dtb # bitbake conflict
phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img
phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz

2. After the duplication e.g.:

phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb # Same contents in our testcase
phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb # Same contents in our testcase
phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-initrd.img # Currently not in the subdir
phyboard-mira/isar-image-base-debian-bookworm-phyboard-mira-vmlinuz # Currently not in the subdir
phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-initrd.img # Currently not in the subdir
phyboard-mira/isar-image-ci-debian-bookworm-phyboard-mira-vmlinuz # Currently not in the subdir

Or we could move the kernels and initrds into the subdir as well:

phyboard-mira/debian-bookworm-isar-image-base/imx6q-phytec-mira-rdk-nand.dtb
phyboard-mira/debian-bookworm-isar-image-base/initrd.img
phyboard-mira/debian-bookworm-isar-image-base/vmlinuz
phyboard-mira/debian-bookworm-isar-image-ci/imx6q-phytec-mira-rdk-nand.dtb
phyboard-mira/debian-bookworm-isar-image-ci/initrd.img
phyboard-mira/debian-bookworm-isar-image-ci/vmlinuz


With kind regards,
Baurzhan

[Jan Kiszka]

On 18.02.26 17:21, Baurzhan Ismagulov wrote:
> On 2026-02-16 18:51, 'Jan Kiszka' via isar-users wrote:
>>> Our use case is:
>>> 1. Two images, same dtb name, same dtb contents.
>>
>> Which is just the tip of this issue. Other layers deploy bootloader
>> files or firmware binaries under the same name, overwriting things right
>> now if two images are built under different names. And that overwriting
>> sometimes triggers the infamous bitbake deployment errors.
>
> Thanks for the detailed elaboration. If the contents are the same, v6 patch 4
> would handle it. The "same name, different contents" case we couldn't identify
> from the past threads and the downstreams we see, so we had left it for later
> when it would actually occur (often the real cases are different from what one
> anticipates).
>
> If "same name, different contents" is already the case today, duplicating is
> the only option.
>
> I'm a bit surprised that the errors occur only sometimes -- we can reproduce
> them reliably given the right mix of build targets.
>

The massive scale of multiconfig targets that isar has is not
representative. I know the error primarily from reconfigured rebuilds.

>
>>> 2. Downstream already has many deliverables.
>>>
>>> My underlying question: Should we always duplicate, or should we at least try
>>> to identify cases where we can avoid duplication? "Same file -- single
>>> deployment".
>>
>> How would you switch between de-duplicated and different artifacts by
>> name then? How would imagers or other users find deployed artifacts in
>> both cases?
>
> Yes, that would depend on the contents and might be counter-intuitive at times.
> Maybe duplicating and having the complete sets for every machine would be the
> easiest to understand for the end users. We need to see how it works in
> practice.
>
>
>>> Patch 4 "deploy dtb once per kernel" associates the dtb with the recipe
>>> producing it, reducing shuffling around in the image recipe. Yes, the kernel is
>>> still deployed in the image which is inconsistent at this step; I'd be fine
>>> doing that later.
>>>
>>> We'll try playing with splitting by image. Even if we end up doing it at this
>>> time, I'd still propose to explore the other options step by step.
>>
>> Please not so many user-visible changes "step by step". We need a design
>> that holds now, and we need one step to get there, at least when that
>> step implies path changes that downstream layers need to adjust to.
>
> I agree -- this was the goal of v6 patch 4 -- it doesn't change the path if it
> is not necessary, and a similar kernel consolidation also wouldn't change the
> path.

Having inconsistent paths would be highly counterintuitive. Better have
stable ones and live with duplications. We may already duplicate kernels
and initramfs images, and both are much larger some normal DTBs.

Moving kernel and initramfs would make things more consistent, indeed.

Only thing to double-check first would be if we are already completely
off compared to how OE deploys or if we can follow a pattern there. If
are off, I would go for this consistent relocation. If not, we should
check if (re-)aligning would both solve our issues and make things
simpler for people coming from that world.