[PATCH] image: allow multiple image recipes to deploy DTB_FILES

[Cedric Hombourger]

sstate is checking for overlapping files in DEPLOY_DIR and will
raise an error when building a second image for a machine that
uses DTB_FILES.

Reproducer:
bitbake mc:phyboard-mira-bookworm:isar-image-base
bitbake mc:phyboard-mira-bookworm:isar-image-debug

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
meta/classes/image.bbclass | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index c29d9e26..8b316c83 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -27,7 +27,11 @@ INITRD_IMAGE ?= ""
INITRD_DEPLOY_FILE = "${@ d.getVar('INITRD_IMAGE') or '${IMAGE_FULLNAME}-initrd.img'}"

# This defines the deployed dtbs for reuse by imagers
+# Since we may be building several images with the same set of DTB_FILES, silent sstate
+# overlap checks
DTB_FILES ?= ""
+DEPLOY_DTB_FILES = "${@ ' '.join(['${DEPLOY_DIR_IMAGE}/' + os.path.basename(dtb) for dtb in d.getVar('DTB_FILES').split()]) }"
+SSTATE_ALLOW_OVERLAP_FILES += "${DEPLOY_DTB_FILES}"

# Useful variables for imager implementations:
PP = "/home/builder/${PN}-${MACHINE}"
--
2.39.2

[Jan Kiszka]

There is still
https://patchwork.isar-build.org/project/isar/list/?series=1209 pending
which looks much less risky to paper over a real conflict.

Jan

--
Siemens AG, Technology
Linux Expert Center

[cedric.h...@siemens.com]

Thanks Jan. I had missed this RFC patch series.

That said, while it will solve the problem we are having with building
multiple images with MACHINEs using DTB_FILES, it does not seem to
solve the problem of a build where we are building multiple kernels
(e.g. -rt and non-rt) for a given MACHINE: both kernels will be
shipping the same DTB_FILES.

On our side, we happen to have that use-case on our side as well.

Cedric

>
> Jan
>

--
Cedric Hombourger
Siemens AG
http://www.siemens.com/

[Uladzimir Bely]

On Fri, 2024-07-05 at 07:47 +0000, 'cedric.h...@siemens.com' via

Hello all.

Yes, patchset v2 (non-RFC) for this is still in "work-in-progress"
state. There are some issues we faced with:
- Parallell building issue. If some machines of same arch use the same
linux recipe, but have different "DTB_FILES" value, we have sstate-
related conflict for the linux recipe ("multiple execution
detected..."). This is currently solved by splitting workdirs by adding
"-${MACHINE}" prefix here.
- Since "linux-distro.bb" now inherits dpkg-raw bbclass, it builds
something. And in case of empty DTB_FILES it looks a bit useless (we
build empty deb in fact).

>
> >
> > Jan
> >
>
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/
>

--
Best regards,
Uladzimir.

[Jan Kiszka]

Adding Uladzimir: The risk of SSTATE_ALLOW_OVERLAP_FILES, also with your
scenario, remains that those kernels are not coming with identical
sources and, thus, with potentially different DTBs.

Jan

>
> On our side, we happen to have that use-case on our side as well.
>
> Cedric
>
>>
>> Jan
>>
>
> --
> Cedric Hombourger
> Siemens AG
> http://www.siemens.com/

[cedric.h...@siemens.com]

Yes I cannot agree more. We probably need to look at the consumers of
${DEPLOY_DIR_IMAGE}/*.dtb.

For instance, if the kernel was to produce a package with dtb files,
that package could be added to the list of packages to be installed in
the IMAGER sbuild environment

I am happy to look into this.

Once concern though is that the tree is currently broken for the
multiple-image / multiple-kernel use-cases we talked about.

If we agree that the deployment of dtb files into DEPLOY_DIR_IMAGE
should go away (and I think it should), I would then believe that a
work-around such as proposed herein would be acceptable (as long as it
does not stay in the tree for too long).

If we cannot fix or work-around this issue soon then I am afraid that
we will see all those work-arounds in downstream layers. Thoughts?