[PATCH 2/2] sbuild: do not build arch all packages on cross

[Felix Moessbauer]

Architecture all packages often cannot be built in cross mode, as the
dependencies cannot be resolved in the host architecture. This
especially applies to all packages which split their build dependencies
into Build-Depends-Arch and Build-Depends-Indep. We already have logic
in isar to built DPKG_ARCH=all packages non-cross, but this does not
work for packages that generate both arch specific and arch all binary
packages.

While sbuild automatically disables builds of arch-all packages on
cross, we previously explicitly overrode this setting. We now change
this by explicitly setting --no-arch-all flag on cross and --arch-all on
non-cross.

Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
RECIPE-API-CHANGELOG.md | 15 +++++++++++++++
meta/classes-recipe/dpkg.bbclass | 3 ++-
2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 856da5de..c83c1599 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1001,3 +1001,18 @@ do_copy_boot_files: The recipe isar-image-base is trying to install
files into a shared area when those files already exists. It happens
when some files have the same names (e.g., dtb files) for different
distros.
+
+### No building of arch=all packages when cross building
+
+Architecture all packages often cannot be built in cross mode, as the
+dependencies cannot be resolved in the foreign architecture. This especially
+applies to all packages which split their build dependencies into
+`Build-Depends-Arch` and `Build-Depends-Indep`. We already have logic
+in isar to built `DPKG_ARCH = "all"` packages non-cross, but this does not
+work for packages that generate both arch specific and arch all binary packages.
+
+We now explicitly disable building arch=all binary packages on cross (this
+technically only affects packages that generate arch specific and arch=all
+binary packages). To build the arch=all binary packages of such source packages,
+add an explicit dependency to `${BPN}-native`. Source packages that only
+generate `arch=all` packages, must now be marked `DPKG_ARCH = "all"`.
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index c1c38184..500aaefe 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -109,8 +109,9 @@ dpkg_runbuild() {

DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)

- sbuild -A -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
+ sbuild -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
--host=${PACKAGE_ARCH} --build=${BUILD_ARCH} ${profiles} \
+ ${@'--no-arch-all' if 'cross' in isar_deb_build_profiles(d).split() else '--arch-all'} \
--no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
--bd-uninstallable-explainer=apt \
--no-apt-update --apt-distupgrade \
--
2.53.0

[Jan Kiszka]

That will be fun to find, though likely a rare one. Can't we detect such
cases in Isar and add a bitbake dependency from $BPN to -native?

Jan

> +generate `arch=all` packages, must now be marked `DPKG_ARCH = "all"`.
> diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
> index c1c38184..500aaefe 100644
> --- a/meta/classes-recipe/dpkg.bbclass
> +++ b/meta/classes-recipe/dpkg.bbclass
> @@ -109,8 +109,9 @@ dpkg_runbuild() {
>
> DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
>
> - sbuild -A -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
> + sbuild -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
> --host=${PACKAGE_ARCH} --build=${BUILD_ARCH} ${profiles} \
> + ${@'--no-arch-all' if 'cross' in isar_deb_build_profiles(d).split() else '--arch-all'} \
> --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
> --bd-uninstallable-explainer=apt \
> --no-apt-update --apt-distupgrade \

--

Siemens AG, Foundational Technologies
Linux Expert Center

[Jan Kiszka]

In fact, -native is practically overshooting. We would need a recipe
target that only build the all package.

Jan

[MOESSBAUER, Felix]

We could try to parse the ${S}/debian/control file and detect this, but
I guess that doesn't work, as we would have to add the dependency
before running bitbake tasks (a common case is that the debian part is
fetched, hence the task need to run and then it's too late).

> >
>
> In fact, -native is practically overshooting. We would need a recipe
> target that only build the all package.

I thought about this as well. We would need a -all variant that is
basically identical to -native but adds the --no-arch-any sbuild flag.
However, this only helps for first-stage dependencies, as the
transitive dependencies still need to be build in -native. Further, we
need to ensure that either -all or -native is built, but not both. This
would need a model similar to the fetch_common_source task. That's a
lot of complexity for a minimal build-time improvement. By that, I
rather would like to avoid this optimization. If we really need it, we
still can add it in the future.

Felix

[Jan Kiszka]

The key question is how to ensure that people will find this highly
non-intuitive recipe dependency. We need to explain this even harder
than the difference between IMAGE_INSTALL and IMAGE_PREINSTALL.

BTW, it also affects how recipe writers have to translate their binary
output packages into PROVIDES statements. If you do that incorrectly,
unconditionally, the mess will only get bigger.

[MOESSBAUER, Felix]

That's an interesting finding. If you model the PROVIDES correctly, you
don't have to add the explicit RDEPENDS to the -native variant. This is
what we should document instead of the -native dependency.

Will do so in a v2, once I have more feedback.

Felix

[Jan Kiszka]

Better test it - I bet there will be further devils in the detail.

> Will do so in a v2, once I have more feedback.
>

I'm afraid we are not stable with this approach yet.

[MOESSBAUER, Felix]

While adding a test, I noticed that this does not work: When building
the -all packages only from the native recipe, these are only added to
the HOST_ARCH repository, but not to the DISTRO_ARCH repo. By that,
they cannot be installed into the image.

>
> > Will do so in a v2, once I have more feedback.
> >
>
> I'm afraid we are not stable with this approach yet.

Yes, this needs more redesign. Probably we need a :class-all that then
deploys to its own repo which is consumed both by the host repo and the
distro repo.

Felix