Cross-build issues over bookworm host for non-bookworm targets

[Jan Kiszka]

Hi all,

the kas-isar:next build container is now based on bookworm, and that
seems to reveal some issues around cross-building over that host. I've
not reproduced that with two layers, meta-iot2050 and isar-cip-core. The
latter builds fine natively, e.g. qemu-amd64, but it fails once
switching to qemu-arm64 e.g. BUT: bookworm as target is fine again.

Some exemplary splat:

ERROR: Logfile of failure stored in: /build/tmp/work/cip-core-bullseye-arm64/sshd-regen-keys/0.4-r0/temp/log.do_dpkg_build.14055
Log data follows:
| DEBUG: Executing python function sstate_task_prefunc
| DEBUG: Python function sstate_task_prefunc finished
| DEBUG: Executing python function do_dpkg_build
| DEBUG: Executing shell function schroot_create_configs
| DEBUG: Shell function schroot_create_configs finished
| DEBUG: Executing shell function dpkg_runbuild
| sbuild (Debian sbuild) 0.85.0 (04 January 2023) on 76d4480b6118
|
| +==============================================================================+
| | sshd-regen-keys 0.4 (arm64) Fri, 02 Jun 2023 08:27:29 +0000 |
| +==============================================================================+
|
| Package: sshd-regen-keys
| Version: 0.4
| Source Version: 0.4
| Distribution: isar
| Machine Architecture: amd64
| Host Architecture: arm64
| Build Architecture: amd64
| Build Profiles: cross
| Build Type: binary
|
| I: NOTICE: Log filtering will replace 'var/run/schroot/mount/isar-builder-80876582-28a6-4cef-89cf-0c1aacac137d-14055-caaa44a8-48f9-4b91-bf10-b777cbca86a3' with '<<CHROOT>>'
|
| +------------------------------------------------------------------------------+
| | Chroot Setup Commands |
| +------------------------------------------------------------------------------+
|
|
| echo "Package: *\nPin: release n=isar\nPin-Priority: 1000" > /etc/apt/preferences.d/isar-apt
| --------------------------------------------------------------------------------------------
|
|
| I: Finished running 'echo "Package: *\nPin: release n=isar\nPin-Priority: 1000" > /etc/apt/preferences.d/isar-apt'.
|
| echo "APT::Get::allow-downgrades 1;" > /etc/apt/apt.conf.d/50isar-apt
| ---------------------------------------------------------------------
|
|
| I: Finished running 'echo "APT::Get::allow-downgrades 1;" > /etc/apt/apt.conf.d/50isar-apt'.
|
| rm -f /var/log/dpkg.log
| -----------------------
|
|
| I: Finished running 'rm -f /var/log/dpkg.log'.
|
| ln -sf /home/builder/sshd-regen-keys/rootfs/var/cache/apt/archives/*.deb -t /var/cache/apt/archives/ || :
| ---------------------------------------------------------------------------------------------------------
|
|
| I: Finished running 'ln -sf /home/builder/sshd-regen-keys/rootfs/var/cache/apt/archives/*.deb -t /var/cache/apt/archives/ || :'.
|
| Finished processing commands.
| --------------------------------------------------------------------------------
| I: NOTICE: Log filtering will replace 'build/sshd-regen-keys-AuXR83/resolver-Ms1W52' with '<<RESOLVERDIR>>'
|
| +------------------------------------------------------------------------------+
| | Update chroot |
| +------------------------------------------------------------------------------+
|
| Reading package lists...
| Building dependency tree...
| Reading state information...
| Calculating upgrade...
| 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
|
| +------------------------------------------------------------------------------+
| | Fetch source files |
| +------------------------------------------------------------------------------+
|
|
| Local sources
| -------------
|
| /build/tmp/work/cip-core-bullseye-arm64/sshd-regen-keys/0.4-r0/sshd-regen-keys_0.4.dsc exists in /build/tmp/work/cip-core-bullseye-arm64/sshd-regen-keys/0.4-r0; copying to chroot
| I: NOTICE: Log filtering will replace 'build/sshd-regen-keys-AuXR83/sshd-regen-keys-0.4' with '<<PKGBUILDDIR>>'
| I: NOTICE: Log filtering will replace 'build/sshd-regen-keys-AuXR83' with '<<BUILDDIR>>'
|
| +------------------------------------------------------------------------------+
| | Install package build dependencies |
| +------------------------------------------------------------------------------+
|
|
| Setup apt archive
| -----------------
|
| Merged Build-Depends: debhelper (>= 10), build-essential:amd64, fakeroot:amd64, crossbuild-essential-arm64:amd64, libc-dev:arm64, libstdc++-dev:arm64
| Filtered Build-Depends: debhelper (>= 10), build-essential:amd64, fakeroot:amd64, crossbuild-essential-arm64:amd64, libc-dev:arm64, libstdc++-dev:arm64
| dpkg-deb: building package 'sbuild-build-depends-main-dummy' in '/<<RESOLVERDIR>>/apt_archive/sbuild-build-depends-main-dummy.deb'.
| Ign:1 copy:/<<RESOLVERDIR>>/apt_archive ./ InRelease
| Get:2 copy:/<<RESOLVERDIR>>/apt_archive ./ Release [609 B]
| Ign:3 copy:/<<RESOLVERDIR>>/apt_archive ./ Release.gpg
| Get:4 copy:/<<RESOLVERDIR>>/apt_archive ./ Sources [683 B]
| Get:5 copy:/<<RESOLVERDIR>>/apt_archive ./ Packages [726 B]
| Fetched 2018 B in 0s (0 B/s)
| Reading package lists...
| Get:1 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar InRelease
| Ign:1 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar InRelease
| Get:2 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar Release [4994 B]
| Get:2 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar Release [4994 B]
| Get:3 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar Release.gpg
| Ign:3 file:/home/builder/sshd-regen-keys/isar-apt/cip-core-bullseye-arm64/apt/cip-core-bullseye isar Release.gpg
| Reading package lists...
| Reading package lists...
|
| Install main build dependencies (apt-based resolver)
| ----------------------------------------------------
|
| Installing build dependencies
| Reading package lists...
| Building dependency tree...
| Reading state information...
| /usr/lib/apt/solvers/apt: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/lib/apt/solvers/apt)
| Execute external solver...
| E: Write error - write (32: Broken pipe)
| E: Write error - write (32: Broken pipe)
| E: Sub-process sbuild-cross-resolver received signal 13.
| apt-get failed.
| E: Package installation failed
| Not removing build depends: cloned chroot in use


"/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found", that looks suspicious.

Are we leaking something from the host into the sbuild env that falls
apart here? Are we missing a dependency for the sbuild env? Or is there
a fundamental incompatibility in bookworm?

Jan

--
Siemens AG, Technology
Competence Center Embedded Linux

[Uladzimir Bely]

> > I: NOTICE: Log filtering will replace 'build/sshd-regen-keys-
> > AuXR83/resolver-Ms1W52' with '<<RESOLVERDIR>>'
> >
> > +------------------------------------------------------------------
> > ------------+
> > > Update
> > > chroot                                                           
> > >      |
> > +------------------------------------------------------------------

Adding "apt-utils" to SBUILD_CHROOT_PREINSTALL_COMMON in
`meta/recipes-devtools/sbuild-chroot` seems to solve the issue.

This package provides `/usr/lib/apt/solvers/apt` in sbuild-chroot.

[Jan Kiszka]

Can you also explain why adding that resolves the specific error above?
So far, apt-utils was not needed. Is some script different that comes
with the host sbuild package? I'm still concerned that some binary of
the host leaks and would like to see that ruled out.

[Uladzimir Bely]

I've looked through the issue and that's what I found.

It seems to be related to new apt solver they use for cross-building
(addded in https://salsa.debian.org/debian/sbuild/-/commit/696e963).

For older sbuild (e.g., bullseye-based sbuild-chroot) we have the only
`dump` solver belonging to apt package:

```
builder@c8c24e2a0203:/build$ ls -1 /usr/lib/apt/solvers/
dump
builder@c8c24e2a0203:/build$ dpkg -S /usr/lib/apt/solvers/*
apt: /usr/lib/apt/solvers/dump
```

With newer sbuild (e.g., for ookworm-based sbuild-chroot) we have three
solvers availabler:

```
builder@834442924aa6:/build$ dpkg -S /usr/lib/apt/solvers/*
apt-utils: /usr/lib/apt/solvers/apt
apt: /usr/lib/apt/solvers/dump
sbuild: /usr/lib/apt/solvers/sbuild-cross-resolver
```

The last one is enabled when cross-building. And since new sbuild has a
strict dependency on apt-utils (ia "sbuild => libsbuild-perl => apt-
utils" chain; in bullseye it was just "recommended"), it makes us add
apt-utils package inside cross-build environment.

Uladzimir

[Jan Kiszka]

OK, sounds consistent to me, thanks. Will you send a patch?