[PATCH 4/4] mmdebstrap: lock isar-apt while bootstrapping

[Andreas Naumann]

isar-apt must not change while we bootstrap from it. Otherwise, e.g.
if a dpkg recipe deploys a deb-file into it, the Releases or Packages
file may not match the hash which causes apt to stop with an error.

In order to use the bitbake lockfile flag on the entire task, which
is preferred to a flock-wrap around the mmdebstrap command, move code
with conflicting locking to a new finalize task.

Signed-off-by: Andreas Naumann <anau...@emlix.com>
---
meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index b03ae7d772..885963fcc4 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -86,6 +86,7 @@ do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}"
DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock"

do_bootstrap[vardeps] += "DISTRO_BOOTSTRAP_BASE_PACKAGES"
+do_bootstrap[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
do_bootstrap() {
if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then
if [ -z "${COMPAT_DISTRO_ARCH}" ]; then
@@ -217,7 +218,10 @@ do_bootstrap() {
"${@get_distro_suite(d)}" \
"${WORKDIR}/rootfs.tar.zst" \
"$bootstrap_list"
+}
+addtask bootstrap before do_build after do_generate_keyrings

+do_bootstrap_finalize() {
# Finalize bootstrap by setting the link in deploy
sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"

@@ -226,9 +230,9 @@ do_bootstrap() {
sudo rm -rf --one-file-system "${WORKDIR}/dl_dir"
fi
}
-addtask bootstrap before do_build after do_generate_keyrings
+addtask do_bootstrap_finalize after do_bootstrap before do_build

-SSTATETASKS += "do_bootstrap"
+SSTATETASKS += "do_bootstrap do_bootstrap_finalize"
SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"

--
2.43.0

[Andreas Naumann]

We have a use case where the bootstrapping takes place using the
contents of the internal isar-apt repo only and not depend on an
external distro repo. (This does not exclude depending on the
underlying fileserver).

In order to achieve that, an number of dpkg-prebuilt packages
are created and deployed to isar-apt before bootstrapping. (out of
scope for this patchset).

The patchset here is to introduce an option which allows for isar-apt
to be used instead of a remote distro source repo.

Notes:
ISAR_USE_CACHED_BASE_REPO was considered but unfortunately is not
exactly what's needed.


Andreas Naumann (4):
mmdebstrap: Make use of defined variables
mmdebstrap: Include isar-apt in bootstrapping sources
mmdebstrap: Allow for DISTRO_SOURCE to be empty
mmdebstrap: lock isar-apt while bootstrapping

meta/classes/bootstrap.bbclass | 2 +
.../isar-mmdebstrap/isar-mmdebstrap.inc | 37 +++++++++++++------
2 files changed, 28 insertions(+), 11 deletions(-)

--
2.43.0

[Zhihang Wei]

Hello,
when testing the patch set on CI, the following test failed:
- citest.py:DevTest.test_dev:  FAIL: Bitbake failed (805.34 s)

Error log follows:
[stdlog] 2025-10-21 09:05:25,524 avocado.app cibuilder L0322 ERROR|
ERROR: Task
(mc:qemuarm-bookworm:virtual:sdk:/build/isar_wzh_3_fast/10/meta-isar/recipes-core/images/isar-image-base.bb:do_rootfs_postprocess)
failed with exit code '1'
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Executing python function do_rootfs_postprocess
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Executing shell function rootfs_do_mounts
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Shell function rootfs_do_mounts finished
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Executing shell function rootfs_do_qemu
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Shell function rootfs_do_qemu finished
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
DEBUG: Executing shell function sdkchroot_configscript
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Get:1 file:/isar-apt isar InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:2 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:1 file:/isar-apt isar InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:3 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar Release
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Get:4 file:/isar-apt isar Release [5,060 B]
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Get:4 file:/isar-apt isar Release [5,060 B]
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:6 http://ftp.de.debian.org/debian bookworm InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:9 http://ftp.de.debian.org/debian-security bookworm-security InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Get:10 file:/isar-apt isar Release.gpg
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:10 file:/isar-apt isar Release.gpg
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:12 http://ftp.de.debian.org/debian bookworm-updates InRelease
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,904 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:8 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main amd64 Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:11 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:13 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main
Translation-en_US.UTF-8
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:14 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main Translation-en
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Err:5 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main armhf Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | | 
 Failed to stat - stat (2: No such file or directory)
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:7 copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm isar/main all Packages
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:6 http://ftp.de.debian.org/debian bookworm InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:9 http://ftp.de.debian.org/debian-security bookworm-security InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:12 http://ftp.de.debian.org/debian bookworm-updates InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:6 http://ftp.de.debian.org/debian bookworm InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:9 http://ftp.de.debian.org/debian-security bookworm-security InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Ign:12 http://ftp.de.debian.org/debian bookworm-updates InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Err:6 http://ftp.de.debian.org/debian bookworm InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | | 
 Temporary failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Err:9 http://ftp.de.debian.org/debian-security bookworm-security InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | | 
 Temporary failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Err:12 http://ftp.de.debian.org/debian bookworm-updates InRelease
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | | 
 Temporary failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
Reading package lists...
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
W: Failed to fetch
http://ftp.de.debian.org/debian/dists/bookworm/InRelease Temporary
failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
W: Failed to fetch
http://ftp.de.debian.org/debian-security/dists/bookworm-security/InRelease
Temporary failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
W: Failed to fetch
http://ftp.de.debian.org/debian/dists/bookworm-updates/InRelease
Temporary failure resolving 'ftp.de.debian.org'
[stdlog] 2025-10-21 09:06:33,905 avocado.test cibuilder L0320 INFO | |
E: Failed to fetch
copy:/tmp/isar-aptJk76Vv7pNq/debian-bookworm/dists/isar/main/binary-armhf/Packages
Failed to stat - stat (2: No such file or directory)

You can redo the test on your machine using avocado:
1. Have a clean clone of isar, checkout to branch next and apply your
patches:
$ git clone -b next https://github.com/ilbers/isar.git
$ cd isar
$ git am /path-to/0001-my-contribution-to-isar.patch
2.Run kas shell, setup CI prerequisites (avocado, qemu) and cleanup:
$ ./kas/kas-container shell kas/isar.yaml --command \
    "rm -rf /work/build/conf && /work/scripts/ci_setup.sh"
3.Run the failed test:
$ cd /work/testsuite
$ avocado run citest.py:DevTest.test_dev$

Best regards,
Zhihang