When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.
This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:
- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar
Testing:
- Basic smoke tests performed successfully (citest.py -t fast)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations
Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])
Changes since v2 patch:
- rootfs_install_pkgs_download will no longer use sudo to run
apt-get install --download-only. This was added to further
demonstrate/test rootfs_cmd in existing Isar code.
Changes since v1 patch:
- Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
- Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes
Changes since RFC patch:
- Let caller decide where to bind-mount the rootfs to
- Make the rootfs argument optional
- Support 32-bit rootfs (no lib64 there)
(Re-)validated with "citest.py -t fast" (using kas-container 4.8.1):
JOB ID : 2724be97c6711e046fbc2169823c293dc99cd97c
JOB LOG : avocado/job-results/job-2025-06-25T15.51-2724be9/job.log
(01/19) citest.py:DevTest.test_dev: STARTED
(01/19) citest.py:DevTest.test_dev: PASS (1573.34 s)
(02/19) citest.py:DevTest.test_dev_apps: STARTED
(02/19) citest.py:DevTest.test_dev_apps: PASS (2158.85 s)
(03/19) citest.py:DevTest.test_dev_rebuild: STARTED
(03/19) citest.py:DevTest.test_dev_rebuild: PASS (349.73 s)
(04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
(04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (77.79 s)
(05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
(05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (55.12 s)
(06/19) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
(06/19) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (58.94 s)
(07/19) citest.py:CrossTest.test_cross: STARTED
(07/19) citest.py:CrossTest.test_cross: PASS (1912.25 s)
(08/19) citest.py:CrossTest.test_cross_debsrc: STARTED
(08/19) citest.py:CrossTest.test_cross_debsrc: PASS (2933.62 s)
(09/19) citest.py:CrossTest.test_cross_kselftest: STARTED
(09/19) citest.py:CrossTest.test_cross_kselftest: PASS (2024.26 s)
(10/19) citest.py:CrossTest.test_cross_rpi: STARTED
(10/19) citest.py:CrossTest.test_cross_rpi: PASS (1543.77 s)
(11/19) citest.py:VmBootTestFast.test_arm_bullseye: STARTED
(11/19) citest.py:VmBootTestFast.test_arm_bullseye: PASS (64.33 s)
(12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED
(12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: PASS (12.72 s)
(13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED
(13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: PASS (10.18 s)
(14/19) citest.py:VmBootTestFast.test_arm_buster: STARTED
(14/19) citest.py:VmBootTestFast.test_arm_buster: PASS (57.01 s)
(15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED
(15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: PASS (9.73 s)
(16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED
(16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: PASS (10.39 s)
(17/19) citest.py:VmBootTestFast.test_arm_bookworm: STARTED
(17/19) citest.py:VmBootTestFast.test_arm_bookworm: PASS (82.93 s)
(18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED
(18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: PASS (30.40 s)
(19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED
(19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: PASS (11.59 s)
RESULTS : PASS 19 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 13003.86 s
Cedric Hombourger (5):
rootfs: introduce wrapper to run commands against a rootfs
deb-dl-dir: optimize caching of source packages using apt natively
image-postproc-extension: refactor systemd version checks
image-postproc-extension: extract systemd's version using rootfs_cmd
bootstrap: create lock for downloads/deb without sudo
rootfs: do not get elevated privileges when downloading packages
RECIPE-API-CHANGELOG.md | 7 ++
doc/user_manual.md | 1 +
meta/classes/deb-dl-dir.bbclass | 58 ++++++-------
meta/classes/image-postproc-extension.bbclass | 12 +--
meta/classes/rootfs.bbclass | 83 ++++++++++++++++++-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 4 +
6 files changed, 126 insertions(+), 39 deletions(-)
--
2.39.5