[RFC PATCH 1/2] rootfs: introduce wrapper to run native commands against a rootfs

[Cedric Hombourger]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a native command
could be used without elevated privileges as long as special folders
such as /isar-apt are mounted (they are often referenced as /isar-apt
in configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
to achieve better performance when execution through QEMU may be
avoided.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
meta/classes/rootfs.bbclass | 64 +++++++++++++++++++++++++++++++++++++
1 file changed, 64 insertions(+)

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index f16ecc00..2975eb6b 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -30,6 +30,70 @@ export LANG = "C"
export LANGUAGE = "C"
export LC_ALL = "C"

+# Execute a native command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}).
+#
+# Usage: rootfs_native_cmd [options] -- command
+#
+rootfs_native_cmd() {
+ set -- "$@"
+ bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
+ rootfs=""
+
+ while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+ case "${1}" in
+ --bind)
+ if [ "${#}" -lt "3" ]; then
+ bbfatal "--bind requires two arguments"
+ fi
+ bwrap_args="${bwrap_args} --bind ${2} ${3}"
+ shift 3
+ ;;
+ --chdir)
+ if [ "${#}" -lt "2" ]; then
+ bbfatal "${1} requires an argument"
+ fi
+ bwrap_args="${bwrap_args} ${1} ${2}"
+ shift 2
+ ;;
+ -*)
+ bbfatal "${1} is not a supported option!"
+ ;;
+ *)
+ if [ -z "${rootfs}" ]; then
+ rootfs="${1}"
+ shift
+ else
+ bbfatal "unexpected argument '${1}'"
+ fi
+ ;;
+ esac
+ done
+
+ [ -n "${rootfs}" ] || bbfatal "no rootfs path provided"
+
+ if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+ bbfatal "no command specified (missing --)"
+ fi
+
+ shift # remove the "--"
+ exec bwrap \
+ ${bwrap_args} \
+ --bind "${rootfs}" "${rootfs}" \
+ --unshare-user \
+ --unshare-pid \
+ --dev-bind /dev /dev --proc /proc --ro-bind /sys /sys \
+ --ro-bind /etc /etc --ro-bind /bin /bin \
+ --ro-bind /lib /lib --ro-bind /lib64 /lib64 \
+ --ro-bind /usr /usr --tmpfs /tmp \
+ -- "${@}"
+}
+
rootfs_do_mounts[weight] = "3"
rootfs_do_mounts() {
sudo -s <<'EOSUDO'
--
2.39.5

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/image-postproc-extension.bbclass | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 991bac4c..0af588d8 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -53,12 +53,17 @@ image_postprocess_mark() {
--build-id "${BUILD_ID}" --variant "${DESCRIPTION}" --version "${PV}"
}

+# Use dpkg to find out which version of systemd is installed into the image or reports "0"
+image_systemd_version() {
+ sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+}
+
ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
image_postprocess_machine_id() {
# systemd(1) takes care of recreating the machine-id on first boot
# for systemd < v247, set to empty string, else set to uninitialized
# (required if initramfs with ro root is used)
- SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
+ SYSTEMD_VERSION=$( image_systemd_version )
MACHINE_ID="uninitialized"
if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
MACHINE_ID=""
@@ -82,10 +87,7 @@ image_postprocess_sshd_key_regen() {

ROOTFS_POSTPROCESS_COMMAND =+ "image_posprocess_disable_systemd_firstboot"
image_posprocess_disable_systemd_firstboot() {
- SYSTEMD_VERSION=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
- --showformat='${source:Upstream-Version}' \
- --show systemd || echo "0" )
-
+ SYSTEMD_VERSION=$( image_systemd_version )
if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then
sudo chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
if ! cmd_output=$(sudo chroot '${ROOTFSDIR}' systemd-firstboot \
--
2.39.5

[Cedric Hombourger]

"sudo chroot" is used in several places to run commands inside rootfs

directories constructed by Isar. There are cases where a command could

be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)

where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 6 ++++
doc/user_manual.md | 1 +
meta/classes/rootfs.bbclass | 66 +++++++++++++++++++++++++++++++++++++
3 files changed, 73 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index a4cf1338..725737b2 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be controlled by adding to the

Changes in next
---------------
+
+### Require bubblewrap to run non-privileged commands with bind-mounts
+
+Isar occasionally needs to run commands within root file-systems that it
+builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be
+used in Isar classes instead of `sudo chroot`.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 0dc317c3..3cf1a9aa 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -75,6 +75,7 @@ Install the following packages:
```
apt install \
binfmt-support \
+ bubblewrap \
bzip2 \
mmdebstrap \
arch-test \
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 5f877962..5b96b414 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -34,6 +34,72 @@ export LANG = "C"

export LANGUAGE = "C"
export LC_ALL = "C"

+# Execute a command against a rootfs and with isar-apt bind-mounted.

+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely

+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the
+# optional rootfs argument is omitted, the host rootfs will be used (e.g. to
+# run native commands): this should be used with care.
+#
+# Usage: rootfs_cmd [options] [rootfs] -- command
+#
+rootfs_cmd() {

+ if [ -n "${rootfs}" ]; then
+ bwrap_args="${bwrap_args} --bind ${rootfs} ${rootfs}"
+ fi

+
+ if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+ bbfatal "no command specified (missing --)"
+ fi

+ shift # remove "--", command and its arguments follows
+
+ for ro_d in bin etc lib lib64 sys usr var; do
+ [ -d ${rootfs}/${ro_d} ] || continue
+ bwrap_args="${bwrap_args} --ro-bind ${rootfs}/${ro_d} /${ro_d}"
+ done
+
+ bwrap --unshare-user --unshare-pid ${bwrap_args} \
+ --dev-bind /dev /dev --proc /proc --tmpfs /tmp \

[Cedric Hombourger]

When building root file-systems for a foreign architecture and needing
to cache source packages, apt will be called from within the rootfs and
consequently be executed under QEMU: this is terribly slow especially
considering that source packages are downloaded one by one. This patch
series introduces a wrapper function to run native commands against a
rootfs with our special mounts such as /isar-apt. Some basic tests were
performed to smoke test this approach and evaluate its performance. It
should be noted that the changes introduce a new host tool dependency:
bubblewrap. Alignment with projects such as kas would be required (had
a positive discussion with Felix but no actions will be taken prior to
receiving some form of agreement for this patch series).

Changes since RFC patch:
- Let caller decide where to bind-mount the rootfs to
- Make the rootfs argument optional
- Support 32-bit rootfs (no lib64 there)

Validated with "citest.py -t dev" (in a kas-container):

(1/6) citest.py:DevTest.test_dev: STARTED
(1/6) citest.py:DevTest.test_dev: PASS (752.07 s)
(2/6) citest.py:DevTest.test_dev_apps: STARTED
(2/6) citest.py:DevTest.test_dev_apps: PASS (770.95 s)
(3/6) citest.py:DevTest.test_dev_rebuild: STARTED
(3/6) citest.py:DevTest.test_dev_rebuild: PASS (275.02 s)
(4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
(4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (47.87 s)
(5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
(5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (31.20 s)
(6/6) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
(6/6) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (32.34 s)

Cedric Hombourger (4):
rootfs: introduce wrapper to run commands against a rootfs
deb-dl-dir: optimize caching of source packages using apt natively
image-postproc-extension: refactor systemd version checks
image-postproc-extension: extract systemd's version using rootfs_cmd

RECIPE-API-CHANGELOG.md | 6 ++
doc/user_manual.md | 1 +
meta/classes/deb-dl-dir.bbclass | 37 +++--------
meta/classes/image-postproc-extension.bbclass | 12 ++--
meta/classes/rootfs.bbclass | 66 +++++++++++++++++++
5 files changed, 89 insertions(+), 33 deletions(-)

--
2.39.5

[Cedric Hombourger]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/deb-dl-dir.bbclass | 37 ++++++++-------------------------
1 file changed, 9 insertions(+), 28 deletions(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 3f560da4..7026f4f4 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -5,25 +5,6 @@

inherit repository

-debsrc_do_mounts() {
- sudo -s <<EOSUDO
- set -e
- mkdir -p "${1}/deb-src"
- mountpoint -q "${1}/deb-src" || \
- mount -o bind,private "${DEBSRCDIR}" "${1}/deb-src"
-EOSUDO
-}
-
-debsrc_undo_mounts() {
- sudo -s <<EOSUDO
- set -e
- mkdir -p "${1}/deb-src"
- mountpoint -q "${1}/deb-src" && \
- umount "${1}/deb-src"
- rm -rf "${1}/deb-src"
-EOSUDO
-}
-
debsrc_source_version_filter() {
# Filter the input to only consider Package, Version and Source lines
#
@@ -51,11 +32,6 @@ debsrc_download() {
export rootfs_distro="$2"
mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}"

- debsrc_do_mounts "${rootfs}"
-
- trap 'exit 1' INT HUP QUIT TERM ALRM USR1
- trap 'debsrc_undo_mounts "${rootfs}"' EXIT
-
( flock 9
set -e
printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -89,13 +65,18 @@ debsrc_download() {
dscname="${src}_${version#*:}.dsc"
[ -f "${DEBSRCDIR}"/"${rootfs_distro}"/"${src}"/"${dscname}" ] || {
# use apt-get source to download sources in DEBSRCDIR
- sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
- sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
+ mkdir -p "${DEBSRCDIR}/${rootfs_distro}"/"${src}"
+ rootfs_cmd \
+ --bind "${DEBSRCDIR}" "/deb-src" \

+ --bind "${rootfs}" "${rootfs}" \

+ --chdir "/deb-src/${rootfs_distro}/${src}" \
+ -- \
+ apt-get -o APT::Architecture=${DISTRO_ARCH} \
+ -o Dir="${rootfs}" -y --download-only \
+ --only-source source "${src}=${version}"
}
done
) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
-
- debsrc_undo_mounts "${rootfs}"
}

dbg_pkgs_download() {
--
2.39.5

[Cedric Hombourger]

Elevated privileges are not required to query the rootfs for the version
of systemd: replace "sudo chroot" with "rootfs_cmd"

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/image-postproc-extension.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 0af588d8..21dcfccc 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -55,7 +55,7 @@ image_postprocess_mark() {

# Use dpkg to find out which version of systemd is installed into the image or reports "0"

image_systemd_version() {
- sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+ rootfs_cmd ${IMAGE_ROOTFS} -- dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
}

ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
--
2.39.5

[Srinuvasan Arjunan]

 Tested-by: Srinuvasan Arjunan <srinuv...@siemens.com> 

--
2.39.5

--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-1-cedric.hombourger%40siemens.com.

[MOESSBAUER, Felix]

On Mon, 2025-05-19 at 13:57 +0200, Cedric Hombourger wrote:
> "sudo chroot" is used in several places to run commands inside rootfs
> directories constructed by Isar. There are cases where a command
> could
> be used without elevated privileges as long as special folders such
> as
> /isar-apt are mounted (they are often referenced as /isar-apt in
> configuration files found in the target rootfs). For such cases,
> bubblewrap may be used to create a non-privileged namespace (either
> in a bare/native environment or within a docker/podman container)
> where the command will be executed as if chroot had been used. The
> rootfs may also be the host root file-system: this should however
> be used with care to avoid host contamination problems (note: Isar
> already relies on a number of host tools).

Hi, this looks promising. I gave it a try on some of our internal
layers (arm64) in a custom kas container under podman.

I'm wondering if this could also be used to run the apt in
do_rootfs_install natively (maybe in combination with dpkg --root).

Tested-by: Felix Moessbauer <felix.mo...@siemens.com>

Felix

--
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

[cedric.h...@siemens.com]

On Thu, 2025-05-22 at 14:32 +0000, Moessbauer, Felix (FT RPD CED OES-

DE) wrote:
> On Mon, 2025-05-19 at 13:57 +0200, Cedric Hombourger wrote:
> > "sudo chroot" is used in several places to run commands inside
> > rootfs
> > directories constructed by Isar. There are cases where a command
> > could
> > be used without elevated privileges as long as special folders such
> > as
> > /isar-apt are mounted (they are often referenced as /isar-apt in
> > configuration files found in the target rootfs). For such cases,
> > bubblewrap may be used to create a non-privileged namespace (either
> > in a bare/native environment or within a docker/podman container)
> > where the command will be executed as if chroot had been used. The
> > rootfs may also be the host root file-system: this should however
> > be used with care to avoid host contamination problems (note: Isar
> > already relies on a number of host tools).
>
> Hi, this looks promising. I gave it a try on some of our internal
> layers (arm64) in a custom kas container under podman.
>
> I'm wondering if this could also be used to run the apt in
> do_rootfs_install natively (maybe in combination with dpkg --root).
>
> Tested-by: Felix Moessbauer <felix.mo...@siemens.com>

Dear maintainers, can we move forward with these changes or are there
any concerns that need to be addressed?

Thank you!

Cedric Hombourger
Siemens AG
www.siemens.com

[MOESSBAUER, Felix]

On Thu, 2025-06-05 at 06:42 +0000, Hombourger, Cedric (FT FDS CES LX)
wrote:

If we decide to integrate this (which I vote for!), we should also add
the bubblewrap package to the upcoming kas 4.8 release (putting Jan in
CC).

Felix

[Baurzhan Ismagulov]

On 2025-06-05 12:20, 'MOESSBAUER, Felix' via isar-users wrote:
> If we decide to integrate this (which I vote for!), we should also add
> the bubblewrap package to the upcoming kas 4.8 release (putting Jan in
> CC).

Yes, that was also my question. We'll check the patches once more and provide
feedback. @Felix, would you then like to update kas first?

This would need to be touched when we'll continue working on sudo removal.
@Cedric, could this be meaningfully tested in a testcase?

With kind regards,
Baurzhan

[Jan Kiszka]

Does the bubblewrap (and kernel features) of bullseye suffice here, or
is that a bookworm+ thing? How about buster (still listed as host)?

Jan

--
Siemens AG, Foundational Technologies
Linux Expert Center

[cedric.h...@siemens.com]

bubblewrap has been around for ages: these older distros did support
flatpak. buster included.

https://packages.debian.org/buster/bubblewrap

>
> Jan

[cedric.h...@siemens.com]

this is tested via caching of Debian source packages but also via the
systemd version check. Are you seeking explicit tests for this new
internal API? I am asking as I was under the impression that our tests
focus on blackbox tests and not so much whitebox tests

>
> With kind regards,
> Baurzhan

[Jan Kiszka]

Then I suppose our CI would catch any nasty difference in our usage
compared to those standard use cases, right?

[Cedric Hombourger]

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Testing:
- Basic smoke tests performed successfully (citest.py -t dev)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v1 patch:
- Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
- Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
- Let caller decide where to bind-mount the rootfs to
- Make the rootfs argument optional
- Support 32-bit rootfs (no lib64 there)

(Re-)validated with "citest.py -t dev" (using kas-container 4.8.1):

JOB ID : be45cf0e3937b95d283e7acd687787df259c4341
JOB LOG : job-results/job-2025-06-18T12.43-be45cf0/job.log
(1/6) citest.py:DevTest.test_dev: STARTED
(1/6) citest.py:DevTest.test_dev: PASS (1177.32 s)
(2/6) citest.py:DevTest.test_dev_apps: STARTED
(2/6) citest.py:DevTest.test_dev_apps: PASS (1128.83 s)
(3/6) citest.py:DevTest.test_dev_rebuild: STARTED
(3/6) citest.py:DevTest.test_dev_rebuild: PASS (412.72 s)
(4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
(4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (77.60 s)
(5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
(5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (50.17 s)
(6/6) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
(6/6) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (52.95 s)
RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 2905.62 s

Cedric Hombourger' via isar-users (4):

rootfs: introduce wrapper to run commands against a rootfs
deb-dl-dir: optimize caching of source packages using apt natively
image-postproc-extension: refactor systemd version checks
image-postproc-extension: extract systemd's version using rootfs_cmd

RECIPE-API-CHANGELOG.md | 7 ++

doc/user_manual.md | 1 +
meta/classes/deb-dl-dir.bbclass | 37 +++--------
meta/classes/image-postproc-extension.bbclass | 12 ++--
meta/classes/rootfs.bbclass | 66 +++++++++++++++++++

5 files changed, 90 insertions(+), 33 deletions(-)

[1] https://lists.isar-build.org/isar-users/20250616155748.56164...@siemens.com/T/#u

--
2.39.5

[Cedric Hombourger]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

[Cedric Hombourger]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a command could
be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 7 ++++

doc/user_manual.md | 1 +
meta/classes/rootfs.bbclass | 66 +++++++++++++++++++++++++++++++++++++

3 files changed, 74 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 8468717d..18b90555 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -727,3 +727,10 @@ Changes in next

This was never documented and never had practical relevance. `oci-archive` is
the useful OCI image format that can be imported, e.g., by podman.

+
+### Require bubblewrap to run non-privileged commands with bind-mounts
+
+Isar occasionally needs to run commands within root file-systems that it
+builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be

+used in Isar classes instead of `sudo chroot`. It is pre-installed in
+kas-container version 4.8 (or later).
diff --git a/doc/user_manual.md b/doc/user_manual.md
index ca551a0d..a4fff34a 100644

--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -75,6 +75,7 @@ Install the following packages:
```
apt install \
binfmt-support \
+ bubblewrap \

bzip2 \
mmdebstrap \
arch-test \
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass

index 5f877962..f0c172b8 100644

--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -34,6 +34,72 @@ export LANG = "C"
export LANGUAGE = "C"
export LC_ALL = "C"

+# Execute a command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the
+# optional rootfs argument is omitted, the host rootfs will be used (e.g. to
+# run native commands): this should be used with care.
+#
+# Usage: rootfs_cmd [options] [rootfs] -- command
+#
+rootfs_cmd() {
+ set -- "$@"
+ bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"

+ bwrap_rootfs=""

+
+ while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+ case "${1}" in
+ --bind)
+ if [ "${#}" -lt "3" ]; then
+ bbfatal "--bind requires two arguments"
+ fi
+ bwrap_args="${bwrap_args} --bind ${2} ${3}"
+ shift 3
+ ;;
+ --chdir)
+ if [ "${#}" -lt "2" ]; then
+ bbfatal "${1} requires an argument"
+ fi
+ bwrap_args="${bwrap_args} ${1} ${2}"
+ shift 2
+ ;;
+ -*)
+ bbfatal "${1} is not a supported option!"
+ ;;
+ *)

+ if [ -z "${bwrap_rootfs}" ]; then
+ bwrap_rootfs="${1}"

+ shift
+ else
+ bbfatal "unexpected argument '${1}'"
+ fi
+ ;;
+ esac
+ done
+

+ if [ -n "${bwrap_rootfs}" ]; then
+ bwrap_args="${bwrap_args} --bind ${bwrap_rootfs} /"

+ fi
+
+ if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+ bbfatal "no command specified (missing --)"
+ fi
+ shift # remove "--", command and its arguments follows
+
+ for ro_d in bin etc lib lib64 sys usr var; do

+ [ -d ${bwrap_rootfs}/${ro_d} ] || continue
+ bwrap_args="${bwrap_args} --ro-bind ${bwrap_rootfs}/${ro_d} /${ro_d}"

+ done
+
+ bwrap --unshare-user --unshare-pid ${bwrap_args} \
+ --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
+ -- "${@}"
+}
+
rootfs_do_mounts[weight] = "3"
rootfs_do_mounts() {
sudo -s <<'EOSUDO'
--

2.39.5

[Cedric Hombourger]

Elevated privileges are not required to query the rootfs for the version
of systemd: replace "sudo chroot" with "rootfs_cmd"

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/image-postproc-extension.bbclass | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 991bac4c..0af588d8 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass

@@ -53,12 +53,17 @@ image_postprocess_mark() {
--build-id "${BUILD_ID}" --variant "${DESCRIPTION}" --version "${PV}"
}

+# Use dpkg to find out which version of systemd is installed into the image or reports "0"
+image_systemd_version() {
+ sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+}

[Quirin Gylstorff]

Why are we not using the root parameter of dpkg-query for this?

Quirin
>
> ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"

[Cedric Hombourger]

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Testing:

- Basic smoke tests performed successfully (citest.py -t fast)

- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v2 patch:
- rootfs_install_pkgs_download will no longer use sudo to run
apt-get install --download-only. This was added to further
demonstrate/test rootfs_cmd in existing Isar code.

Changes since v1 patch:
- Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
- Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
- Let caller decide where to bind-mount the rootfs to
- Make the rootfs argument optional
- Support 32-bit rootfs (no lib64 there)

(Re-)validated with "citest.py -t fast" (using kas-container 4.8.1):

JOB ID : 2724be97c6711e046fbc2169823c293dc99cd97c
JOB LOG : avocado/job-results/job-2025-06-25T15.51-2724be9/job.log
(01/19) citest.py:DevTest.test_dev: STARTED
(01/19) citest.py:DevTest.test_dev: PASS (1573.34 s)
(02/19) citest.py:DevTest.test_dev_apps: STARTED
(02/19) citest.py:DevTest.test_dev_apps: PASS (2158.85 s)
(03/19) citest.py:DevTest.test_dev_rebuild: STARTED
(03/19) citest.py:DevTest.test_dev_rebuild: PASS (349.73 s)
(04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
(04/19) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (77.79 s)
(05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
(05/19) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (55.12 s)
(06/19) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
(06/19) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (58.94 s)
(07/19) citest.py:CrossTest.test_cross: STARTED
(07/19) citest.py:CrossTest.test_cross: PASS (1912.25 s)
(08/19) citest.py:CrossTest.test_cross_debsrc: STARTED
(08/19) citest.py:CrossTest.test_cross_debsrc: PASS (2933.62 s)
(09/19) citest.py:CrossTest.test_cross_kselftest: STARTED
(09/19) citest.py:CrossTest.test_cross_kselftest: PASS (2024.26 s)
(10/19) citest.py:CrossTest.test_cross_rpi: STARTED
(10/19) citest.py:CrossTest.test_cross_rpi: PASS (1543.77 s)
(11/19) citest.py:VmBootTestFast.test_arm_bullseye: STARTED
(11/19) citest.py:VmBootTestFast.test_arm_bullseye: PASS (64.33 s)
(12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED
(12/19) citest.py:VmBootTestFast.test_arm_bullseye_example_module: PASS (12.72 s)
(13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED
(13/19) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: PASS (10.18 s)
(14/19) citest.py:VmBootTestFast.test_arm_buster: STARTED
(14/19) citest.py:VmBootTestFast.test_arm_buster: PASS (57.01 s)
(15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED
(15/19) citest.py:VmBootTestFast.test_arm_buster_getty_target: PASS (9.73 s)
(16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED
(16/19) citest.py:VmBootTestFast.test_arm_buster_example_module: PASS (10.39 s)
(17/19) citest.py:VmBootTestFast.test_arm_bookworm: STARTED
(17/19) citest.py:VmBootTestFast.test_arm_bookworm: PASS (82.93 s)
(18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED
(18/19) citest.py:VmBootTestFast.test_arm_bookworm_example_module: PASS (30.40 s)
(19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED
(19/19) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: PASS (11.59 s)
RESULTS : PASS 19 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 13003.86 s

Cedric Hombourger (5):

rootfs: introduce wrapper to run commands against a rootfs
deb-dl-dir: optimize caching of source packages using apt natively
image-postproc-extension: refactor systemd version checks
image-postproc-extension: extract systemd's version using rootfs_cmd

bootstrap: create lock for downloads/deb without sudo
rootfs: do not get elevated privileges when downloading packages

RECIPE-API-CHANGELOG.md | 7 ++
doc/user_manual.md | 1 +

meta/classes/deb-dl-dir.bbclass | 58 ++++++-------
meta/classes/image-postproc-extension.bbclass | 12 +--
meta/classes/rootfs.bbclass | 83 ++++++++++++++++++-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 4 +
6 files changed, 126 insertions(+), 39 deletions(-)

--
2.39.5

[Cedric Hombourger]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

[Cedric Hombourger]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a command could
be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 7 ++++
doc/user_manual.md | 1 +

meta/classes/rootfs.bbclass | 67 +++++++++++++++++++++++++++++++++++++
3 files changed, 75 insertions(+)

index 5f877962..429494ae 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -34,6 +34,73 @@ export LANG = "C"

export LANGUAGE = "C"
export LC_ALL = "C"

+# Execute a command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the
+# optional rootfs argument is omitted, the host rootfs will be used (e.g. to
+# run native commands): this should be used with care.
+#
+# Usage: rootfs_cmd [options] [rootfs] -- command
+#
+rootfs_cmd() {
+ set -- "$@"
+ bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"

+ bwrap_binds=""

+ bwrap_rootfs=""
+
+ while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+ case "${1}" in
+ --bind)
+ if [ "${#}" -lt "3" ]; then
+ bbfatal "--bind requires two arguments"
+ fi

+ bwrap_binds="${bwrap_binds} --bind ${2} ${3}"

+ ${bwrap_binds} -- "${@}"

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/image-postproc-extension.bbclass | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 991bac4c..0af588d8 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass

@@ -53,12 +53,17 @@ image_postprocess_mark() {
--build-id "${BUILD_ID}" --variant "${DESCRIPTION}" --version "${PV}"
}

+# Use dpkg to find out which version of systemd is installed into the image or reports "0"
+image_systemd_version() {
+ sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+}

[Cedric Hombourger]

From: Cedric Hombourger' via isar-users <isar-...@googlegroups.com>

Elevated privileges are not required to query the rootfs for the version
of systemd: replace "sudo chroot" with "rootfs_cmd"

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/image-postproc-extension.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 0af588d8..21dcfccc 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass

@@ -55,7 +55,7 @@ image_postprocess_mark() {

# Use dpkg to find out which version of systemd is installed into the image or reports "0"

[Cedric Hombourger]

The syncin/syncout commands passed to mmdebstrap will create a lock
file in downloads/deb if it does not exist. As mmdebstrap is being
executed as root, the lock would also be owned by root and this will
cause problems for rootless commands that may be executed later (such
as downloading of Debian packages). Create the lock file without
sudo prior to running mmdebstrap for it to be owned by the build user
rather than root.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index 931f6f13..b2de61ad 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -181,6 +181,10 @@ do_bootstrap() {
&& sudo umount $base_apt_tmp \
&& rm -rf --one-file-system $base_apt_tmp' EXIT

+ # Create lock file so that it is owned by the user running the build (not root)
+ mkdir -p ${DEBDIR}
+ touch ${DEB_DL_LOCK}
+
sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
$arch_param \
--mode=unshare \
--
2.39.5

[Cedric Hombourger]

Use rootfs_cmd() to run "apt-get install --download-only" without sudo.
This requires /var/cache/apt/archives/ to be writable by the build
user: change ownership while populating that folder with previously
downloaded packages (those in downloads/deb/).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

meta/classes/deb-dl-dir.bbclass | 21 ++++++++++++++++++---
meta/classes/rootfs.bbclass | 16 +++++++++++++---
2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 7026f4f4..7fe052ef 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -100,9 +100,24 @@ dbg_pkgs_download() {
deb_dl_dir_import() {
export pc="${DEBDIR}/${2}"
export rootfs="${1}"
- sudo mkdir -p "${rootfs}"/var/cache/apt/archives/
+ export uid=$(id -u)
+ export gid=$(id -g)
+
+ # let our unprivileged user place downloaded packages in /var/cache/apt/archives/
+ sudo -Es << ' EOSUDO'
+ mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
+ touch "${rootfs}"/var/cache/apt/archives/lock
+ chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
+ EOSUDO
+
+ # nothing to copy if download directory does not exist just yet
[ ! -d "${pc}" ] && return 0
- flock -s "${pc}".lock sudo -Es << 'EOSUDO'
+
+ # attempt to create hard-links for .deb files from downloads/ into
+ # /var/cache/apt/archives/ so apt will only download packages we
+ # have not yet downloaded. perform a regular copy whenever hard-links
+ # cannot be created
+ ( flock 9

set -e
printenv | grep -q BB_VERBOSE_LOGS && set -x

@@ -111,7 +126,7 @@ deb_dl_dir_import() {
ln -Pf -t "${rootfs}"/var/cache/apt/archives/ "$p" 2>/dev/null ||
cp -n --no-preserve=owner -t "${rootfs}"/var/cache/apt/archives/ "$p"
done
-EOSUDO
+ ) 9>"${pc}".lock
}

deb_dl_dir_export() {
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 429494ae..977bbec8 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -277,10 +277,20 @@ ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download"
rootfs_install_pkgs_download[weight] = "600"
rootfs_install_pkgs_download[progress] = "custom:rootfs_progress.PkgsDownloadProgressHandler"
rootfs_install_pkgs_download[isar-apt-lock] = "release-after"
-rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
+rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK}"
rootfs_install_pkgs_download() {
- sudo -E chroot '${ROOTFSDIR}' \
- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
+ mkdir -p "${WORKDIR}/dpkg"
+
+ # Use our own dpkg lock files rather than those in the rootfs since we are not root
+ # (this is safe as there are no concurrent apt/dpkg operations for that rootfs)
+ touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend"
+
+ # download packages using apt in a non-privileged namespace
+ rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
+ --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \
+ --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \
+ ${ROOTFSDIR} \
+ -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
}

ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""
--
2.39.5

[Jan Kiszka]

Pro: even faster than any wrapper
Con: brings some dependency between dpkg and data base versions

But this option should still be considered in the discussion, maybe also
depending on the type of access (read as above vs. write as in other cases).

Cedric, did you think about it?

[MOESSBAUER, Felix]

On Thu, 2025-06-26 at 03:37 +0800, Cedric Hombourger wrote:
> When building root filesystems for foreign architectures with package
> source
> caching enabled, apt operations are executed within the rootfs
> through QEMU
> emulation. This results in significantly degraded performance,
> particularly
> when downloading source packages sequentially.
>
> This patch series introduces a new wrapper function that enables
> native
> command execution against a rootfs while preserving special mount
> points
> (such as /isar-apt). The approach:

Hi, are there any news on this series? Was there a follow up version?

The upcoming SBOM feature from Christoph already wants to make use of
bubblewrap. We also have the tool in kas 4.8.

Some quick tests against amd64 and arm64 also did not show any issues.

Best regards,
Felix

[Jan Kiszka]

On 25.06.25 21:37, 'Cedric Hombourger' via isar-users wrote:
> "sudo chroot" is used in several places to run commands inside rootfs
> directories constructed by Isar. There are cases where a command could
> be used without elevated privileges as long as special folders such as
> /isar-apt are mounted (they are often referenced as /isar-apt in
> configuration files found in the target rootfs). For such cases,
> bubblewrap may be used to create a non-privileged namespace (either
> in a bare/native environment or within a docker/podman container)
> where the command will be executed as if chroot had been used. The
> rootfs may also be the host root file-system: this should however
> be used with care to avoid host contamination problems (note: Isar
> already relies on a number of host tools).

Where does this take the commands from then, the host env or some better
defined rootfs that is aligned with the target rootfs release-wise? Is
that controlled by the caller or implicitly by the wrapper.

I have to remind that we cannot blindly use host-side tools on the
target rootfs (except for the very basic ones) as the latter may be
newer than the former and not necessarily compatible.

Jan

[cedric.h...@siemens.com]

On Mon, 2025-09-15 at 10:28 +0200, Jan Kiszka wrote:
> On 25.06.25 21:37, 'Cedric Hombourger' via isar-users wrote:
> > "sudo chroot" is used in several places to run commands inside
> > rootfs
> > directories constructed by Isar. There are cases where a command
> > could
> > be used without elevated privileges as long as special folders such
> > as
> > /isar-apt are mounted (they are often referenced as /isar-apt in
> > configuration files found in the target rootfs). For such cases,
> > bubblewrap may be used to create a non-privileged namespace (either
> > in a bare/native environment or within a docker/podman container)
> > where the command will be executed as if chroot had been used. The
> > rootfs may also be the host root file-system: this should however
> > be used with care to avoid host contamination problems (note: Isar
> > already relies on a number of host tools).
>
> Where does this take the commands from then, the host env or some
> better
> defined rootfs that is aligned with the target rootfs release-wise?
> Is
> that controlled by the caller or implicitly by the wrapper.

rootfs_cmd is a general-purpose helper and does not select a rootfs of
its own where it will run commands from. This is left to the caller to
decide. given a rootfs, it will let bubblewrap create a namespace with
relevant mappings, optionally chdir to a specified directory and run
the user-specified command.

>
> I have to remind that we cannot blindly use host-side tools on the
> target rootfs (except for the very basic ones) as the latter may be
> newer than the former and not necessarily compatible.

Agreed. if we agree on introducing rootfs_cmd then uses shall be
audited. Reliance on host-tools shall be kept to a minimum to avoid
host-contamination problems but also avoid incompatibilities as you
have correctly noted.

We can debate whether the 1st user of rootfs_cmd from this patch series
(using apt to download source packages from a target rootfs) should
have used apt from / (hopefully a kas-container but not guaranteed) or
from Isar's host rootfs. With mmdebstrap (used from /) using apt (also
from /), I felt that it was ok.

If you prefer that I switch to have rootfs_cmd call apt from an Isar
host rootfs then I can rework the patch series to do so. We may have
other cases where we need a host tool (pulled into an Isar's host
rootfs) to operate on a target rootfs.

Please advise.

[Jan Kiszka]

So none of the patches 2..6 changes the source rootfs for the command to
run?

>
>>
>> I have to remind that we cannot blindly use host-side tools on the
>> target rootfs (except for the very basic ones) as the latter may be
>> newer than the former and not necessarily compatible.
>
> Agreed. if we agree on introducing rootfs_cmd then uses shall be
> audited. Reliance on host-tools shall be kept to a minimum to avoid
> host-contamination problems but also avoid incompatibilities as you
> have correctly noted.
>
> We can debate whether the 1st user of rootfs_cmd from this patch series
> (using apt to download source packages from a target rootfs) should
> have used apt from / (hopefully a kas-container but not guaranteed) or
> from Isar's host rootfs. With mmdebstrap (used from /) using apt (also
> from /), I felt that it was ok.

Really? mmdebstrap builds an maintains a sid rootfs via a bookworm or
even older toolset?

>
> If you prefer that I switch to have rootfs_cmd call apt from an Isar
> host rootfs then I can rework the patch series to do so. We may have
> other cases where we need a host tool (pulled into an Isar's host
> rootfs) to operate on a target rootfs.

See my question above: If this series does not change the behavior in
step 1, we can move forward and change the tooling source later on. If
it does already, we should address that in the same run.

Jan

[cedric.h...@siemens.com]

patch #2: rootfs_cmd will use / as rootfs (to be discussed/decided if
we would prefer to use an Isar host rootfs instead). Use limited to
"apt-get s--download-only source"

patch #3: does not use rootfs_cmd. this is a preparation step

patch #4: rootfs_cmd will use ${IMAGE_ROOTFS} to query the dpkg
database of the image

patch #5: does introduce/use rootfs_cmd. creating a folder and file as
a regular user before we "sudo mmdebstrap" so that the created files
are owned by our calling user and not root

patch #6: a somewhat similar story than #5. This patch does not use
rootfs_cmd but addresses some file ownership "problems"

in summary, only #2 and #4 are using rootfs_cmd
the whole series is an attempt to require less elevated privileges than
we currently are

>
> >
> > >
> > > I have to remind that we cannot blindly use host-side tools on
> > > the
> > > target rootfs (except for the very basic ones) as the latter may
> > > be
> > > newer than the former and not necessarily compatible.
> >
> > Agreed. if we agree on introducing rootfs_cmd then uses shall be
> > audited. Reliance on host-tools shall be kept to a minimum to avoid
> > host-contamination problems but also avoid incompatibilities as you
> > have correctly noted.
> >
> > We can debate whether the 1st user of rootfs_cmd from this patch
> > series
> > (using apt to download source packages from a target rootfs) should
> > have used apt from / (hopefully a kas-container but not guaranteed)
> > or
> > from Isar's host rootfs. With mmdebstrap (used from /) using apt
> > (also
> > from /), I felt that it was ok.
>
> Really? mmdebstrap builds an maintains a sid rootfs via a bookworm or
> even older toolset?

strace on a (manual) mmdebstrap run on trixie to create a bookworm
rootfs suggests that it *does* use apt from the host (trixie):

[pid 2055141] execve("/usr/bin/apt-get", ["apt-get", "-o",
"Dir::Bin::dpkg=env", "-o", "DPkg::Options::=--unset=TMPDIR", "-o",
"DPkg::Options::=dpkg", "--yes", "install", "-oAPT::Status-Fd=7", "-
oDpkg::Use-Pty=false", "apt", "?narrow(?or(?archive(^bookworm$)"...],
0x5cf908e87f60 /* 20 vars */) = 0

It appears to use chroot to run some (selected) dpkg --install commands
but not to run apt

>
> >
> > If you prefer that I switch to have rootfs_cmd call apt from an
> > Isar
> > host rootfs then I can rework the patch series to do so. We may
> > have
> > other cases where we need a host tool (pulled into an Isar's host
> > rootfs) to operate on a target rootfs.
>
> See my question above: If this series does not change the behavior in
> step 1, we can move forward and change the tooling source later on.
> If
> it does already, we should address that in the same run.

I hope the above answers clarify. Do let me know if the above findings
give us a new direction to work towards.

>
> Jan

[Andreas Naumann]

Hi Cedric, all

Am 20.08.25 um 17:38 schrieb 'MOESSBAUER, Felix' via isar-users:

> On Thu, 2025-06-26 at 03:37 +0800, Cedric Hombourger wrote:
>> When building root filesystems for foreign architectures with package
>> source
>> caching enabled, apt operations are executed within the rootfs
>> through QEMU
>> emulation. This results in significantly degraded performance,
>> particularly
>> when downloading source packages sequentially.
>>
>> This patch series introduces a new wrapper function that enables
>> native
>> command execution against a rootfs while preserving special mount
>> points
>> (such as /isar-apt). The approach:
> Hi, are there any news on this series? Was there a follow up version?

I also find this series very interesting, but unfortunately I was not
the ML when you originally sent it, and getting a patch out of
googlegroups is a nightmare.

So I'd really appreciate a follow-up. Eg. the patch which touches the
deb-dl-lock could imho be merged as is. It would fix an issue we have in
on of our setups.

Other than that, I have played with various scenarios to run the build
in an unprivileged container and being able to use bwrap for
bind-mounting /dev and /proc without sudo or privileges would be a big
step forward.

So looking forward :-)

regards,
Andreas

>
> The upcoming SBOM feature from Christoph already wants to make use of
> bubblewrap. We also have the tool in kas 4.8.
>
> Some quick tests against amd64 and arm64 also did not show any issues.
>
> Best regards,
> Felix
>
--

Andreas Naumann

emlix GmbH
Headquarters: Berliner Str. 12, 37073 Goettingen, Germany
Phone +49 (0)551 30664-0, e-mail in...@emlix.com
District Court of Goettingen, Registry Number HR B 3160
Managing Directors: Heike Jordan, Dr. Uwe Kracke
VAT ID No. DE 205 198 055
Office Berlin: Panoramastr. 1, 10178 Berlin, Germany
Office Bonn: Bachstr. 6, 53115 Bonn, Germany
http://www.emlix.com

[Jan Kiszka]

On 16.09.25 17:53, Andreas Naumann wrote:
> I also find this series very interesting, but unfortunately I was not
> the ML when you originally sent it, and getting a patch out of
> googlegroups is a nightmare.
>

We have https://patchwork.isar-build.org/ for those cases. Yes, the
googlegroups archive sucks. Unfortunately, kernel.org didn't accept
hosting isar-users.

[Cedric Hombourger]

From: "cedric.h...@siemens.com" <cedric.h...@siemens.com>

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a command could
be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---

RECIPE-API-CHANGELOG.md | 8 +++++

doc/user_manual.md | 1 +
meta/classes/rootfs.bbclass | 67 +++++++++++++++++++++++++++++++++++++

3 files changed, 76 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 92e7811c..53e650d4 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -741,3 +741,11 @@ By setting `MS_TPM_20_REF_DIR` in an optee-ftpm recipe, it is now possible to
use the new optee_ftpm code base from the OP-TEE project. That variable has to
point to a subdir in `WORKDIR` which contains the unpacked ms-tpm-20-ref source
code.

+
+### Require bubblewrap to run non-privileged commands with bind-mounts
+
+Isar occasionally needs to run commands within root file-systems that it
+builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be

+used in Isar classes instead of `sudo chroot` to avoid unecessary privilege
+elevations (when we "just" need to chroot but do not require root). It is
+pre-installed in kas-container version 4.8 (or later).
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 67f91973..be89ce1d 100644

--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -75,6 +75,7 @@ Install the following packages:
```
apt install \
binfmt-support \
+ bubblewrap \
bzip2 \
mmdebstrap \
arch-test \
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass

index ebe3bf4a..f740c6e1 100644

2.47.3

[Cedric Hombourger]

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Additional notes:
- rootfs_cmd may be used to run commands from the host root file-system:
use with extreme care to avoid host contamination problems.
- mmdebstrap already calls apt-get of the host to download packages (in
other words, a build of a bookworm image from a trixie host will
result in mmdebstrap (from trixie), call apt-get (from trixie) to
download bookworm packages. This is the behavior we have before and
after these changes.
- With these changes and when caching of Debian source packages is
enabled/requested, Isar will use apt-get of the host to download
source packages (it will however do this from a bubblewrap'ed
environment to avoid a non-required privilege elevation; Isar has
many but we need to start from somewhere).

Testing:
- Tested against 9e62337953fbb8371c846c44e8a99d62a8d220ba

- Basic smoke tests performed successfully (citest.py -t fast)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v3 patch:
- drop image-postproc-extension patches (refactoring and use of
rootfs_cmd). They are not strictly needed and were only meant
to provide another potential use of rootfs_cmd.
- Rebase changes to RECIPE-API-CHANGELOG.md and added a few extra
words about the motivation.

Changes since v2 patch:
- rootfs_install_pkgs_download will no longer use sudo to run
apt-get install --download-only. This was added to further
demonstrate/test rootfs_cmd in existing Isar code.

Changes since v1 patch:
- Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
- Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
- Let caller decide where to bind-mount the rootfs to
- Make the rootfs argument optional
- Support 32-bit rootfs (no lib64 there)

Test Results (avocado started from a kas-container version 4.8.1):
(01/22) citest.py:DevTest.test_dev: STARTED
(01/22) citest.py:DevTest.test_dev: PASS (1132.17 s)
(02/22) citest.py:DevTest.test_dev_apps: STARTED
(02/22) citest.py:DevTest.test_dev_apps: PASS (845.24 s)
(03/22) citest.py:DevTest.test_dev_rebuild: STARTED
(03/22) citest.py:DevTest.test_dev_rebuild: PASS (689.53 s)
(04/22) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
(04/22) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (53.79 s)
(05/22) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
(05/22) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (32.64 s)
(06/22) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
(06/22) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (34.15 s)
(07/22) citest.py:CrossTest.test_cross: STARTED
(07/22) citest.py:CrossTest.test_cross: PASS (488.24 s)
(08/22) citest.py:CrossTest.test_cross_debsrc: STARTED
(08/22) citest.py:CrossTest.test_cross_debsrc: PASS (1409.06 s)
(09/22) citest.py:CrossTest.test_cross_trixie: STARTED
(09/22) citest.py:CrossTest.test_cross_trixie: PASS (216.54 s)
(10/22) citest.py:CrossTest.test_cross_kselftest: STARTED
(10/22) citest.py:CrossTest.test_cross_kselftest: PASS (340.48 s)
(11/22) citest.py:CrossTest.test_cross_rpi: STARTED
(11/22) citest.py:CrossTest.test_cross_rpi: PASS (1053.48 s)
(12/22) citest.py:VmBootTestFast.test_arm_bullseye: STARTED
(12/22) citest.py:VmBootTestFast.test_arm_bullseye: PASS (41.03 s)
(13/22) citest.py:VmBootTestFast.test_arm_bullseye_example_module: STARTED
(13/22) citest.py:VmBootTestFast.test_arm_bullseye_example_module: PASS (7.07 s)
(14/22) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: STARTED
(14/22) citest.py:VmBootTestFast.test_arm_bullseye_getty_target: PASS (7.82 s)
(15/22) citest.py:VmBootTestFast.test_arm_buster: STARTED
(15/22) citest.py:VmBootTestFast.test_arm_buster: PASS (37.54 s)
(16/22) citest.py:VmBootTestFast.test_arm_buster_getty_target: STARTED
(16/22) citest.py:VmBootTestFast.test_arm_buster_getty_target: PASS (6.79 s)
(17/22) citest.py:VmBootTestFast.test_arm_buster_example_module: STARTED
(17/22) citest.py:VmBootTestFast.test_arm_buster_example_module: PASS (7.57 s)
(18/22) citest.py:VmBootTestFast.test_arm_bookworm: STARTED
(18/22) citest.py:VmBootTestFast.test_arm_bookworm: PASS (49.58 s)
(19/22) citest.py:VmBootTestFast.test_arm_bookworm_example_module: STARTED
(19/22) citest.py:VmBootTestFast.test_arm_bookworm_example_module: PASS (8.06 s)
(20/22) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: STARTED
(20/22) citest.py:VmBootTestFast.test_arm_bookworm_getty_target: PASS (8.18 s)
(21/22) citest.py:VmBootTestFast.test_amd64_trixie: STARTED
(21/22) citest.py:VmBootTestFast.test_amd64_trixie: PASS (37.14 s)
(22/22) citest.py:VmBootTestFast.test_arm64_trixie: STARTED
(22/22) citest.py:VmBootTestFast.test_arm64_trixie: PASS (41.79 s)
RESULTS : PASS 22 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 6585.87 s

cedric.h...@siemens.com (4):

rootfs: introduce wrapper to run commands against a rootfs
deb-dl-dir: optimize caching of source packages using apt natively

bootstrap: create lock for downloads/deb without sudo
rootfs: do not get elevated privileges when downloading packages

RECIPE-API-CHANGELOG.md | 8 ++

doc/user_manual.md | 1 +
meta/classes/deb-dl-dir.bbclass | 58 ++++++-------

meta/classes/rootfs.bbclass | 83 ++++++++++++++++++-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 4 +

5 files changed, 120 insertions(+), 34 deletions(-)

--
2.47.3

[MOESSBAUER, Felix]

On Thu, 2025-09-25 at 08:54 +0200, 'Cedric Hombourger' via isar-users
wrote:

Thanks for bringing this up again. I successfully tested this series on
(amd64, arm64) x (bookworm, trixie).

Tested-by: Felix Moessbauer <felix.mo...@siemens.com>