[PATCH v3 00/16] add support to build isar unprivileged
0 views
Skip to first unread message
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
Dear isar-users,
currently isar requires password-less sudo and an environment
where mounting file systems is possible. This has proven problematic
for security reasons, both when running in a privileged container or
locally.
To solve this, we implement fully rootless builds that rely on the
unshare syscall which allows us to avoid sudo and instead operate in
temporary kernel namespaces as a user that is just privileged within
that namespace. This comes with some challenges regarding the handling
of mounts (they are cleared when leaving the namespace), as well as
cross namespace deployments (the outer user might not be able to access
the inner data). For that, we rework the handling of mounts and artifact
passing to make it compatible with both chroot modes (schroot and
unshare).
Note, that this series can be tested on a custom kas-container build
provided in [1]. Hints how to migrate downstream layers are provided
in the API changelog.
Changes since PATCH v2:
- add support for cached base apt
- rootfs sstate: do not rely on fd3 for copy out, as not always available
- sbom: use local copy of sbom rootfs to not leave shared instance behind
- testsuite: add parameter to run in rootless mode
- rebased onto v1.0
Changes since PATCH v1:
- fixed broken rebase onto next
- fix root_cleandirs implementation
NOTE: This requires the kas series (v3) from [1] for rootless building.
Changes since RFC 2:
- rebased onto next
- fix usage of root_cleandirs
- simplify file permission handling by mapping caller user to
root inside the namespace. By that, in most cases no changes
to the imager are needed anymore.
- implement support for devshell under rootless
- switch to getpass.getuser() to query user (needed for dynamically
created / remapped kas builder user)
- rework mapping to be more similar to mapping used by mmdebstrap
- sbuild: only copy-out of dpkg.log on schroot (unclear if needed
on unshare. To be clarified)
- imager-sbom: ensure sbom is extracted before entering the chroot
Changes since RFC 1:
- switch build_type to isar-rootless in isar.yaml (Note: switch back
if testing locally in a unprepared kas container)
- complete overhaul of the mounting in unshared namespaces
- fixes the systemd presetting
- fixes hangs when pulling from snapshot mirrors
- rename the run_privileged_here to run_privileged_heredoc to clarify its intention
- add support for
- dpkg-source with do_fetch_common_source
- vm images
- container images
- discoverable disk images
- add helper script to clean build dir in unprivileged mode
- reduce clutter we leave after finishing a build
- fix issues when running in a privileged environment without sub user ids
- bugfixes
Note, that the rootless build dir must not reside in a git worktree (a normal git
dir is fine). This is probably a bug in combination with kas-container.
[1] https://groups.google.com/g/kas-devel/c/NWQFCU2aUHg
Best regards,
Felix Moessbauer
Siemens AG
Felix Moessbauer (16):
refactor bootstrap: store rootfs tar with user permissions
deb-dl-dir: export without root privileges
download debs without locking
introduce wrappers for privileged execution
bootstrap: move cleanup trap to function
rootfs: rework sstate caching of rootfs artifact
rootfs_generate_initramfs: rework deployment to avoid chowning
use bitbake function to generate mounting scripts
apt-fetcher: prepare for chroot specific fetching
add support for fully rootless builds
add helper script to clean artifacts in build dir
apt-fetcher: implement support for unshare backend
dpkg-source: implement multiarch support for unshare backend
use copy of sbom-chroot for sbom creation
add support for devshell on unshare backend
testsuite: add parameter to run tests in rootless mode
Kconfig | 2 +-
RECIPE-API-CHANGELOG.md | 42 ++++
doc/user_manual.md | 2 +
kas/isar.yaml | 2 +-
meta/classes-global/base.bbclass | 124 ++++++++++-
meta/classes-recipe/deb-dl-dir.bbclass | 24 ++-
meta/classes-recipe/dpkg-base.bbclass | 94 ++++++--
meta/classes-recipe/dpkg-source.bbclass | 40 +++-
meta/classes-recipe/dpkg.bbclass | 19 +-
.../image-account-extension.bbclass | 4 +-
.../image-locales-extension.bbclass | 13 +-
.../image-postproc-extension.bbclass | 30 +--
.../image-tools-extension.bbclass | 114 +++++++++-
meta/classes-recipe/image.bbclass | 21 +-
.../imagetypes_container.bbclass | 28 +--
meta/classes-recipe/imagetypes_wic.bbclass | 10 +-
meta/classes-recipe/rootfs.bbclass | 203 +++++++++---------
meta/classes-recipe/sbuild.bbclass | 34 ++-
meta/classes-recipe/sdk.bbclass | 22 +-
meta/classes/sbom.bbclass | 28 ++-
meta/conf/bitbake.conf | 7 +-
meta/lib/aptsrc_fetcher.py | 87 +++++++-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 55 +++--
.../sbom-chroot/sbom-chroot.bb | 11 +-
.../sbuild-chroot/sbuild-chroot.inc | 24 ++-
scripts/isar-clean-builddir | 73 +++++++
testsuite/cibuilder.py | 6 +
.../unittests/test_image_account_extension.py | 9 +-
28 files changed, 882 insertions(+), 246 deletions(-)
create mode 100755 scripts/isar-clean-builddir
--
2.53.0
currently isar requires password-less sudo and an environment
where mounting file systems is possible. This has proven problematic
for security reasons, both when running in a privileged container or
locally.
To solve this, we implement fully rootless builds that rely on the
unshare syscall which allows us to avoid sudo and instead operate in
temporary kernel namespaces as a user that is just privileged within
that namespace. This comes with some challenges regarding the handling
of mounts (they are cleared when leaving the namespace), as well as
cross namespace deployments (the outer user might not be able to access
the inner data). For that, we rework the handling of mounts and artifact
passing to make it compatible with both chroot modes (schroot and
unshare).
Note, that this series can be tested on a custom kas-container build
provided in [1]. Hints how to migrate downstream layers are provided
in the API changelog.
Changes since PATCH v2:
- add support for cached base apt
- rootfs sstate: do not rely on fd3 for copy out, as not always available
- sbom: use local copy of sbom rootfs to not leave shared instance behind
- testsuite: add parameter to run in rootless mode
- rebased onto v1.0
Changes since PATCH v1:
- fixed broken rebase onto next
- fix root_cleandirs implementation
NOTE: This requires the kas series (v3) from [1] for rootless building.
Changes since RFC 2:
- rebased onto next
- fix usage of root_cleandirs
- simplify file permission handling by mapping caller user to
root inside the namespace. By that, in most cases no changes
to the imager are needed anymore.
- implement support for devshell under rootless
- switch to getpass.getuser() to query user (needed for dynamically
created / remapped kas builder user)
- rework mapping to be more similar to mapping used by mmdebstrap
- sbuild: only copy-out of dpkg.log on schroot (unclear if needed
on unshare. To be clarified)
- imager-sbom: ensure sbom is extracted before entering the chroot
Changes since RFC 1:
- switch build_type to isar-rootless in isar.yaml (Note: switch back
if testing locally in a unprepared kas container)
- complete overhaul of the mounting in unshared namespaces
- fixes the systemd presetting
- fixes hangs when pulling from snapshot mirrors
- rename the run_privileged_here to run_privileged_heredoc to clarify its intention
- add support for
- dpkg-source with do_fetch_common_source
- vm images
- container images
- discoverable disk images
- add helper script to clean build dir in unprivileged mode
- reduce clutter we leave after finishing a build
- fix issues when running in a privileged environment without sub user ids
- bugfixes
Note, that the rootless build dir must not reside in a git worktree (a normal git
dir is fine). This is probably a bug in combination with kas-container.
[1] https://groups.google.com/g/kas-devel/c/NWQFCU2aUHg
Best regards,
Felix Moessbauer
Siemens AG
Felix Moessbauer (16):
refactor bootstrap: store rootfs tar with user permissions
deb-dl-dir: export without root privileges
download debs without locking
introduce wrappers for privileged execution
bootstrap: move cleanup trap to function
rootfs: rework sstate caching of rootfs artifact
rootfs_generate_initramfs: rework deployment to avoid chowning
use bitbake function to generate mounting scripts
apt-fetcher: prepare for chroot specific fetching
add support for fully rootless builds
add helper script to clean artifacts in build dir
apt-fetcher: implement support for unshare backend
dpkg-source: implement multiarch support for unshare backend
use copy of sbom-chroot for sbom creation
add support for devshell on unshare backend
testsuite: add parameter to run tests in rootless mode
Kconfig | 2 +-
RECIPE-API-CHANGELOG.md | 42 ++++
doc/user_manual.md | 2 +
kas/isar.yaml | 2 +-
meta/classes-global/base.bbclass | 124 ++++++++++-
meta/classes-recipe/deb-dl-dir.bbclass | 24 ++-
meta/classes-recipe/dpkg-base.bbclass | 94 ++++++--
meta/classes-recipe/dpkg-source.bbclass | 40 +++-
meta/classes-recipe/dpkg.bbclass | 19 +-
.../image-account-extension.bbclass | 4 +-
.../image-locales-extension.bbclass | 13 +-
.../image-postproc-extension.bbclass | 30 +--
.../image-tools-extension.bbclass | 114 +++++++++-
meta/classes-recipe/image.bbclass | 21 +-
.../imagetypes_container.bbclass | 28 +--
meta/classes-recipe/imagetypes_wic.bbclass | 10 +-
meta/classes-recipe/rootfs.bbclass | 203 +++++++++---------
meta/classes-recipe/sbuild.bbclass | 34 ++-
meta/classes-recipe/sdk.bbclass | 22 +-
meta/classes/sbom.bbclass | 28 ++-
meta/conf/bitbake.conf | 7 +-
meta/lib/aptsrc_fetcher.py | 87 +++++++-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 55 +++--
.../sbom-chroot/sbom-chroot.bb | 11 +-
.../sbuild-chroot/sbuild-chroot.inc | 24 ++-
scripts/isar-clean-builddir | 73 +++++++
testsuite/cibuilder.py | 6 +
.../unittests/test_image_account_extension.py | 9 +-
28 files changed, 882 insertions(+), 246 deletions(-)
create mode 100755 scripts/isar-clean-builddir
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
As we are only downloading and we are the only one acting on the rootfs,
it is safe to not lock the apt cache. By that, we can avoid complex file
ownerships in the tree.
Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
meta/classes-recipe/deb-dl-dir.bbclass | 2 +-
meta/classes-recipe/dpkg-source.bbclass | 2 +-
meta/classes-recipe/image-locales-extension.bbclass | 2 +-
meta/classes-recipe/image-tools-extension.bbclass | 3 ++-
meta/classes-recipe/rootfs.bbclass | 10 +---------
meta/lib/aptsrc_fetcher.py | 2 +-
6 files changed, 7 insertions(+), 14 deletions(-)
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 3ea75996..e3f055c5 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -80,6 +80,7 @@ debsrc_download() {
--chdir "/deb-src/${rootfs_distro}/${src}" \
-- \
apt-get -o APT::Architecture=${DISTRO_ARCH} \
+ -oDebug::NoLocking=1 \
-o Dir="${rootfs}" -y --download-only \
--only-source source "${src}=${version}" \
|| echo "${src} ${version}" >> ${missing}
@@ -121,7 +122,6 @@ deb_dl_dir_import() {
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
sudo -Es << ' EOSUDO'
mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
- touch "${rootfs}"/var/cache/apt/archives/lock
chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
EOSUDO
diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index d14d56aa..629796d6 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -77,7 +77,7 @@ do_fetch_common_source() {
schroot -r -c ${session_id} -d / -- \
sh -c '
cd /work
- apt-get -y --download-only --only-source -o Acquire::Source-Symlinks="false" source ${DEBIAN_SOURCE}'
+ apt-get -y --download-only --only-source -o Debug::NoLocking=1 -o Acquire::Source-Symlinks="false" source ${DEBIAN_SOURCE}'
schroot -e -c ${session_id}
remove_mounts
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index 32666311..c90280aa 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -30,7 +30,7 @@ image_install_localepurge_download[weight] = "40"
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
sudo -E chroot '${ROOTFSDIR}' \
- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only localepurge
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only localepurge
}
ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index ab616b7e..766f386d 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -54,7 +54,8 @@ imager_run() {
apt-get update \
-o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \
-o Dir::Etc::SourceParts='-' \
- -o APT::Get::List-Cleanup='0'
+ -o APT::Get::List-Cleanup='0' \
+ -o Debug::NoLocking=1
apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
--allow-unauthenticated --allow-downgrades --download-only install \
${local_install}"
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 8b502a50..b201b97d 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -340,18 +340,10 @@ rootfs_install_pkgs_download[progress] = "custom:rootfs_progress.PkgsDownloadPro
rootfs_install_pkgs_download[isar-apt-lock] = "release-after"
rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK}"
rootfs_install_pkgs_download() {
- mkdir -p "${WORKDIR}/dpkg"
-
- # Use our own dpkg lock files rather than those in the rootfs since we are not root
- # (this is safe as there are no concurrent apt/dpkg operations for that rootfs)
- touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend"
-
# download packages using apt in a non-privileged namespace
rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
- --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \
- --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \
${ROOTFSDIR} \
- -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
+ -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
}
ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index 4fe5a9ea..da2d94c2 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -41,7 +41,7 @@ class AptSrc(FetchMethod):
set -e
mkdir -p /downloads/{ud.localfile}
cd /downloads/{ud.localfile}
- apt-get -y --download-only --only-source source {ud.src_package}
+ apt-get -y -oDebug::NoLocking=1 --download-only --only-source source {ud.src_package}
'
''', d)
except (OSError, FetchError):
--
2.53.0
it is safe to not lock the apt cache. By that, we can avoid complex file
ownerships in the tree.
Signed-off-by: Felix Moessbauer <felix.mo...@siemens.com>
---
meta/classes-recipe/deb-dl-dir.bbclass | 2 +-
meta/classes-recipe/dpkg-source.bbclass | 2 +-
meta/classes-recipe/image-locales-extension.bbclass | 2 +-
meta/classes-recipe/image-tools-extension.bbclass | 3 ++-
meta/classes-recipe/rootfs.bbclass | 10 +---------
meta/lib/aptsrc_fetcher.py | 2 +-
6 files changed, 7 insertions(+), 14 deletions(-)
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 3ea75996..e3f055c5 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -80,6 +80,7 @@ debsrc_download() {
--chdir "/deb-src/${rootfs_distro}/${src}" \
-- \
apt-get -o APT::Architecture=${DISTRO_ARCH} \
+ -oDebug::NoLocking=1 \
-o Dir="${rootfs}" -y --download-only \
--only-source source "${src}=${version}" \
|| echo "${src} ${version}" >> ${missing}
@@ -121,7 +122,6 @@ deb_dl_dir_import() {
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
sudo -Es << ' EOSUDO'
mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
- touch "${rootfs}"/var/cache/apt/archives/lock
chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
EOSUDO
diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index d14d56aa..629796d6 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -77,7 +77,7 @@ do_fetch_common_source() {
schroot -r -c ${session_id} -d / -- \
sh -c '
cd /work
- apt-get -y --download-only --only-source -o Acquire::Source-Symlinks="false" source ${DEBIAN_SOURCE}'
+ apt-get -y --download-only --only-source -o Debug::NoLocking=1 -o Acquire::Source-Symlinks="false" source ${DEBIAN_SOURCE}'
schroot -e -c ${session_id}
remove_mounts
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index 32666311..c90280aa 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -30,7 +30,7 @@ image_install_localepurge_download[weight] = "40"
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
sudo -E chroot '${ROOTFSDIR}' \
- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only localepurge
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only localepurge
}
ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index ab616b7e..766f386d 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -54,7 +54,8 @@ imager_run() {
apt-get update \
-o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \
-o Dir::Etc::SourceParts='-' \
- -o APT::Get::List-Cleanup='0'
+ -o APT::Get::List-Cleanup='0' \
+ -o Debug::NoLocking=1
apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
--allow-unauthenticated --allow-downgrades --download-only install \
${local_install}"
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 8b502a50..b201b97d 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -340,18 +340,10 @@ rootfs_install_pkgs_download[progress] = "custom:rootfs_progress.PkgsDownloadPro
rootfs_install_pkgs_download[isar-apt-lock] = "release-after"
rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK}"
rootfs_install_pkgs_download() {
- mkdir -p "${WORKDIR}/dpkg"
-
- # Use our own dpkg lock files rather than those in the rootfs since we are not root
- # (this is safe as there are no concurrent apt/dpkg operations for that rootfs)
- touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend"
-
# download packages using apt in a non-privileged namespace
rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
- --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \
- --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \
${ROOTFSDIR} \
- -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
+ -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
}
ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index 4fe5a9ea..da2d94c2 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -41,7 +41,7 @@ class AptSrc(FetchMethod):
set -e
mkdir -p /downloads/{ud.localfile}
cd /downloads/{ud.localfile}
- apt-get -y --download-only --only-source source {ud.src_package}
+ apt-get -y -oDebug::NoLocking=1 --download-only --only-source source {ud.src_package}
'
''', d)
except (OSError, FetchError):
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
As the mmdebstrap itself is executed as root, the generated rootfs will
also be owned by root when stored as file. To avoid this, we let
mmdebstrap emit the rootfs on stdout and write it to a file outside of
the sudo call.
.../isar-mmdebstrap/isar-mmdebstrap.inc | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index 6c2cb170..f21a6164 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -246,11 +246,11 @@ do_bootstrap() {
${@get_apt_opts(d, '--aptopt')} \
${@get_distro_components_argument(d)} \
"${@get_distro_suite(d)}" \
- "${WORKDIR}/rootfs.tar.zst" \
- "$bootstrap_list"
+ - \
+ "$bootstrap_list" > ${WORKDIR}/rootfs.tar.zst
# Finalize bootstrap by setting the link in deploy
- sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
+ ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
@@ -265,8 +265,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
bootstrap_sstate_prepare() {
# this runs in SSTATE_BUILDDIR, which will be deleted automatically
- sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
- sudo chown $(id -u):$(id -g) bootstrap.tar.zst
+ cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
}
bootstrap_sstate_finalize() {
@@ -274,8 +273,8 @@ bootstrap_sstate_finalize() {
# we should restore symlinks after using tar
if [ -f bootstrap.tar.zst ]; then
mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst"
- sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \
- "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
+ ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \
+ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
fi
}
--
2.53.0
also be owned by root when stored as file. To avoid this, we let
mmdebstrap emit the rootfs on stdout and write it to a file outside of
the sudo call.
.../isar-mmdebstrap/isar-mmdebstrap.inc | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index 6c2cb170..f21a6164 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -246,11 +246,11 @@ do_bootstrap() {
${@get_apt_opts(d, '--aptopt')} \
${@get_distro_components_argument(d)} \
"${@get_distro_suite(d)}" \
- "${WORKDIR}/rootfs.tar.zst" \
- "$bootstrap_list"
+ - \
+ "$bootstrap_list" > ${WORKDIR}/rootfs.tar.zst
# Finalize bootstrap by setting the link in deploy
- sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
+ ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
@@ -265,8 +265,7 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
bootstrap_sstate_prepare() {
# this runs in SSTATE_BUILDDIR, which will be deleted automatically
- sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
- sudo chown $(id -u):$(id -g) bootstrap.tar.zst
+ cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
}
bootstrap_sstate_finalize() {
@@ -274,8 +273,8 @@ bootstrap_sstate_finalize() {
# we should restore symlinks after using tar
if [ -f bootstrap.tar.zst ]; then
mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst"
- sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \
- "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
+ ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \
+ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst"
fi
}
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
The archive is world readable, so we can access it without root
privileges. By that, the files in the download dir are also owned by the
calling user, making the additional chown obsolete.
meta/classes-recipe/deb-dl-dir.bbclass | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 27d1fb24..3ea75996 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -153,7 +153,7 @@ deb_dl_dir_export() {
isar_debs="$(${SCRIPTSDIR}/lockrun.py -r -f '${REPO_ISAR_DIR}/isar.lock' -c \
"find '${REPO_ISAR_DIR}/${DISTRO}' -name '*.deb' -print")"
- flock "${pc}".lock sudo -Es << 'EOSUDO'
+ flock "${pc}".lock /bin/bash -s << 'EOF'
set -e
printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -170,6 +170,5 @@ deb_dl_dir_export() {
ln -Pf "${p}" "${pc}" 2>/dev/null ||
cp -n "${p}" "${pc}"
done
- chown -R ${owner} "${pc}"
-EOSUDO
+EOF
}
--
2.53.0
privileges. By that, the files in the download dir are also owned by the
calling user, making the additional chown obsolete.
meta/classes-recipe/deb-dl-dir.bbclass | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 27d1fb24..3ea75996 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -153,7 +153,7 @@ deb_dl_dir_export() {
isar_debs="$(${SCRIPTSDIR}/lockrun.py -r -f '${REPO_ISAR_DIR}/isar.lock' -c \
"find '${REPO_ISAR_DIR}/${DISTRO}' -name '*.deb' -print")"
- flock "${pc}".lock sudo -Es << 'EOSUDO'
+ flock "${pc}".lock /bin/bash -s << 'EOF'
set -e
printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -170,6 +170,5 @@ deb_dl_dir_export() {
ln -Pf "${p}" "${pc}" 2>/dev/null ||
cp -n "${p}" "${pc}"
done
- chown -R ${owner} "${pc}"
-EOSUDO
+EOF
}
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
By that, we can make the trap more easily conditional.
.../isar-mmdebstrap/isar-mmdebstrap.inc | 24 +++++++++++--------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index da8bc52d..cf6c355c 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -110,6 +110,19 @@ def get_apt_auth_opts(d):
f"--setup-hook='upload \"{workdir}/apt-auth\" /etc/apt/auth.conf.d/isar.conf'"
return ''
+bootstrap_cleanup() {
+ [ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
+ && rm "${WORKDIR}/mmtmpdir"
+ [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \
+ && run_privileged umount $tmpdir/$base_apt_tmp
+ [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \
+ && run_privileged umount $tmpdir/base-apt
+ [ -d "$tmpdir" ] && run_privileged rm -rf --one-file-system $tmpdir
+ [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \
+ && run_privileged umount $base_apt_tmp \
+ && rm -rf --one-file-system $base_apt_tmp
+}
+
do_bootstrap[vardeps] += " \
DISTRO_APT_PREMIRRORS \
ISAR_ENABLE_COMPAT_ARCH \
@@ -206,16 +219,7 @@ do_bootstrap() {
# Cleanup mounts if fails
trap 'exit 1' INT HUP QUIT TERM ALRM USR1
- trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
- && rm "${WORKDIR}/mmtmpdir"; \
- [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \
- && run_privileged umount $tmpdir/$base_apt_tmp; \
- [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \
- && run_privileged umount $tmpdir/base-apt; \
- [ -d "$tmpdir" ] && run_privileged rm -rf --one-file-system $tmpdir; \
- [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \
- && run_privileged umount $base_apt_tmp \
- && rm -rf --one-file-system $base_apt_tmp' EXIT
+ trap 'bootstrap_cleanup' EXIT
# Create lock file so that it is owned by the user running the build (not root)
mkdir -p ${DEBDIR}
--
2.53.0
.../isar-mmdebstrap/isar-mmdebstrap.inc | 24 +++++++++++--------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index da8bc52d..cf6c355c 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -110,6 +110,19 @@ def get_apt_auth_opts(d):
f"--setup-hook='upload \"{workdir}/apt-auth\" /etc/apt/auth.conf.d/isar.conf'"
return ''
+bootstrap_cleanup() {
+ [ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
+ && rm "${WORKDIR}/mmtmpdir"
+ [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \
+ && run_privileged umount $tmpdir/$base_apt_tmp
+ [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \
+ && run_privileged umount $tmpdir/base-apt
+ [ -d "$tmpdir" ] && run_privileged rm -rf --one-file-system $tmpdir
+ [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \
+ && run_privileged umount $base_apt_tmp \
+ && rm -rf --one-file-system $base_apt_tmp
+}
+
do_bootstrap[vardeps] += " \
DISTRO_APT_PREMIRRORS \
ISAR_ENABLE_COMPAT_ARCH \
@@ -206,16 +219,7 @@ do_bootstrap() {
# Cleanup mounts if fails
trap 'exit 1' INT HUP QUIT TERM ALRM USR1
- trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
- && rm "${WORKDIR}/mmtmpdir"; \
- [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \
- && run_privileged umount $tmpdir/$base_apt_tmp; \
- [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \
- && run_privileged umount $tmpdir/base-apt; \
- [ -d "$tmpdir" ] && run_privileged rm -rf --one-file-system $tmpdir; \
- [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \
- && run_privileged umount $base_apt_tmp \
- && rm -rf --one-file-system $base_apt_tmp' EXIT
+ trap 'bootstrap_cleanup' EXIT
# Create lock file so that it is owned by the user running the build (not root)
mkdir -p ${DEBDIR}
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
As a preparation to enable rootless builds, we introduce wrappers for
common cases of privileged command execution. The wrappers are defined
in the base class where later on the executor dispatching will be
implemented as well.
The wrappers are introduced throughout the whole codebase and downstream
layers are also encouraged to use them to increase compatibility with
upcoming API changes.
RECIPE-API-CHANGELOG.md | 16 ++++
meta/classes-global/base.bbclass | 26 +++++-
meta/classes-recipe/deb-dl-dir.bbclass | 8 +-
meta/classes-recipe/dpkg-base.bbclass | 2 +-
meta/classes-recipe/dpkg.bbclass | 2 +-
.../image-account-extension.bbclass | 4 +-
.../image-locales-extension.bbclass | 4 +-
.../image-postproc-extension.bbclass | 30 +++----
meta/classes-recipe/image.bbclass | 14 +--
.../imagetypes_container.bbclass | 26 +++---
meta/classes-recipe/imagetypes_wic.bbclass | 4 +-
meta/classes-recipe/rootfs.bbclass | 89 ++++++++++---------
meta/classes-recipe/sbuild.bbclass | 10 +--
meta/classes-recipe/sdk.bbclass | 14 +--
meta/classes/sbom.bbclass | 2 +-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 12 +--
.../unittests/test_image_account_extension.py | 9 +-
17 files changed, 156 insertions(+), 116 deletions(-)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index c5962969..d0aa6e1a 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1061,3 +1061,19 @@ incorrect build results.
Changes in next
---------------
+
+### Execution of privileged commands
+
+When operations require higher privileges than those available to the build user,
+the following helper functions shall be used:
+
+**run_privileged**: Run a command as root while preserving the environment.
+
+**run_privileged_heredoc**: Execute commands provided via stdin in a root shell.
+
+**run_in_chroot**: Run a command within a chroot environment. The first argument
+specifies the rootfs path.
+
+Using these helpers instead of direct `sudo` invocations centralizes platform-specific
+privileged execution logic in `base.bbclass`. Direct use of `sudo` is discouraged
+in downstream layers.
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 70b4565b..d4dbbc3a 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -141,7 +141,7 @@ root_cleandirs() {
die "Could not remove $i, because subdir is mounted"
done
for i in $ROOT_CLEANDIRS_DIRS; do
- sudo rm -rf --one-file-system "$TMPDIR$i"
+ run_privileged rm -rf --one-file-system "$TMPDIR$i"
mkdir -p "$TMPDIR$i"
done
}
@@ -375,3 +375,27 @@ def deb_list_beautify(d, varname):
if stripped:
var_list.append(stripped)
return ', '.join(var_list)
+
+# Helpers for privileged execution. Only the non-underscore functions
+# shall be used outside of this class.
+
+def run_privileged_cmd(d):
+ cmd = 'sudo -E'
+ bb.debug(1, "privileged cmd: %s" % cmd)
+ return cmd
+
+RUN_PRIVILEGED_CMD := "${@run_privileged_cmd(d)}"
+
+run_privileged() {
+ ${RUN_PRIVILEGED_CMD} "$@"
+}
+
+run_privileged_heredoc() {
+ ${RUN_PRIVILEGED_CMD} /bin/bash -s "$@"
+}
+
+run_in_chroot() {
+ rootfs="$1"
+ shift
+ ${RUN_PRIVILEGED_CMD} chroot "$rootfs" "$@"
+}
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index e3f055c5..04fd6414 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -98,7 +98,7 @@ debsrc_download() {
dbg_pkgs_download() {
export rootfs="$1"
- apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+ dbg_pkgs=$(apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
-a "${DISTRO_ARCH}" packages \
"${rootfs}/var/cache/apt/archives" \
| awk '/^Package:/ {print $2}' \
@@ -110,7 +110,9 @@ dbg_pkgs_download() {
| grep "${DISTRO_ARCH}" \
| awk '!/Binary:/ {print $1}' \
| sort -u
- done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' --
+ done)
+
+ [ -z "${dbg_pkgs}" ] || run_in_chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install $@' -- ${dbg_pkgs}
}
deb_dl_dir_import() {
@@ -120,7 +122,7 @@ deb_dl_dir_import() {
export gid=$(id -g)
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
- sudo -Es << ' EOSUDO'
+ run_privileged_heredoc << ' EOSUDO'
index f7a12302..e5987554 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -161,7 +161,7 @@ def isar_export_build_settings(d):
dpkg_schroot_create_configs() {
schroot_create_configs
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
fstab_isarapt="${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} /isar-apt none rw,bind 0 0"
grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab}
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index c1c38184..dcdef487 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -129,5 +129,5 @@ dpkg_runbuild() {
deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
# Cleanup apt artifacts
- sudo rm -rf ${WORKDIR}/rootfs
+ run_privileged rm -rf ${WORKDIR}/rootfs
}
diff --git a/meta/classes-recipe/image-account-extension.bbclass b/meta/classes-recipe/image-account-extension.bbclass
index e874f3c7..de01484c 100644
--- a/meta/classes-recipe/image-account-extension.bbclass
+++ b/meta/classes-recipe/image-account-extension.bbclass
@@ -34,7 +34,7 @@ def image_create_groups(d: "DataSmart") -> None:
"""
entries = (d.getVar("GROUPS") or "").split()
rootfsdir = d.getVar("ROOTFSDIR")
- chroot = ["sudo", "-E", "chroot", rootfsdir]
+ chroot = run_privileged_cmd(d).split() + ["chroot", rootfsdir]
for entry in entries:
args = []
@@ -72,7 +72,7 @@ def image_create_users(d: "DataSmart") -> None:
entries = (d.getVar("USERS") or "").split()
rootfsdir = d.getVar("ROOTFSDIR")
- chroot = ["sudo", "-E", "chroot", rootfsdir]
+ chroot = run_privileged_cmd(d).split() + ["chroot", rootfsdir]
for entry in entries:
args = []
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index c90280aa..029caec7 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -29,7 +29,7 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download"
+ run_in_chroot '${ROOTFSDIR}' \
__EOF__
# Install configuration into image:
- sudo -E -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
localepurge_state='i'
if chroot '${ROOTFSDIR}' dpkg -s localepurge 2>/dev/null >&2
diff --git a/meta/classes-recipe/image-postproc-extension.bbclass b/meta/classes-recipe/image-postproc-extension.bbclass
index 43ab750c..59128c2a 100644
--- a/meta/classes-recipe/image-postproc-extension.bbclass
+++ b/meta/classes-recipe/image-postproc-extension.bbclass
@@ -17,19 +17,19 @@ update_etc_os_release() {
done
if [ -n "${OS_RELEASE_BUILD_ID}" ]; then
- sudo sed -i '/^BUILD_ID=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^BUILD_ID=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
if [ -n "${OS_RELEASE_VARIANT}" ]; then
- sudo sed -i '/^VARIANT=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^VARIANT=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
if [ -n "${OS_RELEASE_VARIANT_VERSION}" ]; then
- sudo sed -i '/^VARIANT_VERSION=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^VARIANT_VERSION=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "VARIANT_VERSION=\"${OS_RELEASE_VARIANT_VERSION}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
}
@@ -37,11 +37,11 @@ ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_configure"
image_postprocess_configure() {
# Configure root filesystem
if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
- sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
+ run_privileged install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
TARGET_DISTRO_CONFIG_SCRIPT="$(basename ${DISTRO_CONFIG_SCRIPT})"
- sudo chroot ${IMAGE_ROOTFS} "/$TARGET_DISTRO_CONFIG_SCRIPT" \
+ run_in_chroot ${IMAGE_ROOTFS} "/$TARGET_DISTRO_CONFIG_SCRIPT" \
"${MACHINE_SERIAL}" "${BAUDRATE_TTY}"
- sudo rm "${IMAGE_ROOTFS}/$TARGET_DISTRO_CONFIG_SCRIPT"
+ run_privileged rm "${IMAGE_ROOTFS}/$TARGET_DISTRO_CONFIG_SCRIPT"
fi
}
@@ -58,13 +58,13 @@ image_postprocess_machine_id() {
# systemd(1) takes care of recreating the machine-id on first boot
# for systemd < v247, set to empty string, else set to uninitialized
# (required if initramfs with ro root is used)
- SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
+ SYSTEMD_VERSION=$( run_in_chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
MACHINE_ID="uninitialized"
if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
MACHINE_ID=""
fi
- echo "$MACHINE_ID" | sudo chroot ${IMAGE_ROOTFS} tee /etc/machine-id
- sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
+ echo "$MACHINE_ID" | run_in_chroot ${IMAGE_ROOTFS} tee /etc/machine-id
+ run_privileged rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
}
ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_sshd_key_regen"
@@ -82,13 +82,13 @@ image_postprocess_sshd_key_regen() {
ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_disable_systemd_firstboot"
image_postprocess_disable_systemd_firstboot() {
- SYSTEMD_VERSION=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
+ SYSTEMD_VERSION=$(run_in_chroot '${ROOTFSDIR}' dpkg-query \
--showformat='${source:Upstream-Version}' \
--show systemd || echo "0" )
if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then
- sudo chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
- if ! cmd_output=$(sudo chroot '${ROOTFSDIR}' systemd-firstboot \
+ run_in_chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
+ if ! cmd_output=$(run_in_chroot '${ROOTFSDIR}' systemd-firstboot \
--prompt --welcome=false </dev/null 2>/dev/null); then
bbwarn "Your image is not configured completely according to systemd-firstboot."
bbwarn "It prompted: \"${cmd_output}\""
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 866df68a..9fcdda48 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -364,7 +364,7 @@ get_build_id() {
ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
image_configure_fstab[weight] = "2"
image_configure_fstab() {
- sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
+ run_privileged tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
# Begin /etc/fstab
proc /proc proc nosuid,noexec,nodev 0 0
sysfs /sys sysfs nosuid,noexec,nodev 0 0
@@ -392,7 +392,7 @@ do_copy_boot_files() {
kernel="$(realpath -q '${IMAGE_ROOTFS}'/boot/vmlinu[xz])"
fi
if [ -f "$kernel" ]; then
- sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
+ run_privileged cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
fi
for file in ${DTB_FILES}; do
@@ -448,7 +448,7 @@ def apt_list_files(d):
IMAGE_LISTS = "${@ ' '.join(apt_list_files(d)) }"
do_rootfs_finalize() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if [ -e "${ROOTFSDIR}/chroot-setup.sh" ]; then
@@ -474,14 +474,14 @@ EOSUDO
# Sometimes qemu-user-static generates coredumps in chroot, move them
# to work temporary directory and inform user about it.
- for f in $(sudo find ${ROOTFSDIR} -type f -name *.core -exec file --mime-type {} \; | grep 'application/x-coredump' | cut -d: -f1); do
- sudo mv "${f}" "${WORKDIR}/temp/"
+ for f in $(run_privileged find ${ROOTFSDIR} -type f -name *.core -exec file --mime-type {} \; | grep 'application/x-coredump' | cut -d: -f1); do
+ run_privileged mv "${f}" "${WORKDIR}/temp/"
bbwarn "found core dump in rootfs, check it in ${WORKDIR}/temp/${f##*/}"
done
# Set same time-stamps to the newly generated file/folders in the
# rootfs image for the purpose of reproducible builds.
- sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
+ run_privileged find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
-exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';'
}
do_rootfs_finalize[network] = "${TASK_USE_SUDO}"
@@ -518,7 +518,7 @@ do_rootfs_quality_check() {
;;
esac
done
- found=$( sudo find ${ROOTFSDIR} -type f -newer $rootfs_install_stamp $args )
+ found=$( run_privileged find ${ROOTFSDIR} -type f -newer $rootfs_install_stamp $args )
if [ -n "$found" ]; then
bbwarn "Files changed after package install. The following files seem"
bbwarn "to have changed where they probably should not have."
diff --git a/meta/classes-recipe/imagetypes_container.bbclass b/meta/classes-recipe/imagetypes_container.bbclass
index fba15503..fb1d0cdf 100644
--- a/meta/classes-recipe/imagetypes_container.bbclass
+++ b/meta/classes-recipe/imagetypes_container.bbclass
@@ -37,38 +37,38 @@ do_containerize() {
# prepare OCI container image skeleton
bbdebug 1 "prepare OCI container image skeleton"
- sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
- sudo umoci init --layout "${oci_img_dir}"
- sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ run_privileged rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
+ run_privileged umoci init --layout "${oci_img_dir}"
+ run_privileged umoci new --image "${oci_img_dir}:${empty_tag}"
if [ -n "${cmd}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.cmd="${cmd}"
fi
if [ -n "${entrypoint}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.entrypoint="${entrypoint}"
fi
if [ -n "${path}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.env="PATH=${path}"
fi
- sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci unpack --image "${oci_img_dir}:${empty_tag}" \
"${oci_img_dir}_unpacked"
# add root filesystem as the flesh of the skeleton
- sudo cp --reflink=auto -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ run_privileged cp --reflink=auto -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
# clean-up temporary files
- sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
+ run_privileged find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
# pack container image
bbdebug 1 "pack container image"
- sudo umoci repack --image "${oci_img_dir}:${tag}" \
+ run_privileged umoci repack --image "${oci_img_dir}:${tag}" \
"${oci_img_dir}_unpacked"
- sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
- sudo rm -rf "${oci_img_dir}_unpacked"
+ run_privileged umoci remove --image "${oci_img_dir}:${empty_tag}"
+ run_privileged rm -rf "${oci_img_dir}_unpacked"
# no root needed anymore
- sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+ run_privileged chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
}
convert_container() {
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index dd6c501d..8b048dc7 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -193,8 +193,8 @@ generate_wic_image() {
fi
EOIMAGER
- sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
- sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+ run_privileged chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
+ run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
rm -rf ${IMAGE_ROOTFS}/../pseudo
cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index b201b97d..440786b5 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -136,7 +136,7 @@ rootfs_cmd() {
rootfs_do_mounts[weight] = "3"
rootfs_do_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mountpoint -q '${ROOTFSDIR}/dev' || \
( mount -o bind,private /dev '${ROOTFSDIR}/dev' &&
@@ -182,7 +182,7 @@ EOSUDO
}
rootfs_do_umounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if mountpoint -q '${ROOTFSDIR}/isar-apt'; then
umount '${ROOTFSDIR}/isar-apt'
@@ -225,7 +225,7 @@ rootfs_do_qemu() {
if [ '${@repr(d.getVar('ROOTFS_ARCH') == d.getVar('HOST_ARCH'))}' = 'False' ]
then
test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \
- sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
+ run_privileged cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
fi
}
@@ -240,16 +240,16 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}"
rootfs_prepare[weight] = "25"
rootfs_prepare(){
- sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
# setup chroot
- sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
+ run_privileged "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
}
ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt"
rootfs_configure_isar_apt[weight] = "2"
rootfs_configure_isar_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
@@ -270,7 +270,7 @@ EOSUDO
ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_apt"
rootfs_configure_apt[weight] = "2"
rootfs_configure_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p '${ROOTFSDIR}/etc/apt/apt.conf.d'
@@ -293,7 +293,7 @@ ROOTFS_CONFIGURE_COMMAND += "rootfs_disable_initrd_generation"
rootfs_disable_initrd_generation[weight] = "1"
rootfs_disable_initrd_generation() {
# fully disable initrd generation
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p "${ROOTFSDIR}${ROOTFS_STUBS_DIR}"
@@ -310,7 +310,7 @@ rootfs_install_pkgs_update[weight] = "5"
rootfs_install_pkgs_update[isar-apt-lock] = "acquire-before"
rootfs_install_pkgs_update[network] = "${TASK_USE_NETWORK_AND_SUDO}"
rootfs_install_pkgs_update() {
- sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
+ run_in_chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
if [ "${@repr(bb.utils.to_boolean(d.getVar('BB_NO_NETWORK')))}" != "True" ]
then
if [ -L "${ROOTFSDIR}/etc/resolv.conf" ]; then
- sudo unlink "${ROOTFSDIR}/etc/resolv.conf"
+ run_privileged unlink "${ROOTFSDIR}/etc/resolv.conf"
fi
- sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
+ run_privileged cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
fi
}
@@ -358,7 +358,7 @@ rootfs_export_package_cache() {
ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
rootfs_install_clean_files[weight] = "2"
rootfs_install_clean_files() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
for clean_file in ${ROOTFS_CLEAN_FILES}; do
rm -f "${ROOTFSDIR}/$clean_file"
done
@@ -370,14 +370,14 @@ rootfs_install_pkgs_install[weight] = "8000"
rootfs_install_pkgs_install[progress] = "custom:rootfs_progress.PkgsInstallProgressHandler"
rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
rootfs_install_pkgs_install() {
- sudo -E chroot "${ROOTFSDIR}" \
+ run_in_chroot "${ROOTFSDIR}" \
/usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
}
ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
rootfs_restore_initrd_tooling[weight] = "1"
rootfs_restore_initrd_tooling() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar-stubs"
rm -rf "${ROOTFSDIR}${ROOTFS_STUBS_DIR}"
@@ -386,8 +386,8 @@ EOSUDO
ROOTFS_INSTALL_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-initrd', '', 'rootfs_clear_initrd_symlinks', d)}"
rootfs_clear_initrd_symlinks() {
- sudo rm -f ${ROOTFSDIR}/initrd.img
- sudo rm -f ${ROOTFSDIR}/initrd.img.old
+ run_privileged rm -f ${ROOTFSDIR}/initrd.img
+ run_privileged rm -f ${ROOTFSDIR}/initrd.img.old
}
do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
@@ -437,21 +437,21 @@ do_cache_deb_src[network] = "${TASK_USE_SUDO}"
do_cache_deb_src() {
if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
fi
rootfs_install_resolvconf
# Note: Isar updates the apt state information(apt-get update) only once during bootstrap and
# relies on that through out the build. Copy that state information instead of apt-get update
# which generates a new state from upstream.
- sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
- sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf
if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
fi
}
@@ -459,21 +459,21 @@ ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache
cache_dbg_pkgs() {
if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
fi
rootfs_install_resolvconf
# Note: Isar updates the apt state information(apt-get update) only once during bootstrap and
# relies on that through out the build. Copy that state information instead of apt-get update
# which generates a new state from upstream.
- sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
dbg_pkgs_download ${ROOTFSDIR}
- sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf
if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
fi
}
@@ -482,17 +482,17 @@ ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
rootfs_postprocess_clean_package_cache() {
- sudo -E chroot '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
/usr/bin/apt-get clean
- sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
+ run_privileged rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
# remove apt-cache folder itself (required in case rootfs is provided by sstate cache)
- sudo rm -rf "${ROOTFSDIR}/var/cache/apt/archives"
+ run_privileged rm -rf "${ROOTFSDIR}/var/cache/apt/archives"
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-log-files', 'rootfs_postprocess_clean_log_files', '', d)}"
rootfs_postprocess_clean_log_files() {
# Delete log files that are not owned by packages
- sudo -E chroot '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
/usr/bin/find /var/log/ -type f \
-exec sh -c '! dpkg -S {} > /dev/null 2>&1' ';' \
-exec rm -f {} ';'
@@ -501,32 +501,32 @@ rootfs_postprocess_clean_log_files() {
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-debconf-cache', 'rootfs_postprocess_clean_debconf_cache', '', d)}"
rootfs_postprocess_clean_debconf_cache() {
# Delete debconf cache files
- sudo rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
+ run_privileged rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-pycache', 'rootfs_postprocess_clean_pycache', '', d)}"
rootfs_postprocess_clean_pycache() {
- sudo find ${ROOTFSDIR}/usr -type f -name '*.pyc' -delete -print
- sudo find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete -print
+ run_privileged find ${ROOTFSDIR}/usr -type f -name '*.pyc' -delete -print
+ run_privileged find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete -print
}
ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_ldconfig_cache"
rootfs_postprocess_clean_ldconfig_cache() {
# the ldconfig aux-cache is not portable and breaks reproducability
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845034#49
- sudo rm -f ${ROOTFSDIR}/var/cache/ldconfig/aux-cache
+ run_privileged rm -f ${ROOTFSDIR}/var/cache/ldconfig/aux-cache
}
ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_tmp"
rootfs_postprocess_clean_tmp() {
# /tmp is by definition non persistent across boots
- sudo rm -rf "${ROOTFSDIR}/tmp/"*
+ run_privileged rm -rf "${ROOTFSDIR}/tmp/"*
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
rootfs_generate_manifest () {
mkdir -p ${ROOTFS_MANIFEST_DEPLOY_DIR}
- sudo -E chroot --userspec=$(id -u):$(id -g) '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
dpkg-query -W -f \
'${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' > \
'${ROOTFS_MANIFEST_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.manifest
@@ -542,7 +542,7 @@ rootfs_export_dpkg_status() {
ROOTFS_POSTPROCESS_COMMAND += "rootfs_cleanup_isar_apt"
rootfs_cleanup_isar_apt[weight] = "2"
rootfs_cleanup_isar_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt"
@@ -553,7 +553,7 @@ EOSUDO
ROOTFS_POSTPROCESS_COMMAND += "${@'rootfs_cleanup_base_apt' if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else ''}"
rootfs_cleanup_base_apt[weight] = "2"
rootfs_cleanup_base_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/"*base-apt.list
EOSUDO
@@ -561,12 +561,12 @@ EOSUDO
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'populate-systemd-preset', 'image_postprocess_populate_systemd_preset', '', d)}"
image_postprocess_populate_systemd_preset() {
- SYSTEMD_INSTALLED=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
+ SYSTEMD_INSTALLED=$(run_in_chroot '${ROOTFSDIR}' dpkg-query \
--showformat='${db:Status-Status}' \
--show systemd || echo "" )
if (test "$SYSTEMD_INSTALLED" = "installed"); then
- sudo chroot '${ROOTFSDIR}' systemctl preset-all --preset-mode="enable-only"
+ run_in_chroot '${ROOTFSDIR}' systemctl preset-all --preset-mode="enable-only"
fi
}
@@ -626,7 +626,7 @@ rootfs_generate_initramfs() {
mods_total="$(find ${ROOTFSDIR}/usr/lib/modules/$kernel_version -type f -name '*.ko*' | wc -l)"
echo "Total number of modules: $mods_total"
echo "Generating initrd for kernel version: $kernel_version"
- sudo -E chroot "${ROOTFSDIR}" sh -ec ' \
+ run_in_chroot "${ROOTFSDIR}" sh -ec ' \
${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}; \
find /boot -name "initrd.img-$kernel_version*" -exec install --mode 0644 {} /isar-work/initrd.img \; \
'
@@ -664,11 +664,12 @@ rootfs_install_sstate_prepare() {
# so we use some mount magic to prevent that
mkdir -p ${WORKDIR}/mnt/rootfs
trap 'rmdir ${WORKDIR}/mnt/rootfs ${WORKDIR}/mnt' EXIT
- sudo mount -o bind,private '${WORKDIR}/rootfs' '${WORKDIR}/mnt/rootfs' -o ro
+
+ run_privileged mount -o bind,private '${WORKDIR}/rootfs' '${WORKDIR}/mnt/rootfs' -o ro
lopts="--one-file-system --exclude=var/cache/apt/archives"
- sudo tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
- sudo umount ${WORKDIR}/mnt/rootfs
- sudo chown $(id -u):$(id -g) rootfs.tar
+ run_privileged tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
+ run_privileged umount ${WORKDIR}/mnt/rootfs
+ run_privileged chown $(id -u):$(id -g) rootfs.tar
}
do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
@@ -677,7 +678,7 @@ rootfs_install_sstate_finalize() {
# - after building the rootfs, the tar won't be there, but we also don't need to unpack
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
- sudo tar -C ${WORKDIR} -xpf rootfs.tar ${SSTATE_TAR_ATTR_FLAGS}
+ run_privileged tar -C ${WORKDIR} -xpf rootfs.tar ${SSTATE_TAR_ATTR_FLAGS}
rm rootfs.tar
fi
}
diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass
index 95dadee3..d9ccce7f 100644
--- a/meta/classes-recipe/sbuild.bbclass
+++ b/meta/classes-recipe/sbuild.bbclass
@@ -20,7 +20,7 @@ SCHROOT_LOCKFILE = "/tmp/schroot.lock"
schroot_create_configs() {
mkdir -p "${TMPDIR}/schroot-overlay"
echo "Creating ${SCHROOT_CONF_FILE}"
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
cat << EOF > "${SCHROOT_CONF_FILE}"
@@ -59,7 +59,7 @@ EOSUDO
schroot_delete_configs() {
(flock -x 9
set -e
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if [ -d "${SBUILD_CONF_DIR}" ]; then
echo "Removing ${SBUILD_CONF_DIR}"
@@ -101,7 +101,7 @@ sbuild_export() {
}
insert_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
for mp in ${SCHROOT_MOUNTS}; do
FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind,private 0 0"
@@ -112,7 +112,7 @@ EOSUDO
}
remove_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
for mp in ${SCHROOT_MOUNTS}; do
FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind,private 0 0"
@@ -123,7 +123,7 @@ EOSUDO
schroot_configure_ccache() {
mkdir -p "${CCACHE_DIR}"
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
diff --git a/meta/classes-recipe/sdk.bbclass b/meta/classes-recipe/sdk.bbclass
index 6f09b5f6..16165792 100644
--- a/meta/classes-recipe/sdk.bbclass
+++ b/meta/classes-recipe/sdk.bbclass
@@ -69,12 +69,12 @@ ROOTFS_POSTPROCESS_COMMAND:remove = "${@'rootfs_cleanup_isar_apt' if bb.utils.to
ROOTFS_CONFIGURE_COMMAND:append:class-sdk = " ${@'rootfs_configure_isar_apt_dir' if bb.utils.to_boolean(d.getVar('SDK_INCLUDE_ISAR_APT')) else ''}"
rootfs_configure_isar_apt_dir() {
# Copy isar-apt instead of mounting:
- sudo cp -Trpfx --reflink=auto ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
+ run_privileged cp -Trpfx --reflink=auto ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
}
ROOTFS_POSTPROCESS_COMMAND:prepend:class-sdk = "sdkchroot_configscript "
sdkchroot_configscript () {
- sudo chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+ run_in_chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
}
ROOTFS_POSTPROCESS_COMMAND:append:class-sdk = " sdkchroot_finalize"
@@ -83,7 +83,7 @@ sdkchroot_finalize() {
rootfs_do_umounts
# Remove setup scripts
- sudo rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
+ run_privileged rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
# Make all links relative
for link in $(find ${ROOTFSDIR}/ -type l); do
@@ -95,16 +95,16 @@ sdkchroot_finalize() {
new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${ROOTFSDIR}${target})
# remove first to allow rewriting directory links
- sudo rm $link
- sudo ln -s $new_target $link
+ run_privileged rm $link
+ run_privileged ln -s $new_target $link
fi
done
# Set up sysroot wrapper
for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
for tool in $(find ${ROOTFSDIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
- sudo mv "${tool}" "${tool}.bin"
- sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+ run_privileged mv "${tool}" "${tool}.bin"
+ run_privileged ln -sf gcc-sysroot-wrapper.sh ${tool}
done
done
}
diff --git a/meta/classes/sbom.bbclass b/meta/classes/sbom.bbclass
index b220f3d9..b4fcddaa 100644
--- a/meta/classes/sbom.bbclass
+++ b/meta/classes/sbom.bbclass
@@ -41,7 +41,7 @@ def sbom_doc_uuid(d):
d.setVar("SBOM_DOCUMENT_UUID", generate_document_uuid(d))
generate_sbom() {
- sudo mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
+ run_privileged mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
bwrap \
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index f21a6164..da8bc52d 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -209,19 +209,19 @@ do_bootstrap() {
+ && run_privileged umount $tmpdir/$base_apt_tmp; \
- [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \
- sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
+ run_privileged TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
$arch_param \
--mode=unshare \
${MMHOOKS} \
@@ -254,7 +254,7 @@ do_bootstrap() {
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
- sudo rm -rf --one-file-system "${WORKDIR}/dl_dir"
+ run_privileged rm -rf --one-file-system "${WORKDIR}/dl_dir"
fi
}
addtask bootstrap before do_build after do_generate_keyrings
diff --git a/testsuite/unittests/test_image_account_extension.py b/testsuite/unittests/test_image_account_extension.py
index f78aa7f8..ff0e47e0 100644
--- a/testsuite/unittests/test_image_account_extension.py
+++ b/testsuite/unittests/test_image_account_extension.py
@@ -54,9 +54,8 @@ class TestImageAccountExtensionImageCreateUsers(
image_create_users(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/useradd',
@@ -136,9 +135,8 @@ class TestImageAccountExtensionImageCreateGroups(
image_create_groups(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/groupadd',
@@ -164,9 +162,8 @@ class TestImageAccountExtensionImageCreateGroups(
image_create_groups(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/groupmod',
--
2.53.0
common cases of privileged command execution. The wrappers are defined
in the base class where later on the executor dispatching will be
implemented as well.
The wrappers are introduced throughout the whole codebase and downstream
layers are also encouraged to use them to increase compatibility with
upcoming API changes.
RECIPE-API-CHANGELOG.md | 16 ++++
meta/classes-global/base.bbclass | 26 +++++-
meta/classes-recipe/deb-dl-dir.bbclass | 8 +-
meta/classes-recipe/dpkg-base.bbclass | 2 +-
meta/classes-recipe/dpkg.bbclass | 2 +-
.../image-account-extension.bbclass | 4 +-
.../image-locales-extension.bbclass | 4 +-
.../image-postproc-extension.bbclass | 30 +++----
meta/classes-recipe/image.bbclass | 14 +--
.../imagetypes_container.bbclass | 26 +++---
meta/classes-recipe/imagetypes_wic.bbclass | 4 +-
meta/classes-recipe/rootfs.bbclass | 89 ++++++++++---------
meta/classes-recipe/sbuild.bbclass | 10 +--
meta/classes-recipe/sdk.bbclass | 14 +--
meta/classes/sbom.bbclass | 2 +-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 12 +--
.../unittests/test_image_account_extension.py | 9 +-
17 files changed, 156 insertions(+), 116 deletions(-)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index c5962969..d0aa6e1a 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1061,3 +1061,19 @@ incorrect build results.
Changes in next
---------------
+
+### Execution of privileged commands
+
+When operations require higher privileges than those available to the build user,
+the following helper functions shall be used:
+
+**run_privileged**: Run a command as root while preserving the environment.
+
+**run_privileged_heredoc**: Execute commands provided via stdin in a root shell.
+
+**run_in_chroot**: Run a command within a chroot environment. The first argument
+specifies the rootfs path.
+
+Using these helpers instead of direct `sudo` invocations centralizes platform-specific
+privileged execution logic in `base.bbclass`. Direct use of `sudo` is discouraged
+in downstream layers.
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 70b4565b..d4dbbc3a 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -141,7 +141,7 @@ root_cleandirs() {
die "Could not remove $i, because subdir is mounted"
done
for i in $ROOT_CLEANDIRS_DIRS; do
- sudo rm -rf --one-file-system "$TMPDIR$i"
+ run_privileged rm -rf --one-file-system "$TMPDIR$i"
mkdir -p "$TMPDIR$i"
done
}
@@ -375,3 +375,27 @@ def deb_list_beautify(d, varname):
if stripped:
var_list.append(stripped)
return ', '.join(var_list)
+
+# Helpers for privileged execution. Only the non-underscore functions
+# shall be used outside of this class.
+
+def run_privileged_cmd(d):
+ cmd = 'sudo -E'
+ bb.debug(1, "privileged cmd: %s" % cmd)
+ return cmd
+
+RUN_PRIVILEGED_CMD := "${@run_privileged_cmd(d)}"
+
+run_privileged() {
+ ${RUN_PRIVILEGED_CMD} "$@"
+}
+
+run_privileged_heredoc() {
+ ${RUN_PRIVILEGED_CMD} /bin/bash -s "$@"
+}
+
+run_in_chroot() {
+ rootfs="$1"
+ shift
+ ${RUN_PRIVILEGED_CMD} chroot "$rootfs" "$@"
+}
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index e3f055c5..04fd6414 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -98,7 +98,7 @@ debsrc_download() {
dbg_pkgs_download() {
export rootfs="$1"
- apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
+ dbg_pkgs=$(apt-ftparchive --md5=no --sha1=no --sha256=no --sha512=no \
-a "${DISTRO_ARCH}" packages \
"${rootfs}/var/cache/apt/archives" \
| awk '/^Package:/ {print $2}' \
@@ -110,7 +110,9 @@ dbg_pkgs_download() {
| grep "${DISTRO_ARCH}" \
| awk '!/Binary:/ {print $1}' \
| sort -u
- done | xargs -r sudo -E chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install "$@"' --
+ done)
+
+ [ -z "${dbg_pkgs}" ] || run_in_chroot ${rootfs} sh -c '/usr/bin/apt-get -y --download-only install $@' -- ${dbg_pkgs}
}
deb_dl_dir_import() {
@@ -120,7 +122,7 @@ deb_dl_dir_import() {
export gid=$(id -g)
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
+ run_privileged_heredoc << ' EOSUDO'
mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
EOSUDO
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
EOSUDO
index f7a12302..e5987554 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -161,7 +161,7 @@ def isar_export_build_settings(d):
dpkg_schroot_create_configs() {
schroot_create_configs
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
fstab_isarapt="${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO} /isar-apt none rw,bind 0 0"
grep -qxF "${fstab_isarapt}" ${sbuild_fstab} || echo "${fstab_isarapt}" >> ${sbuild_fstab}
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index c1c38184..dcdef487 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -129,5 +129,5 @@ dpkg_runbuild() {
deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
# Cleanup apt artifacts
- sudo rm -rf ${WORKDIR}/rootfs
+ run_privileged rm -rf ${WORKDIR}/rootfs
}
diff --git a/meta/classes-recipe/image-account-extension.bbclass b/meta/classes-recipe/image-account-extension.bbclass
index e874f3c7..de01484c 100644
--- a/meta/classes-recipe/image-account-extension.bbclass
+++ b/meta/classes-recipe/image-account-extension.bbclass
@@ -34,7 +34,7 @@ def image_create_groups(d: "DataSmart") -> None:
"""
entries = (d.getVar("GROUPS") or "").split()
rootfsdir = d.getVar("ROOTFSDIR")
- chroot = ["sudo", "-E", "chroot", rootfsdir]
+ chroot = run_privileged_cmd(d).split() + ["chroot", rootfsdir]
for entry in entries:
args = []
@@ -72,7 +72,7 @@ def image_create_users(d: "DataSmart") -> None:
entries = (d.getVar("USERS") or "").split()
rootfsdir = d.getVar("ROOTFSDIR")
- chroot = ["sudo", "-E", "chroot", rootfsdir]
+ chroot = run_privileged_cmd(d).split() + ["chroot", rootfsdir]
for entry in entries:
args = []
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
index c90280aa..029caec7 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -29,7 +29,7 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download"
image_install_localepurge_download[weight] = "40"
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
- sudo -E chroot '${ROOTFSDIR}' \
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
+ run_in_chroot '${ROOTFSDIR}' \
/usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only localepurge
}
@@ -60,7 +60,7 @@ ${@get_nopurge(d)}
}
__EOF__
# Install configuration into image:
- sudo -E -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
localepurge_state='i'
if chroot '${ROOTFSDIR}' dpkg -s localepurge 2>/dev/null >&2
diff --git a/meta/classes-recipe/image-postproc-extension.bbclass b/meta/classes-recipe/image-postproc-extension.bbclass
index 43ab750c..59128c2a 100644
--- a/meta/classes-recipe/image-postproc-extension.bbclass
+++ b/meta/classes-recipe/image-postproc-extension.bbclass
@@ -17,19 +17,19 @@ update_etc_os_release() {
done
if [ -n "${OS_RELEASE_BUILD_ID}" ]; then
- sudo sed -i '/^BUILD_ID=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^BUILD_ID=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "BUILD_ID=\"${OS_RELEASE_BUILD_ID}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
if [ -n "${OS_RELEASE_VARIANT}" ]; then
- sudo sed -i '/^VARIANT=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^VARIANT=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "VARIANT=\"${OS_RELEASE_VARIANT}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
if [ -n "${OS_RELEASE_VARIANT_VERSION}" ]; then
- sudo sed -i '/^VARIANT_VERSION=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged sed -i '/^VARIANT_VERSION=.*/d' '${IMAGE_ROOTFS}/etc/os-release'
echo "VARIANT_VERSION=\"${OS_RELEASE_VARIANT_VERSION}\"" | \
- sudo tee -a '${IMAGE_ROOTFS}/etc/os-release'
+ run_privileged tee -a '${IMAGE_ROOTFS}/etc/os-release'
fi
}
@@ -37,11 +37,11 @@ ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_configure"
image_postprocess_configure() {
# Configure root filesystem
if [ -n "${DISTRO_CONFIG_SCRIPT}" ]; then
- sudo install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
+ run_privileged install -m 755 "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}" "${IMAGE_ROOTFS}"
TARGET_DISTRO_CONFIG_SCRIPT="$(basename ${DISTRO_CONFIG_SCRIPT})"
- sudo chroot ${IMAGE_ROOTFS} "/$TARGET_DISTRO_CONFIG_SCRIPT" \
+ run_in_chroot ${IMAGE_ROOTFS} "/$TARGET_DISTRO_CONFIG_SCRIPT" \
"${MACHINE_SERIAL}" "${BAUDRATE_TTY}"
- sudo rm "${IMAGE_ROOTFS}/$TARGET_DISTRO_CONFIG_SCRIPT"
+ run_privileged rm "${IMAGE_ROOTFS}/$TARGET_DISTRO_CONFIG_SCRIPT"
fi
}
@@ -58,13 +58,13 @@ image_postprocess_machine_id() {
# systemd(1) takes care of recreating the machine-id on first boot
# for systemd < v247, set to empty string, else set to uninitialized
# (required if initramfs with ro root is used)
- SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
+ SYSTEMD_VERSION=$( run_in_chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
MACHINE_ID="uninitialized"
if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
MACHINE_ID=""
fi
- echo "$MACHINE_ID" | sudo chroot ${IMAGE_ROOTFS} tee /etc/machine-id
- sudo rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
+ echo "$MACHINE_ID" | run_in_chroot ${IMAGE_ROOTFS} tee /etc/machine-id
+ run_privileged rm -f '${IMAGE_ROOTFS}/var/lib/dbus/machine-id'
}
ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_sshd_key_regen"
@@ -82,13 +82,13 @@ image_postprocess_sshd_key_regen() {
ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_disable_systemd_firstboot"
image_postprocess_disable_systemd_firstboot() {
- SYSTEMD_VERSION=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
+ SYSTEMD_VERSION=$(run_in_chroot '${ROOTFSDIR}' dpkg-query \
--showformat='${source:Upstream-Version}' \
--show systemd || echo "0" )
if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then
- sudo chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
- if ! cmd_output=$(sudo chroot '${ROOTFSDIR}' systemd-firstboot \
+ run_in_chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
+ if ! cmd_output=$(run_in_chroot '${ROOTFSDIR}' systemd-firstboot \
--prompt --welcome=false </dev/null 2>/dev/null); then
bbwarn "Your image is not configured completely according to systemd-firstboot."
bbwarn "It prompted: \"${cmd_output}\""
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 866df68a..9fcdda48 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -364,7 +364,7 @@ get_build_id() {
ROOTFS_CONFIGURE_COMMAND += "image_configure_fstab"
image_configure_fstab[weight] = "2"
image_configure_fstab() {
- sudo tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
+ run_privileged tee '${IMAGE_ROOTFS}/etc/fstab' << EOF
# Begin /etc/fstab
proc /proc proc nosuid,noexec,nodev 0 0
sysfs /sys sysfs nosuid,noexec,nodev 0 0
@@ -392,7 +392,7 @@ do_copy_boot_files() {
kernel="$(realpath -q '${IMAGE_ROOTFS}'/boot/vmlinu[xz])"
fi
if [ -f "$kernel" ]; then
- sudo cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
+ run_privileged cat "$kernel" > "${DEPLOYDIR}/${KERNEL_IMAGE}"
fi
for file in ${DTB_FILES}; do
@@ -448,7 +448,7 @@ def apt_list_files(d):
IMAGE_LISTS = "${@ ' '.join(apt_list_files(d)) }"
do_rootfs_finalize() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if [ -e "${ROOTFSDIR}/chroot-setup.sh" ]; then
@@ -474,14 +474,14 @@ EOSUDO
# Sometimes qemu-user-static generates coredumps in chroot, move them
# to work temporary directory and inform user about it.
- for f in $(sudo find ${ROOTFSDIR} -type f -name *.core -exec file --mime-type {} \; | grep 'application/x-coredump' | cut -d: -f1); do
- sudo mv "${f}" "${WORKDIR}/temp/"
+ for f in $(run_privileged find ${ROOTFSDIR} -type f -name *.core -exec file --mime-type {} \; | grep 'application/x-coredump' | cut -d: -f1); do
+ run_privileged mv "${f}" "${WORKDIR}/temp/"
bbwarn "found core dump in rootfs, check it in ${WORKDIR}/temp/${f##*/}"
done
# Set same time-stamps to the newly generated file/folders in the
# rootfs image for the purpose of reproducible builds.
- sudo find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
+ run_privileged find ${ROOTFSDIR} -newermt "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \
-exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';'
}
do_rootfs_finalize[network] = "${TASK_USE_SUDO}"
@@ -518,7 +518,7 @@ do_rootfs_quality_check() {
;;
esac
done
- found=$( sudo find ${ROOTFSDIR} -type f -newer $rootfs_install_stamp $args )
+ found=$( run_privileged find ${ROOTFSDIR} -type f -newer $rootfs_install_stamp $args )
if [ -n "$found" ]; then
bbwarn "Files changed after package install. The following files seem"
bbwarn "to have changed where they probably should not have."
diff --git a/meta/classes-recipe/imagetypes_container.bbclass b/meta/classes-recipe/imagetypes_container.bbclass
index fba15503..fb1d0cdf 100644
--- a/meta/classes-recipe/imagetypes_container.bbclass
+++ b/meta/classes-recipe/imagetypes_container.bbclass
@@ -37,38 +37,38 @@ do_containerize() {
# prepare OCI container image skeleton
bbdebug 1 "prepare OCI container image skeleton"
- sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
- sudo umoci init --layout "${oci_img_dir}"
- sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ run_privileged rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
+ run_privileged umoci init --layout "${oci_img_dir}"
+ run_privileged umoci new --image "${oci_img_dir}:${empty_tag}"
if [ -n "${cmd}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.cmd="${cmd}"
fi
if [ -n "${entrypoint}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.entrypoint="${entrypoint}"
fi
if [ -n "${path}" ]; then
- sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci config --image "${oci_img_dir}:${empty_tag}" \
--config.env="PATH=${path}"
fi
- sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ run_privileged umoci unpack --image "${oci_img_dir}:${empty_tag}" \
"${oci_img_dir}_unpacked"
# add root filesystem as the flesh of the skeleton
- sudo cp --reflink=auto -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ run_privileged cp --reflink=auto -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
# clean-up temporary files
- sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
+ run_privileged find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
# pack container image
bbdebug 1 "pack container image"
- sudo umoci repack --image "${oci_img_dir}:${tag}" \
+ run_privileged umoci repack --image "${oci_img_dir}:${tag}" \
"${oci_img_dir}_unpacked"
- sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
- sudo rm -rf "${oci_img_dir}_unpacked"
+ run_privileged umoci remove --image "${oci_img_dir}:${empty_tag}"
+ run_privileged rm -rf "${oci_img_dir}_unpacked"
# no root needed anymore
- sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+ run_privileged chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
}
convert_container() {
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index dd6c501d..8b048dc7 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -193,8 +193,8 @@ generate_wic_image() {
fi
EOIMAGER
- sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
- sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+ run_privileged chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
+ run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
rm -rf ${IMAGE_ROOTFS}/../pseudo
cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index b201b97d..440786b5 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -136,7 +136,7 @@ rootfs_cmd() {
rootfs_do_mounts[weight] = "3"
rootfs_do_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mountpoint -q '${ROOTFSDIR}/dev' || \
( mount -o bind,private /dev '${ROOTFSDIR}/dev' &&
@@ -182,7 +182,7 @@ EOSUDO
}
rootfs_do_umounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if mountpoint -q '${ROOTFSDIR}/isar-apt'; then
umount '${ROOTFSDIR}/isar-apt'
@@ -225,7 +225,7 @@ rootfs_do_qemu() {
if [ '${@repr(d.getVar('ROOTFS_ARCH') == d.getVar('HOST_ARCH'))}' = 'False' ]
then
test -e '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static' || \
- sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
+ run_privileged cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${ROOTFSDIR}/usr/bin/qemu-${QEMU_ARCH}-static'
fi
}
@@ -240,16 +240,16 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}"
rootfs_prepare[weight] = "25"
rootfs_prepare(){
- sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
# setup chroot
- sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
+ run_privileged "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
}
ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt"
rootfs_configure_isar_apt[weight] = "2"
rootfs_configure_isar_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
@@ -270,7 +270,7 @@ EOSUDO
ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_apt"
rootfs_configure_apt[weight] = "2"
rootfs_configure_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p '${ROOTFSDIR}/etc/apt/apt.conf.d'
@@ -293,7 +293,7 @@ ROOTFS_CONFIGURE_COMMAND += "rootfs_disable_initrd_generation"
rootfs_disable_initrd_generation[weight] = "1"
rootfs_disable_initrd_generation() {
# fully disable initrd generation
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
mkdir -p "${ROOTFSDIR}${ROOTFS_STUBS_DIR}"
@@ -310,7 +310,7 @@ rootfs_install_pkgs_update[weight] = "5"
rootfs_install_pkgs_update[isar-apt-lock] = "acquire-before"
rootfs_install_pkgs_update[network] = "${TASK_USE_NETWORK_AND_SUDO}"
rootfs_install_pkgs_update() {
- sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
+ run_in_chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
-o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
-o Dir::Etc::SourceParts="-" \
-o APT::Get::List-Cleanup="0"
@@ -322,9 +322,9 @@ rootfs_install_resolvconf() {
if [ "${@repr(bb.utils.to_boolean(d.getVar('BB_NO_NETWORK')))}" != "True" ]
then
if [ -L "${ROOTFSDIR}/etc/resolv.conf" ]; then
- sudo unlink "${ROOTFSDIR}/etc/resolv.conf"
+ run_privileged unlink "${ROOTFSDIR}/etc/resolv.conf"
fi
- sudo cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
+ run_privileged cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
fi
}
@@ -358,7 +358,7 @@ rootfs_export_package_cache() {
ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
rootfs_install_clean_files[weight] = "2"
rootfs_install_clean_files() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
for clean_file in ${ROOTFS_CLEAN_FILES}; do
rm -f "${ROOTFSDIR}/$clean_file"
done
@@ -370,14 +370,14 @@ rootfs_install_pkgs_install[weight] = "8000"
rootfs_install_pkgs_install[progress] = "custom:rootfs_progress.PkgsInstallProgressHandler"
rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
rootfs_install_pkgs_install() {
- sudo -E chroot "${ROOTFSDIR}" \
+ run_in_chroot "${ROOTFSDIR}" \
/usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
}
ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
rootfs_restore_initrd_tooling[weight] = "1"
rootfs_restore_initrd_tooling() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar-stubs"
rm -rf "${ROOTFSDIR}${ROOTFS_STUBS_DIR}"
@@ -386,8 +386,8 @@ EOSUDO
ROOTFS_INSTALL_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-initrd', '', 'rootfs_clear_initrd_symlinks', d)}"
rootfs_clear_initrd_symlinks() {
- sudo rm -f ${ROOTFSDIR}/initrd.img
- sudo rm -f ${ROOTFSDIR}/initrd.img.old
+ run_privileged rm -f ${ROOTFSDIR}/initrd.img
+ run_privileged rm -f ${ROOTFSDIR}/initrd.img.old
}
do_rootfs_install[root_cleandirs] = "${ROOTFSDIR}"
@@ -437,21 +437,21 @@ do_cache_deb_src[network] = "${TASK_USE_SUDO}"
do_cache_deb_src() {
if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
fi
rootfs_install_resolvconf
# Note: Isar updates the apt state information(apt-get update) only once during bootstrap and
# relies on that through out the build. Copy that state information instead of apt-get update
# which generates a new state from upstream.
- sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
- sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf
if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
fi
}
@@ -459,21 +459,21 @@ ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('BASE_REPO_FEATURES', 'cache
cache_dbg_pkgs() {
if [ -e "${ROOTFSDIR}"/etc/resolv.conf ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf "${ROOTFSDIR}"/etc/resolv.conf.isar
fi
rootfs_install_resolvconf
# Note: Isar updates the apt state information(apt-get update) only once during bootstrap and
# relies on that through out the build. Copy that state information instead of apt-get update
# which generates a new state from upstream.
- sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
+ run_privileged tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}"
deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
dbg_pkgs_download ${ROOTFSDIR}
- sudo rm -f "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged rm -f "${ROOTFSDIR}"/etc/resolv.conf
if [ -e "${ROOTFSDIR}"/etc/resolv.conf.isar ] ||
[ -h "${ROOTFSDIR}"/etc/resolv.conf.isar ]; then
- sudo mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
+ run_privileged mv "${ROOTFSDIR}"/etc/resolv.conf.isar "${ROOTFSDIR}"/etc/resolv.conf
fi
}
@@ -482,17 +482,17 @@ ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-package-cache', 'rootfs_postprocess_clean_package_cache', '', d)}"
rootfs_postprocess_clean_package_cache() {
- sudo -E chroot '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
/usr/bin/apt-get clean
- sudo rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
+ run_privileged rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
# remove apt-cache folder itself (required in case rootfs is provided by sstate cache)
- sudo rm -rf "${ROOTFSDIR}/var/cache/apt/archives"
+ run_privileged rm -rf "${ROOTFSDIR}/var/cache/apt/archives"
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-log-files', 'rootfs_postprocess_clean_log_files', '', d)}"
rootfs_postprocess_clean_log_files() {
# Delete log files that are not owned by packages
- sudo -E chroot '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
/usr/bin/find /var/log/ -type f \
-exec sh -c '! dpkg -S {} > /dev/null 2>&1' ';' \
-exec rm -f {} ';'
@@ -501,32 +501,32 @@ rootfs_postprocess_clean_log_files() {
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-debconf-cache', 'rootfs_postprocess_clean_debconf_cache', '', d)}"
rootfs_postprocess_clean_debconf_cache() {
# Delete debconf cache files
- sudo rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
+ run_privileged rm -rf "${ROOTFSDIR}/var/cache/debconf/"*
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'clean-pycache', 'rootfs_postprocess_clean_pycache', '', d)}"
rootfs_postprocess_clean_pycache() {
- sudo find ${ROOTFSDIR}/usr -type f -name '*.pyc' -delete -print
- sudo find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete -print
+ run_privileged find ${ROOTFSDIR}/usr -type f -name '*.pyc' -delete -print
+ run_privileged find ${ROOTFSDIR}/usr -type d -name '__pycache__' -delete -print
}
ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_ldconfig_cache"
rootfs_postprocess_clean_ldconfig_cache() {
# the ldconfig aux-cache is not portable and breaks reproducability
# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845034#49
- sudo rm -f ${ROOTFSDIR}/var/cache/ldconfig/aux-cache
+ run_privileged rm -f ${ROOTFSDIR}/var/cache/ldconfig/aux-cache
}
ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_tmp"
rootfs_postprocess_clean_tmp() {
# /tmp is by definition non persistent across boots
- sudo rm -rf "${ROOTFSDIR}/tmp/"*
+ run_privileged rm -rf "${ROOTFSDIR}/tmp/"*
}
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
rootfs_generate_manifest () {
mkdir -p ${ROOTFS_MANIFEST_DEPLOY_DIR}
- sudo -E chroot --userspec=$(id -u):$(id -g) '${ROOTFSDIR}' \
+ run_in_chroot '${ROOTFSDIR}' \
dpkg-query -W -f \
'${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' > \
'${ROOTFS_MANIFEST_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.manifest
@@ -542,7 +542,7 @@ rootfs_export_dpkg_status() {
ROOTFS_POSTPROCESS_COMMAND += "rootfs_cleanup_isar_apt"
rootfs_cleanup_isar_apt[weight] = "2"
rootfs_cleanup_isar_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt"
@@ -553,7 +553,7 @@ EOSUDO
ROOTFS_POSTPROCESS_COMMAND += "${@'rootfs_cleanup_base_apt' if bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else ''}"
rootfs_cleanup_base_apt[weight] = "2"
rootfs_cleanup_base_apt() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/"*base-apt.list
EOSUDO
@@ -561,12 +561,12 @@ EOSUDO
ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'populate-systemd-preset', 'image_postprocess_populate_systemd_preset', '', d)}"
image_postprocess_populate_systemd_preset() {
- SYSTEMD_INSTALLED=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
+ SYSTEMD_INSTALLED=$(run_in_chroot '${ROOTFSDIR}' dpkg-query \
--showformat='${db:Status-Status}' \
--show systemd || echo "" )
if (test "$SYSTEMD_INSTALLED" = "installed"); then
- sudo chroot '${ROOTFSDIR}' systemctl preset-all --preset-mode="enable-only"
+ run_in_chroot '${ROOTFSDIR}' systemctl preset-all --preset-mode="enable-only"
fi
}
@@ -626,7 +626,7 @@ rootfs_generate_initramfs() {
mods_total="$(find ${ROOTFSDIR}/usr/lib/modules/$kernel_version -type f -name '*.ko*' | wc -l)"
echo "Total number of modules: $mods_total"
echo "Generating initrd for kernel version: $kernel_version"
- sudo -E chroot "${ROOTFSDIR}" sh -ec ' \
+ run_in_chroot "${ROOTFSDIR}" sh -ec ' \
${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}; \
find /boot -name "initrd.img-$kernel_version*" -exec install --mode 0644 {} /isar-work/initrd.img \; \
'
@@ -664,11 +664,12 @@ rootfs_install_sstate_prepare() {
# so we use some mount magic to prevent that
mkdir -p ${WORKDIR}/mnt/rootfs
trap 'rmdir ${WORKDIR}/mnt/rootfs ${WORKDIR}/mnt' EXIT
- sudo mount -o bind,private '${WORKDIR}/rootfs' '${WORKDIR}/mnt/rootfs' -o ro
+
+ run_privileged mount -o bind,private '${WORKDIR}/rootfs' '${WORKDIR}/mnt/rootfs' -o ro
lopts="--one-file-system --exclude=var/cache/apt/archives"
- sudo tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
- sudo umount ${WORKDIR}/mnt/rootfs
- sudo chown $(id -u):$(id -g) rootfs.tar
+ run_privileged tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
+ run_privileged umount ${WORKDIR}/mnt/rootfs
+ run_privileged chown $(id -u):$(id -g) rootfs.tar
}
do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
@@ -677,7 +678,7 @@ rootfs_install_sstate_finalize() {
# - after building the rootfs, the tar won't be there, but we also don't need to unpack
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
- sudo tar -C ${WORKDIR} -xpf rootfs.tar ${SSTATE_TAR_ATTR_FLAGS}
+ run_privileged tar -C ${WORKDIR} -xpf rootfs.tar ${SSTATE_TAR_ATTR_FLAGS}
rm rootfs.tar
fi
}
diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass
index 95dadee3..d9ccce7f 100644
--- a/meta/classes-recipe/sbuild.bbclass
+++ b/meta/classes-recipe/sbuild.bbclass
@@ -20,7 +20,7 @@ SCHROOT_LOCKFILE = "/tmp/schroot.lock"
schroot_create_configs() {
mkdir -p "${TMPDIR}/schroot-overlay"
echo "Creating ${SCHROOT_CONF_FILE}"
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
cat << EOF > "${SCHROOT_CONF_FILE}"
@@ -59,7 +59,7 @@ EOSUDO
schroot_delete_configs() {
(flock -x 9
set -e
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
if [ -d "${SBUILD_CONF_DIR}" ]; then
echo "Removing ${SBUILD_CONF_DIR}"
@@ -101,7 +101,7 @@ sbuild_export() {
}
insert_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
for mp in ${SCHROOT_MOUNTS}; do
FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind,private 0 0"
@@ -112,7 +112,7 @@ EOSUDO
}
remove_mounts() {
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
for mp in ${SCHROOT_MOUNTS}; do
FSTAB_LINE="${mp%%:*} ${mp#*:} none rw,bind,private 0 0"
@@ -123,7 +123,7 @@ EOSUDO
schroot_configure_ccache() {
mkdir -p "${CCACHE_DIR}"
- sudo -s <<'EOSUDO'
+ run_privileged_heredoc <<'EOSUDO'
set -e
sbuild_fstab="${SBUILD_CONF_DIR}/fstab"
diff --git a/meta/classes-recipe/sdk.bbclass b/meta/classes-recipe/sdk.bbclass
index 6f09b5f6..16165792 100644
--- a/meta/classes-recipe/sdk.bbclass
+++ b/meta/classes-recipe/sdk.bbclass
@@ -69,12 +69,12 @@ ROOTFS_POSTPROCESS_COMMAND:remove = "${@'rootfs_cleanup_isar_apt' if bb.utils.to
ROOTFS_CONFIGURE_COMMAND:append:class-sdk = " ${@'rootfs_configure_isar_apt_dir' if bb.utils.to_boolean(d.getVar('SDK_INCLUDE_ISAR_APT')) else ''}"
rootfs_configure_isar_apt_dir() {
# Copy isar-apt instead of mounting:
- sudo cp -Trpfx --reflink=auto ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
+ run_privileged cp -Trpfx --reflink=auto ${REPO_ISAR_DIR}/${DISTRO} ${ROOTFSDIR}/isar-apt
}
ROOTFS_POSTPROCESS_COMMAND:prepend:class-sdk = "sdkchroot_configscript "
sdkchroot_configscript () {
- sudo chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+ run_in_chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
}
ROOTFS_POSTPROCESS_COMMAND:append:class-sdk = " sdkchroot_finalize"
@@ -83,7 +83,7 @@ sdkchroot_finalize() {
rootfs_do_umounts
# Remove setup scripts
- sudo rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
+ run_privileged rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
# Make all links relative
for link in $(find ${ROOTFSDIR}/ -type l); do
@@ -95,16 +95,16 @@ sdkchroot_finalize() {
new_target=$(realpath --no-symlinks -m --relative-to=$basedir ${ROOTFSDIR}${target})
# remove first to allow rewriting directory links
- sudo rm $link
- sudo ln -s $new_target $link
+ run_privileged rm $link
+ run_privileged ln -s $new_target $link
fi
done
# Set up sysroot wrapper
for tool_pattern in "gcc-[0-9]*" "g++-[0-9]*" "cpp-[0-9]*" "ld.bfd" "ld.gold"; do
for tool in $(find ${ROOTFSDIR}/usr/bin -type f -name "*-linux-gnu*-${tool_pattern}"); do
- sudo mv "${tool}" "${tool}.bin"
- sudo ln -sf gcc-sysroot-wrapper.sh ${tool}
+ run_privileged mv "${tool}" "${tool}.bin"
+ run_privileged ln -sf gcc-sysroot-wrapper.sh ${tool}
done
done
}
diff --git a/meta/classes/sbom.bbclass b/meta/classes/sbom.bbclass
index b220f3d9..b4fcddaa 100644
--- a/meta/classes/sbom.bbclass
+++ b/meta/classes/sbom.bbclass
@@ -41,7 +41,7 @@ def sbom_doc_uuid(d):
d.setVar("SBOM_DOCUMENT_UUID", generate_document_uuid(d))
generate_sbom() {
- sudo mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
+ run_privileged mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
bwrap \
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index f21a6164..da8bc52d 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -209,19 +209,19 @@ do_bootstrap() {
trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \
&& rm "${WORKDIR}/mmtmpdir"; \
[ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \
- && sudo umount $tmpdir/$base_apt_tmp; \
+ && run_privileged umount $tmpdir/$base_apt_tmp; \
[ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \
- && sudo umount $tmpdir/base-apt; \
- [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \
+ && run_privileged umount $tmpdir/base-apt; \
+ [ -d "$tmpdir" ] && run_privileged rm -rf --one-file-system $tmpdir; \
[ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \
- && sudo umount $base_apt_tmp \
+ && run_privileged umount $base_apt_tmp \
&& rm -rf --one-file-system $base_apt_tmp' EXIT
# Create lock file so that it is owned by the user running the build (not root)
mkdir -p ${DEBDIR}
touch ${DEB_DL_LOCK}
mkdir -p ${DEBDIR}
- sudo TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
+ run_privileged TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
$arch_param \
--mode=unshare \
${MMHOOKS} \
@@ -254,7 +254,7 @@ do_bootstrap() {
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ run_privileged rm -rf --one-file-system "${WORKDIR}/dl_dir"
fi
}
addtask bootstrap before do_build after do_generate_keyrings
diff --git a/testsuite/unittests/test_image_account_extension.py b/testsuite/unittests/test_image_account_extension.py
index f78aa7f8..ff0e47e0 100644
--- a/testsuite/unittests/test_image_account_extension.py
+++ b/testsuite/unittests/test_image_account_extension.py
@@ -54,9 +54,8 @@ class TestImageAccountExtensionImageCreateUsers(
image_create_users(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/useradd',
@@ -136,9 +135,8 @@ class TestImageAccountExtensionImageCreateGroups(
image_create_groups(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/groupadd',
@@ -164,9 +162,8 @@ class TestImageAccountExtensionImageCreateGroups(
image_create_groups(d)
run_mock.assert_called_once_with(
+ run_privileged_cmd(d).split() +
[
- 'sudo',
- '-E',
'chroot',
rootfs.path(),
'/usr/sbin/groupmod',
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
We ensure that the sstate artifact is always generated for the correct
rootfs directory by using the ROOTFSDIR variable instead of the
assumption that it is in "rootfs". Further, we avoid file permission
on unshare, as root inside the container maps to the caller outside of
the container.
meta/classes-recipe/rootfs.bbclass | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 440786b5..aa65cec4 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -665,11 +665,13 @@ rootfs_install_sstate_prepare() {
- lopts="--one-file-system --exclude=var/cache/apt/archives"
- run_privileged tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
- run_privileged umount ${WORKDIR}/mnt/rootfs
- run_privileged chown $(id -u):$(id -g) rootfs.tar
+ run_privileged_heredoc <<'EOF'
+ mount -o bind,private '${ROOTFSDIR}' '${WORKDIR}/mnt/rootfs' -o ro
+ lopts="--one-file-system --exclude=var/cache/apt/archives"
+ tar -C ${WORKDIR}/mnt/rootfs -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} .
+ umount -q ${WORKDIR}/mnt/rootfs
+EOF
+ ${@ 'sudo chown $(id -u):$(id -g) rootfs.tar' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' }
+ mkdir -p ${ROOTFSDIR}
+ run_privileged tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} < rootfs.tar
rm rootfs.tar
fi
}
--
2.53.0
rootfs directory by using the ROOTFSDIR variable instead of the
assumption that it is in "rootfs". Further, we avoid file permission
on unshare, as root inside the container maps to the caller outside of
the container.
meta/classes-recipe/rootfs.bbclass | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 440786b5..aa65cec4 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -665,11 +665,13 @@ rootfs_install_sstate_prepare() {
mkdir -p ${WORKDIR}/mnt/rootfs
trap 'rmdir ${WORKDIR}/mnt/rootfs ${WORKDIR}/mnt' EXIT
- run_privileged mount -o bind,private '${WORKDIR}/rootfs' '${WORKDIR}/mnt/rootfs' -o ro
trap 'rmdir ${WORKDIR}/mnt/rootfs ${WORKDIR}/mnt' EXIT
- lopts="--one-file-system --exclude=var/cache/apt/archives"
- run_privileged tar -C ${WORKDIR}/mnt -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} rootfs
- run_privileged umount ${WORKDIR}/mnt/rootfs
- run_privileged chown $(id -u):$(id -g) rootfs.tar
+ run_privileged_heredoc <<'EOF'
+ mount -o bind,private '${ROOTFSDIR}' '${WORKDIR}/mnt/rootfs' -o ro
+ lopts="--one-file-system --exclude=var/cache/apt/archives"
+ tar -C ${WORKDIR}/mnt/rootfs -cpSf rootfs.tar $lopts ${SSTATE_TAR_ATTR_FLAGS} .
+ umount -q ${WORKDIR}/mnt/rootfs
+EOF
+ ${@ 'sudo chown $(id -u):$(id -g) rootfs.tar' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' }
}
do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
@@ -678,7 +680,8 @@ rootfs_install_sstate_finalize() {
do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
# - after building the rootfs, the tar won't be there, but we also don't need to unpack
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
- run_privileged tar -C ${WORKDIR} -xpf rootfs.tar ${SSTATE_TAR_ATTR_FLAGS}
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
+ mkdir -p ${ROOTFSDIR}
+ run_privileged tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} < rootfs.tar
rm rootfs.tar
fi
}
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
Previously the initrd was deployed as root and later chowned in the
deploy dir. This involves privileged operations which will no longer be
possible when running rootless. To prepare for that, we deploy via a
stdout and create the target file by the correct user.
While doing this, we also remove a useless sudo invocation when listing
the ROOTFS/boot dir, as this can be listed by all users.
meta/classes-recipe/rootfs.bbclass | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index aa65cec4..60ea66ed 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -620,18 +620,16 @@ python do_generate_initramfs_setscene () {
rootfs_generate_initramfs[progress] = "custom:rootfs_progress.InitrdProgressHandler"
rootfs_generate_initramfs() {
- if [ -n "$(sudo find '${ROOTFSDIR}/boot' -type f -name 'vmlinu[xz]*')" ]; then
+ if [ -n "$(find '${ROOTFSDIR}/boot' -type f -name 'vmlinu[xz]*')" ]; then
for kernel in ${ROOTFSDIR}/boot/vmlinu[xz]-*; do
export kernel_version=$(basename $kernel | cut -d'-' -f2-)
- ${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}; \
- find /boot -name "initrd.img-$kernel_version*" -exec install --mode 0644 {} /isar-work/initrd.img \; \
- '
+ run_in_chroot "${ROOTFSDIR}" sh -ec '${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}'
+ find ${ROOTFSDIR}/boot -name "initrd.img-$kernel_version*" -exec cat {} \; \
+ > ${DEPLOYDIR}/${INITRD_DEPLOY_FILE}
done
- install --owner $(id -u) --group $(id -g) ${WORKDIR}/initrd.img ${DEPLOYDIR}/${INITRD_DEPLOY_FILE}
else
echo "no kernel in this rootfs, do not generate initrd"
fi
--
2.53.0
deploy dir. This involves privileged operations which will no longer be
possible when running rootless. To prepare for that, we deploy via a
stdout and create the target file by the correct user.
While doing this, we also remove a useless sudo invocation when listing
the ROOTFS/boot dir, as this can be listed by all users.
meta/classes-recipe/rootfs.bbclass | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index aa65cec4..60ea66ed 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -620,18 +620,16 @@ python do_generate_initramfs_setscene () {
rootfs_generate_initramfs[progress] = "custom:rootfs_progress.InitrdProgressHandler"
rootfs_generate_initramfs() {
- if [ -n "$(sudo find '${ROOTFSDIR}/boot' -type f -name 'vmlinu[xz]*')" ]; then
+ if [ -n "$(find '${ROOTFSDIR}/boot' -type f -name 'vmlinu[xz]*')" ]; then
for kernel in ${ROOTFSDIR}/boot/vmlinu[xz]-*; do
export kernel_version=$(basename $kernel | cut -d'-' -f2-)
mods_total="$(find ${ROOTFSDIR}/usr/lib/modules/$kernel_version -type f -name '*.ko*' | wc -l)"
echo "Total number of modules: $mods_total"
echo "Generating initrd for kernel version: $kernel_version"
- run_in_chroot "${ROOTFSDIR}" sh -ec ' \
echo "Total number of modules: $mods_total"
echo "Generating initrd for kernel version: $kernel_version"
- ${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}; \
- find /boot -name "initrd.img-$kernel_version*" -exec install --mode 0644 {} /isar-work/initrd.img \; \
- '
+ run_in_chroot "${ROOTFSDIR}" sh -ec '${ROOTFS_INITRAMFS_GENERATOR_CMDLINE}'
+ find ${ROOTFSDIR}/boot -name "initrd.img-$kernel_version*" -exec cat {} \; \
+ > ${DEPLOYDIR}/${INITRD_DEPLOY_FILE}
done
- install --owner $(id -u) --group $(id -g) ${WORKDIR}/initrd.img ${DEPLOYDIR}/${INITRD_DEPLOY_FILE}
else
echo "no kernel in this rootfs, do not generate initrd"
fi
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
By introducing a bitbake python function (a code generator) to
generate the mount shell code, we make it reusable within here
documents where external shell functions cannot be called.
meta/classes-global/base.bbclass | 18 +++++++++++
meta/classes-recipe/rootfs.bbclass | 49 ++++++++----------------------
2 files changed, 30 insertions(+), 37 deletions(-)
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index d4dbbc3a..90e4525e 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -379,6 +379,24 @@ def deb_list_beautify(d, varname):
+ lines = []
+ for m in mounts.split():
+ host, inner = m.split(':') if ':' in m else (m, m)
+ inner_full = os.path.join(rootfs, inner[1:])
+ lines.append('mkdir -p {}'.format(inner_full))
+ lines.append('mount -o bind,private {} {}'.format(host, inner_full))
+ return '\n'.join(lines)
+
+def insert_isar_umounts(d, rootfs, mounts):
+ lines = []
+ for m in mounts.split():
+ host, inner = m.split(':') if ':' in m else (m, m)
+ mp = '{}/{}'.format(rootfs, inner)
+ lines.append('mountpoint -q {} && umount {}'.format(mp, mp))
+ lines.append('[ -d {} ] && rmdir --ignore-fail-on-non-empty {}'.format(mp, mp))
+ return '\n'.join(lines)
+
def run_privileged_cmd(d):
cmd = 'sudo -E'
index 60ea66ed..7352a87c 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -49,6 +49,16 @@ ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
# path to deploy stubbed versions of initrd update scripts during do_rootfs_install
ROOTFS_STUBS_DIR = "/usr/local/isar-sbin"
+# list of <outer>:<inner> or <outer> mount entries
+ROOTFS_MOUNTS ??= "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt ${WORKDIR}:/isar-work"
+
+python () {
+ mounts = d.getVar('ROOTFS_MOUNTS', False)
+ if d.getVar('ISAR_USE_CACHED_BASE_REPO') and not ':/base-apt' in mounts:
+ base_apt = '{}:/base-apt'.format(d.getVar('REPO_BASE_DIR'))
+ d.setVar('ROOTFS_MOUNTS', '{} {}'.format(mounts, base_apt))
+}
+
# helper to compute the rootfs distro also under cross building
def get_rootfs_distro(d):
host_arch = d.getVar('HOST_ARCH')
@@ -154,50 +164,15 @@ rootfs_do_mounts() {
mount -t tmpfs -o size=1m,nosuid,nodev none '${ROOTFSDIR}/sys/firmware'
fi
- # Mount isar-apt if the directory does not exist or if it is empty
- # This prevents overwriting something that was copied there
- if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \
- [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 | wc -l)" = "0" ]
- then
- mkdir -p '${ROOTFSDIR}/isar-apt'
- mountpoint -q '${ROOTFSDIR}/isar-apt' || \
- mount -o bind,private '${REPO_ISAR_DIR}/${DISTRO}' '${ROOTFSDIR}/isar-apt'
- fi
-
- if [ ! -e '$ROOTFSDIR'/isar-work ]; then
- mkdir -p '${ROOTFSDIR}/isar-work'
- mountpoint -q '${ROOTFSDIR}/isar-work' || \
- mount -o bind,private '${WORKDIR}' '${ROOTFSDIR}/isar-work'
- fi
-
- # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
- if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ]
- then
- mkdir -p '${ROOTFSDIR}/base-apt'
- mountpoint -q '${ROOTFSDIR}/base-apt' || \
- mount -o bind,private '${REPO_BASE_DIR}' '${ROOTFSDIR}/base-apt'
- fi
-
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS'))}
EOSUDO
}
rootfs_do_umounts() {
run_privileged_heredoc <<'EOSUDO'
set -e
- if mountpoint -q '${ROOTFSDIR}/isar-apt'; then
- umount '${ROOTFSDIR}/isar-apt'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-apt
- fi
- if mountpoint -q '${ROOTFSDIR}/base-apt'; then
- umount '${ROOTFSDIR}/base-apt'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/base-apt
- fi
-
- if mountpoint -q '${ROOTFSDIR}/isar-work'; then
- umount '${ROOTFSDIR}/isar-work'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-work
- fi
+ ${@insert_isar_umounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS'))}
if mountpoint -q '${ROOTFSDIR}/dev/pts'; then
umount '${ROOTFSDIR}/dev/pts'
--
2.53.0
generate the mount shell code, we make it reusable within here
documents where external shell functions cannot be called.
meta/classes-global/base.bbclass | 18 +++++++++++
meta/classes-recipe/rootfs.bbclass | 49 ++++++++----------------------
2 files changed, 30 insertions(+), 37 deletions(-)
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index d4dbbc3a..90e4525e 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -379,6 +379,24 @@ def deb_list_beautify(d, varname):
# Helpers for privileged execution. Only the non-underscore functions
# shall be used outside of this class.
+def insert_isar_mounts(d, rootfs, mounts):
+ lines = []
+ for m in mounts.split():
+ host, inner = m.split(':') if ':' in m else (m, m)
+ inner_full = os.path.join(rootfs, inner[1:])
+ lines.append('mkdir -p {}'.format(inner_full))
+ lines.append('mount -o bind,private {} {}'.format(host, inner_full))
+ return '\n'.join(lines)
+
+def insert_isar_umounts(d, rootfs, mounts):
+ lines = []
+ for m in mounts.split():
+ host, inner = m.split(':') if ':' in m else (m, m)
+ mp = '{}/{}'.format(rootfs, inner)
+ lines.append('mountpoint -q {} && umount {}'.format(mp, mp))
+ lines.append('[ -d {} ] && rmdir --ignore-fail-on-non-empty {}'.format(mp, mp))
+ return '\n'.join(lines)
+
def run_privileged_cmd(d):
cmd = 'sudo -E'
bb.debug(1, "privileged cmd: %s" % cmd)
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 60ea66ed..7352a87c 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -49,6 +49,16 @@ ROOTFS_PACKAGE_SUFFIX ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
# path to deploy stubbed versions of initrd update scripts during do_rootfs_install
ROOTFS_STUBS_DIR = "/usr/local/isar-sbin"
+# list of <outer>:<inner> or <outer> mount entries
+ROOTFS_MOUNTS ??= "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt ${WORKDIR}:/isar-work"
+
+python () {
+ mounts = d.getVar('ROOTFS_MOUNTS', False)
+ if d.getVar('ISAR_USE_CACHED_BASE_REPO') and not ':/base-apt' in mounts:
+ base_apt = '{}:/base-apt'.format(d.getVar('REPO_BASE_DIR'))
+ d.setVar('ROOTFS_MOUNTS', '{} {}'.format(mounts, base_apt))
+}
+
# helper to compute the rootfs distro also under cross building
def get_rootfs_distro(d):
host_arch = d.getVar('HOST_ARCH')
@@ -154,50 +164,15 @@ rootfs_do_mounts() {
mount -t tmpfs -o size=1m,nosuid,nodev none '${ROOTFSDIR}/sys/firmware'
fi
- # Mount isar-apt if the directory does not exist or if it is empty
- # This prevents overwriting something that was copied there
- if [ ! -e '${ROOTFSDIR}/isar-apt' ] || \
- [ "$(find '${ROOTFSDIR}/isar-apt' -maxdepth 1 -mindepth 1 | wc -l)" = "0" ]
- then
- mkdir -p '${ROOTFSDIR}/isar-apt'
- mountpoint -q '${ROOTFSDIR}/isar-apt' || \
- mount -o bind,private '${REPO_ISAR_DIR}/${DISTRO}' '${ROOTFSDIR}/isar-apt'
- fi
-
- if [ ! -e '$ROOTFSDIR'/isar-work ]; then
- mkdir -p '${ROOTFSDIR}/isar-work'
- mountpoint -q '${ROOTFSDIR}/isar-work' || \
- mount -o bind,private '${WORKDIR}' '${ROOTFSDIR}/isar-work'
- fi
-
- # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
- if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ]
- then
- mkdir -p '${ROOTFSDIR}/base-apt'
- mountpoint -q '${ROOTFSDIR}/base-apt' || \
- mount -o bind,private '${REPO_BASE_DIR}' '${ROOTFSDIR}/base-apt'
- fi
-
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS'))}
EOSUDO
}
rootfs_do_umounts() {
run_privileged_heredoc <<'EOSUDO'
set -e
- if mountpoint -q '${ROOTFSDIR}/isar-apt'; then
- umount '${ROOTFSDIR}/isar-apt'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-apt
- fi
- if mountpoint -q '${ROOTFSDIR}/base-apt'; then
- umount '${ROOTFSDIR}/base-apt'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/base-apt
- fi
-
- if mountpoint -q '${ROOTFSDIR}/isar-work'; then
- umount '${ROOTFSDIR}/isar-work'
- rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-work
- fi
+ ${@insert_isar_umounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS'))}
if mountpoint -q '${ROOTFSDIR}/dev/pts'; then
umount '${ROOTFSDIR}/dev/pts'
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
The implementation of the fetching depends on the chroot mode (e.g.
schroot or unshare). As a preparation for the unshare mode, we hide the
concrete fetcher implementation behind a factory, so that we will be
able to dispatch based on the mode.
meta/classes-recipe/dpkg-base.bbclass | 2 +-
meta/lib/aptsrc_fetcher.py | 12 +++++++++---
2 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index e5987554..e8721c79 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -84,7 +84,7 @@ python() {
# apt-src fetcher
import aptsrc_fetcher
- methods.append(aptsrc_fetcher.AptSrc())
+ methods.append(aptsrc_fetcher.AptSrc.create(d))
src_uri = (d.getVar('SRC_URI', False) or "").split()
for u in src_uri:
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index da2d94c2..1d133aae 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -9,6 +9,10 @@ from bb.fetch2 import logger
from bb.fetch2 import runfetchcmd
class AptSrc(FetchMethod):
+ @classmethod
+ def create(cls, d):
+ return AptSrcSchroot()
+
def supports(self, ud, d):
return ud.type in ['apt']
@@ -20,6 +24,11 @@ class AptSrc(FetchMethod):
codename = d.getVar('BASE_DISTRO_CODENAME')
ud.localfile='deb-src/' + base_distro + '-' + codename + '/' + ud.host
+ def clean(self, ud, d):
+ bb.utils.remove(ud.localpath, recurse=True)
+
+
+class AptSrcSchroot(AptSrc):
def download(self, ud, d):
bb.utils.exec_flat_python_func('isar_export_proxies', d)
bb.build.exec_func('schroot_create_configs', d)
@@ -83,6 +92,3 @@ class AptSrc(FetchMethod):
finally:
runfetchcmd(f'schroot -q -f -e -c {session_id}', d)
bb.build.exec_func('schroot_delete_configs', d)
-
- def clean(self, ud, d):
- bb.utils.remove(ud.localpath, recurse=True)
--
2.53.0
schroot or unshare). As a preparation for the unshare mode, we hide the
concrete fetcher implementation behind a factory, so that we will be
able to dispatch based on the mode.
meta/classes-recipe/dpkg-base.bbclass | 2 +-
meta/lib/aptsrc_fetcher.py | 12 +++++++++---
2 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index e5987554..e8721c79 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -84,7 +84,7 @@ python() {
# apt-src fetcher
import aptsrc_fetcher
- methods.append(aptsrc_fetcher.AptSrc())
+ methods.append(aptsrc_fetcher.AptSrc.create(d))
src_uri = (d.getVar('SRC_URI', False) or "").split()
for u in src_uri:
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index da2d94c2..1d133aae 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -9,6 +9,10 @@ from bb.fetch2 import logger
from bb.fetch2 import runfetchcmd
class AptSrc(FetchMethod):
+ @classmethod
+ def create(cls, d):
+ return AptSrcSchroot()
+
def supports(self, ud, d):
return ud.type in ['apt']
@@ -20,6 +24,11 @@ class AptSrc(FetchMethod):
codename = d.getVar('BASE_DISTRO_CODENAME')
ud.localfile='deb-src/' + base_distro + '-' + codename + '/' + ud.host
+ def clean(self, ud, d):
+ bb.utils.remove(ud.localpath, recurse=True)
+
+
+class AptSrcSchroot(AptSrc):
def download(self, ud, d):
bb.utils.exec_flat_python_func('isar_export_proxies', d)
bb.build.exec_func('schroot_create_configs', d)
@@ -83,6 +92,3 @@ class AptSrc(FetchMethod):
finally:
runfetchcmd(f'schroot -q -f -e -c {session_id}', d)
bb.build.exec_func('schroot_delete_configs', d)
-
- def clean(self, ud, d):
- bb.utils.remove(ud.localpath, recurse=True)
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
When running in rootless mode, cleaning the build directory from outside
the build environment is a non trivial task due to mixed file
ownerships. To simplify this, we introduce the isar-clean-builddir
script that can perform the cleanup without requiring root privileges.
RECIPE-API-CHANGELOG.md | 5 +++
scripts/isar-clean-builddir | 73 +++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+)
create mode 100755 scripts/isar-clean-builddir
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 27d14dc4..26a4c084 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1098,3 +1098,8 @@ This internally switches the chroot mode from `schroot` to `unshare`.
When using kas, the `build_system` needs to be set to `isar-rootless`, but the final
interfaces still need to be clarified. Further, kas patches are needed (for details,
check the kas mailing list).
+
+Note, that the build dir may contain files that were generated within the rootless
+environment and cannot be deleted from the outside by the calling user. To simplify
+the cleanup, we provide the `isar-clean-builddir` script that helps purging
+directories with mixed ownerships (without requiring root privileges).
diff --git a/scripts/isar-clean-builddir b/scripts/isar-clean-builddir
new file mode 100755
index 00000000..6bc90b1d
--- /dev/null
+++ b/scripts/isar-clean-builddir
@@ -0,0 +1,73 @@
+#!/bin/sh
+# isar-clean-builddir - Clean the build/tmp directory
+#
+# This script removes all files from the specified directory, including those
+# owned by other users (which requires elevated privileges).
+#
+# Rootless Mode:
+# When --rootless is specified, no privileged commands are executed. This
+# requires that the UID namespace where files were generated matches the
+# cleanup environment. When running from a container, this script must be
+# called from within the same container.
+#
+# Part of the Isar API. External tools may call this script for cleanup.
+#
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+DRY_RUN=0
+ROOTLESS=0
+
+usage()
+{
+ EXIT_CODE="$1"
+ SELF="isar-clean-builddir"
+ printf "%b" "Usage: ${SELF} [--rootless] [--dry-run] [dir]\n"
+
+ exit "${EXIT_CODE:-1}"
+}
+
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --dry-run)
+ DRY_RUN=1
+ shift 1
+ ;;
+ -h | --help)
+ usage 0
+ ;;
+ --rootless)
+ ROOTLESS=1
+ shift 1
+ ;;
+ --*)
+ usage 1
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+[ $# -eq 1 ] || usage 1
+if ! [ -d "$1" ]; then
+ echo "error: \"$1\" is not a directory"
+ exit 1
+fi
+
+if [ $ROOTLESS -eq 1 ]; then
+ PRIVILEGED_CMD="unshare --map-auto --map-root-user --keep-caps"
+else
+ PRIVILEGED_CMD="sudo"
+fi
+
+if [ $DRY_RUN -eq 1 ]; then
+ echo "dry-run, not executing"
+ DRY_RUN_PREFIX="/bin/echo"
+fi
+
+# clean all files that do not belong to us
+# shellcheck disable=2086
+find "$1" \( ! -user "$(whoami)" -type d -prune \) -exec $DRY_RUN_PREFIX $PRIVILEGED_CMD rm -rf {} \;
+# clean remaining files
+$DRY_RUN_PREFIX rm -rf "$1"
--
2.53.0
the build environment is a non trivial task due to mixed file
ownerships. To simplify this, we introduce the isar-clean-builddir
script that can perform the cleanup without requiring root privileges.
RECIPE-API-CHANGELOG.md | 5 +++
scripts/isar-clean-builddir | 73 +++++++++++++++++++++++++++++++++++++
2 files changed, 78 insertions(+)
create mode 100755 scripts/isar-clean-builddir
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 27d14dc4..26a4c084 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1098,3 +1098,8 @@ This internally switches the chroot mode from `schroot` to `unshare`.
When using kas, the `build_system` needs to be set to `isar-rootless`, but the final
interfaces still need to be clarified. Further, kas patches are needed (for details,
check the kas mailing list).
+
+Note, that the build dir may contain files that were generated within the rootless
+environment and cannot be deleted from the outside by the calling user. To simplify
+the cleanup, we provide the `isar-clean-builddir` script that helps purging
+directories with mixed ownerships (without requiring root privileges).
diff --git a/scripts/isar-clean-builddir b/scripts/isar-clean-builddir
new file mode 100755
index 00000000..6bc90b1d
--- /dev/null
+++ b/scripts/isar-clean-builddir
@@ -0,0 +1,73 @@
+#!/bin/sh
+# isar-clean-builddir - Clean the build/tmp directory
+#
+# This script removes all files from the specified directory, including those
+# owned by other users (which requires elevated privileges).
+#
+# Rootless Mode:
+# When --rootless is specified, no privileged commands are executed. This
+# requires that the UID namespace where files were generated matches the
+# cleanup environment. When running from a container, this script must be
+# called from within the same container.
+#
+# Part of the Isar API. External tools may call this script for cleanup.
+#
+# Copyright (c) Siemens AG, 2026
+# SPDX-License-Identifier: MIT
+
+DRY_RUN=0
+ROOTLESS=0
+
+usage()
+{
+ EXIT_CODE="$1"
+ SELF="isar-clean-builddir"
+ printf "%b" "Usage: ${SELF} [--rootless] [--dry-run] [dir]\n"
+
+ exit "${EXIT_CODE:-1}"
+}
+
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --dry-run)
+ DRY_RUN=1
+ shift 1
+ ;;
+ -h | --help)
+ usage 0
+ ;;
+ --rootless)
+ ROOTLESS=1
+ shift 1
+ ;;
+ --*)
+ usage 1
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+[ $# -eq 1 ] || usage 1
+if ! [ -d "$1" ]; then
+ echo "error: \"$1\" is not a directory"
+ exit 1
+fi
+
+if [ $ROOTLESS -eq 1 ]; then
+ PRIVILEGED_CMD="unshare --map-auto --map-root-user --keep-caps"
+else
+ PRIVILEGED_CMD="sudo"
+fi
+
+if [ $DRY_RUN -eq 1 ]; then
+ echo "dry-run, not executing"
+ DRY_RUN_PREFIX="/bin/echo"
+fi
+
+# clean all files that do not belong to us
+# shellcheck disable=2086
+find "$1" \( ! -user "$(whoami)" -type d -prune \) -exec $DRY_RUN_PREFIX $PRIVILEGED_CMD rm -rf {} \;
+# clean remaining files
+$DRY_RUN_PREFIX rm -rf "$1"
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
Currently isar requires passwordless sudo and an environment
RECIPE-API-CHANGELOG.md | 21 +++++
meta/classes-recipe/deb-dl-dir.bbclass | 9 +-
meta/classes-recipe/dpkg-base.bbclass | 22 ++++-
meta/classes-recipe/dpkg.bbclass | 17 ++--
.../image-locales-extension.bbclass | 9 +-
.../image-tools-extension.bbclass | 84 ++++++++++++++++++
meta/classes-recipe/image.bbclass | 7 +-
.../imagetypes_container.bbclass | 4 +-
meta/classes-recipe/imagetypes_wic.bbclass | 6 +-
meta/classes-recipe/rootfs.bbclass | 52 ++++++++---
meta/classes-recipe/sbuild.bbclass | 24 +++++-
meta/classes-recipe/sdk.bbclass | 10 ++-
meta/conf/bitbake.conf | 7 +-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 18 ++--
.../sbuild-chroot/sbuild-chroot.inc | 24 +++++-
19 files changed, 362 insertions(+), 44 deletions(-)
diff --git a/Kconfig b/Kconfig
index 86a4aac3..66dd4112 100644
--- a/Kconfig
+++ b/Kconfig
@@ -14,7 +14,7 @@ config KAS_INCLUDE_MAIN
config KAS_BUILD_SYSTEM
string
- default "isar"
+ default "isar-rootless"
source "kas/machine/Kconfig"
source "kas/distro/Kconfig"
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index d0aa6e1a..27d14dc4 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1077,3 +1077,24 @@ specifies the rootfs path.
+
+### Rootless isar execution
+
+Isar is able to run without the need for `sudo` in an environment that
+allows unprivileged users to unshare the kernels `user namespace`. Further,
+a sufficiently large set of sub ids needs to be configured in `/etc/subuid` / `etc/subgid`.
+This range should be `> 65536`, but smaller ranges might work as well, depending on the
+ids used in the rootfs.
+
+A simple check if rootless is supported can be done by running:
+
+```bash
+mmdebstrap --unshare-helper /bin/echo "rootless supported" || echo "rootless not supported"
+```
+
+To enable rootless builds, set the bitbake variable `ISAR_ROOTLESS = "1"`.
+This internally switches the chroot mode from `schroot` to `unshare`.
+
+When using kas, the `build_system` needs to be set to `isar-rootless`, but the final
+interfaces still need to be clarified. Further, kas patches are needed (for details,
+check the kas mailing list).
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 69e8dfef..26041f9a 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -74,6 +74,7 @@ Building `debian-trixie` requires host system >= bookworm.
Install the following packages:
```
apt install \
+ acl \
binfmt-support \
bubblewrap \
bzip2 \
@@ -88,6 +89,7 @@ apt install \
qemu-user-static \
reprepro \
sudo \
+ uidmap \
unzip \
xz-utils \
git-buildpackage \
diff --git a/kas/isar.yaml b/kas/isar.yaml
index 16ce8b42..3cfc4f96 100644
--- a/kas/isar.yaml
+++ b/kas/isar.yaml
@@ -4,7 +4,7 @@
header:
version: 14
-build_system: isar
+build_system: isar-rootless
repos:
isar:
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 90e4525e..7167cbb1 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -141,7 +141,9 @@ root_cleandirs() {
+ [ -d "$TMPDIR$i" ] || continue
+ find "$TMPDIR$i" \( ! -user "$(whoami)" -type d -prune \) -exec ${RUN_PRIVILEGED_CMD} rm -rf --one-file-system {} \;
+ rm -rf --one-file-system "$TMPDIR$i"
mkdir -p "$TMPDIR$i"
done
}
@@ -380,7 +382,28 @@ def deb_list_beautify(d, varname):
+ """
+ In unshare mode, all mounts must be created after unsharing the
+ mount namespace. As needs to happen within the unshared session,
+ we implement it as a code generator. Note, that the random and urandom
+ mounts are needed for DDI images.
+ """
lines = []
+ to_touch = ['/dev/null', '/dev/random', '/dev/urandom']
+ to_mkdir = ['/dev/pts', '/dev/shm']
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ lines.append('touch ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_touch]))
+ lines.append('mkdir -p ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_mkdir]))
+ lines.append('mount -o bind,private,mode=666 /dev/null {}/dev/null'.format(rootfs))
+ lines.append('mount -t devpts -o noexec,nosuid,uid=5,mode=620,ptmxmode=666 none {}/dev/pts'.format(rootfs))
+ lines.append('( cd {}/dev; ln -sf pts/ptmx . )'.format(rootfs))
+ lines.append('mount -t tmpfs none {}/dev/shm'.format(rootfs))
+ lines.append('mount -o bind /dev/random {}/dev/random'.format(rootfs))
+ lines.append('mount -o bind /dev/urandom {}/dev/urandom'.format(rootfs))
+ lines.append('mount -t proc none {}/proc'.format(rootfs))
+ # we do not unshare the network namespace, so we cannot create a sysfs, hence bind-mount
+ lines.append('mount -o rbind /sys {}/sys'.format(rootfs))
return '\n'.join(lines)
def insert_isar_umounts(d, rootfs, mounts):
+ """
+ In unshare mount we don't unmount the system mounts but just
+ remove the mountpoints.
+ """
lines = []
+ to_unlink = ['/dev/null', '/dev/random', '/dev/urandom', '/dev/ptmx']
+ to_rmdir = ['/dev/pts', '/dev/shm']
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ lines.append('rm -f ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_unlink]))
+ for d in ['{}/{}'.format(rootfs, _d) for _d in to_rmdir]:
+ lines.append('[ -d {} ] && rmdir {}'.format(d, d))
+def get_subid_range(idmap, d):
+ import getpass
+ with open(idmap, 'r') as f:
+ entries = f.readlines()
+ for e in entries:
+ user, base, cnt = e.split(':')
+ if user == os.getuid() or user == getpass.getuser():
+ return int(base), int(cnt)
+ bb.error("No sub-id range specified in %s" % idmap)
+
def run_privileged_cmd(d):
- cmd = 'sudo -E'
+ """
+ In unshare mode we need to map the rootfs uid/gid range into the
+ subuid/subgid range of the parent namespace. As we usually only
+ get 65534 ids, we cannot map the whole range, as two ids are already
+ used by the calling environment (root and builder user). Hence, map
+ as much as we can but also map the highest id (nobody / nogroup) as
+ these are used within the rootfs. It would be easier to use
+ mmdebstrap --unshare-helper as command (which is also internally used
+ by sbuild), but this only maps linear ranges, hence it cannot map the
+ nobody / nogroup on the default subid range. By that, we have to avoid
+ the nobody / nogroup when building packages in this case.
+ """
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ nobody_id = 65534
+ uid_base, uid_cnt = get_subid_range('/etc/subuid', d)
+ nobody_subid = uid_base + uid_cnt - 1
+ gid_base, gid_cnt = get_subid_range('/etc/subgid', d)
+ nogroup_subid = gid_base + gid_cnt - 1
+ cmd = 'unshare --mount --pid --uts --ipc --user' \
+ ' --kill-child' \
+ ' --setuid 0 --setgid 0 --fork' \
+ f' --map-users 1:{uid_base+1}:{uid_cnt-2}' \
+ f' --map-groups 1:{gid_base+1}:{gid_cnt-2}'
+ if uid_cnt < nobody_id:
+ cmd += f' --map-users {nobody_id}:{nobody_subid}:1'
+ if gid_cnt < nobody_id:
+ cmd += f' --map-groups {nobody_id}:{nogroup_subid}:1'
+ cmd += " --map-root-user"
+ else:
+ cmd = 'sudo -E'
+UNSHARE_SUBUID_BASE := "${@get_subid_range('/etc/subuid', d)[0] if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '0'}"
+# store in variable to only compute once and make available to fetcher
RUN_PRIVILEGED_CMD := "${@run_privileged_cmd(d)}"
run_privileged() {
@@ -415,5 +490,10 @@ run_privileged_heredoc() {
run_in_chroot() {
rootfs="$1"
shift
- ${RUN_PRIVILEGED_CMD} chroot "$rootfs" "$@"
+
+ rootfs=$rootfs run_privileged_heredoc <<'EORIC' "$@"
+ set -e
+ ${@insert_isar_mounts(d, '$rootfs', '')}
+ chroot "$rootfs" "$@"
+EORIC
}
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 04fd6414..0e268f06 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -123,8 +123,13 @@ deb_dl_dir_import() {
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
run_privileged_heredoc << ' EOSUDO'
- mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
- chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
+ if [ "${ISAR_CHROOT_MODE}" = "unshare" ]; then
+ mkdir -p "${rootfs}"/var/cache/apt/archives
+ chmod 777 "${rootfs}"/var/cache/apt/archives
+ else
+ mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
+ chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
+ fi
EOSUDO
# nothing to copy if download directory does not exist just yet
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index e8721c79..a0d4fd05 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -168,12 +168,30 @@ dpkg_schroot_create_configs() {
EOSUDO
}
+dpkg_chroot_prepare() {
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ dpkg_schroot_create_configs
+ fi
+}
+
+dpkg_chroot_finalize() {
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ schroot_delete_configs
+ fi
+}
+
+dpkg_prepare_unshare_ccache() {
+ mkdir -p "${CCACHE_DIR}"
+ # sbuild id from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1110942
+ setfacl -m u:${UNSHARE_SUBUID_BASE}:rwX -m u:${@int(d.getVar('UNSHARE_SUBUID_BASE')) + 999}:rwx "${CCACHE_DIR}"
+}
+
python do_dpkg_build() {
- bb.build.exec_func('dpkg_schroot_create_configs', d)
+ bb.build.exec_func('dpkg_chroot_prepare', d)
try:
bb.build.exec_func("dpkg_runbuild", d)
finally:
- bb.build.exec_func('schroot_delete_configs', d)
+ bb.build.exec_func('dpkg_chroot_finalize', d)
}
do_dpkg_build[network] = "${TASK_USE_NETWORK_AND_SUDO}"
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index dcdef487..57fe042b 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -85,7 +85,10 @@ dpkg_runbuild() {
ext_deb_dir="${ext_root}${deb_dir}"
if [ ${USE_CCACHE} -eq 1 ]; then
- schroot_configure_ccache
+ ${ISAR_CHROOT_MODE}_configure_ccache
+ fi
+ if [ "${ISAR_CHROOT_MODE}" = "unshare" ]; then
+ sbuild_add_unshare_mounts
fi
profiles="${@ isar_deb_build_profiles(d)}"
@@ -109,23 +112,27 @@ dpkg_runbuild() {
DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
- sbuild -A -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
+ sbuild -A -n -c ${SBUILD_CHROOT} \
+ --chroot-mode=${ISAR_CHROOT_MODE} \
--host=${PACKAGE_ARCH} --build=${BUILD_ARCH} ${profiles} \
--no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
--bd-uninstallable-explainer=apt \
--no-apt-update --apt-distupgrade \
--chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \
- --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
+ --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;${@'\nAPT::Sandbox::User root;' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}\" > /etc/apt/apt.conf.d/50isar-apt" \
--chroot-setup-commands="rm -f /var/log/dpkg.log" \
--chroot-setup-commands="mkdir -p ${deb_dir}" \
--chroot-setup-commands="find ${ext_deb_dir} -maxdepth 1 -name '*.deb' -exec ln -t ${deb_dir}/ -sf {} +" \
--chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \
--finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-*-dummy_*.deb" \
--finished-build-commands="find ${deb_dir} -maxdepth 1 -type f -name '*.deb' -print -exec cp ${CP_FLAGS} -t ${ext_deb_dir}/ {} +" \
- --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \
+ ${@ '--finished-build-commands="cp /var/log/dpkg.log $ext_root/dpkg_partial.log"' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' } \
--build-path="" --build-dir=${WORKDIR} --dist="${DEBDISTRONAME}" ${DSC_FILE}
- sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
+ # TODO: port to unshare backend
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
+ fi
index 029caec7..9bb43a8d 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -29,8 +29,12 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download"
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')}
+ chroot ${ROOTFSDIR} \
}
ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
@@ -62,6 +66,9 @@ __EOF__
set -e
+
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')}
+
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index 766f386d..cc046fdb 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -16,7 +16,14 @@ do_image_tools[depends] += " \
SCHROOT_MOUNTS = "${WORKDIR}:${PP_WORK} ${IMAGE_ROOTFS}:${PP_ROOTFS} ${DEPLOY_DIR_IMAGE}:${PP_DEPLOY}"
SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
+# only used on unshare
+ROOTFS_IMAGETOOLS ?= "${WORKDIR}/rootfs-imgtools-${BB_CURRENTTASK}"
+
imager_run() {
+ imager_run_${ISAR_CHROOT_MODE} "$@"
+}
+
+imager_run_schroot() {
local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
@@ -103,3 +110,80 @@ generate_imager_sbom() {
--timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS} \
< ${WORKDIR}/imager.manifest
}
+
+imager_run_unshare() {
+ exec 3<&0
+
+ # ignore everything before '--'. If the remaining list is empty,
+ # assume a here document is passed via stdin
+ while [ "$#" -gt 0 ]; do
+ case "$1" in
+ --) shift 1; break ;;
+ *) shift 1 ;;
+ esac
+ done
+
+ if [ "$#" -eq 0 ]; then
+ set -- "$@" '/bin/bash' '-s'
+ fi
+
+ local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+
+ run_privileged_heredoc <<'EOF'
+ set -e
+ mkdir -p ${ROOTFS_IMAGETOOLS}
+ tar -xf "${SBUILD_CHROOT}" -C "${ROOTFS_IMAGETOOLS}"
+ mkdir -p ${ROOTFS_IMAGETOOLS}/isar-apt
+ cp -rL /etc/resolv.conf "${ROOTFS_IMAGETOOLS}/etc"
+EOF
+
+ # setting up error handler
+ imager_cleanup() {
+ run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
+ }
+ trap 'exit 1' INT HUP QUIT TERM ALRM USR1
+ trap 'imager_cleanup' EXIT
+
+ if [ -n "${local_install}" ]; then
+ echo "Installing imager deps: ${local_install}"
+
+ distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+ distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ fi
+
+ E="${@ isar_export_proxies(d)}"
+ deb_dl_dir_import ${ROOTFS_IMAGETOOLS} ${distro}
+ ${SCRIPTSDIR}/lockrun.py -r -f "${REPO_ISAR_DIR}/isar.lock" -s <<'EOAPT'
+ local_install=$local_install ${@run_privileged_cmd(d)} /bin/bash -s <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} apt-get update \
+ -o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \
+ -o Dir::Etc::SourceParts='-' \
+ -o APT::Get::List-Cleanup='0'
+ chroot ${ROOTFS_IMAGETOOLS} apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades --download-only install \
+ $local_install
+EOF
+EOAPT
+
+ deb_dl_dir_export ${ROOTFS_IMAGETOOLS} ${distro}
+ local_install=$local_install run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades install \
+ $local_install
+EOF
+ fi
+
+ run_privileged_heredoc <<'EOF' "$@"
+ set -e
+ mkdir -p ${ROOTFS_IMAGETOOLS}/${SCRIPTSDIR}
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} "$@" <&3
+EOF
+
+ run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
+}
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 9fcdda48..0fa15a87 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -189,6 +189,7 @@ SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --"
python() {
image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
+ chroot_mode = d.getVar('ISAR_CHROOT_MODE')
basetypes = {}
typedeps = {}
@@ -264,7 +265,8 @@ python() {
if image_cmd:
localdata.setVar('type', bt)
cmds.append(localdata.expand(image_cmd))
- cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
+ if chroot_mode == 'schroot':
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
else:
bb.fatal("No IMAGE_CMD for %s" % bt)
vardeps.add('IMAGE_CMD:' + bt_clean)
@@ -294,7 +296,8 @@ python() {
cmd = '\t' + localdata.getVar('CONVERSION_CMD:' + c)
if cmd not in cmds:
cmds.append(cmd)
- cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
+ if chroot_mode == 'schroot':
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
vardeps.add('CONVERSION_CMD:' + c)
for dep in (localdata.getVar('CONVERSION_DEPS:' + c) or '').split():
conversion_install.add(dep)
diff --git a/meta/classes-recipe/imagetypes_container.bbclass b/meta/classes-recipe/imagetypes_container.bbclass
index fb1d0cdf..a68438e9 100644
--- a/meta/classes-recipe/imagetypes_container.bbclass
+++ b/meta/classes-recipe/imagetypes_container.bbclass
@@ -68,7 +68,9 @@ do_containerize() {
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -193,8 +193,10 @@ generate_wic_image() {
fi
EOIMAGER
- run_privileged chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
- run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
index 7352a87c..f31964db 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -145,7 +145,12 @@ rootfs_cmd() {
}
rootfs_do_mounts[weight] = "3"
-rootfs_do_mounts() {
+python rootfs_do_mounts() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'schroot':
+ bb.build.exec_func('rootfs_do_mounts_priv', d)
+}
+
+rootfs_do_mounts_priv() {
EOSUDO
}
-rootfs_do_umounts() {
+python rootfs_do_umounts() {
+ # unconditionally run the unmount code as this ignores missing
+ # mountpoints but also does the cleanup of the directories
+ bb.build.exec_func('rootfs_do_umounts_priv', d)
+}
+
+rootfs_do_umounts_priv() {
run_privileged_heredoc <<'EOSUDO'
set -e
@@ -215,7 +226,11 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}"
rootfs_prepare[weight] = "25"
rootfs_prepare(){
- run_privileged tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
+ rm -rf ${ROOTFSDIR}
+ run_privileged_heredoc << 'EOF'
+EOF
# setup chroot
- -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
- -o Dir::Etc::SourceParts="-" \
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
+ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
+ -o Dir::Etc::SourceParts="-" \
+ -o APT::Get::List-Cleanup="0"
+EOF
}
ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
@@ -316,9 +335,12 @@ rootfs_install_pkgs_download[isar-apt-lock] = "release-after"
- ${ROOTFSDIR} \
- -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot ${ROOTFSDIR} \
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
+EOF
}
ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""
@@ -345,8 +367,12 @@ rootfs_install_pkgs_install[weight] = "8000"
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot "${ROOTFSDIR}" \
}
ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
@@ -653,8 +679,10 @@ rootfs_install_sstate_finalize() {
mkdir -p ${ROOTFSDIR}
- run_privileged tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} < rootfs.tar
+ tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} -f rootfs.tar
+EOF
--- a/meta/classes-recipe/sbuild.bbclass
+++ b/meta/classes-recipe/sbuild.bbclass
@@ -7,7 +7,8 @@ SCHROOT_MOUNTS ?= ""
inherit crossvars
-SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
+SBUILD_CHROOT:unshare ?= "${SCHROOT_DIR}.tar.zst"
+SBUILD_CHROOT:schroot ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
@@ -144,6 +145,13 @@ END
EOSUDO
}
+unshare_configure_ccache() {
+ # ccache must be below /build for file permissions to work properly
+ cat <<'EOF' >> ${SBUILD_CONFIG}
+$path = "/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games";
+EOF
+}
+
sbuild_dpkg_log_export() {
export dpkg_partial_log="${1}"
@@ -152,3 +160,17 @@ sbuild_dpkg_log_export() {
cat ${dpkg_partial_log} >> ${SCHROOT_DIR}/tmp/dpkg_common.log
) 9>"${SCHROOT_DIR}/tmp/dpkg_common.log.lock"
}
+
+# additional mounts managed by sbuild
+sbuild_add_unshare_mounts() {
+ dpkg_prepare_unshare_ccache
+
+ cat <<'EOF' >> ${SBUILD_CONFIG}
+$unshare_bind_mounts = [
+ { directory => '${WORKDIR}/rootfs', mountpoint => '${PP}/rootfs' },
+ { directory => '${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}', mountpoint => '/isar-apt' },
+ { directory => '${REPO_BASE_DIR}', mountpoint => '/base-apt' },
+ { directory => "${CCACHE_DIR}", mountpoint => "/ccache" }
+];
+EOF
+}
diff --git a/meta/classes-recipe/sdk.bbclass b/meta/classes-recipe/sdk.bbclass
index 16165792..7a8d5ff4 100644
--- a/meta/classes-recipe/sdk.bbclass
+++ b/meta/classes-recipe/sdk.bbclass
@@ -74,13 +74,17 @@ rootfs_configure_isar_apt_dir() {
ROOTFS_POSTPROCESS_COMMAND:prepend:class-sdk = "sdkchroot_configscript "
sdkchroot_configscript () {
- run_in_chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
+ chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+EOF
-
- rootfs_do_umounts
+ rootfs_do_umounts_priv
# Remove setup scripts
index 5f339d40..4d3fd62e 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -73,7 +73,7 @@ KERNEL_FILE:arm64 ?= "vmlinux"
MACHINEOVERRIDES ?= "${MACHINE}"
DISTROOVERRIDES ?= "${DISTRO}"
-OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}:forcevariable"
+OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}:${ISAR_CHROOT_MODE}:forcevariable"
FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
@@ -152,6 +152,10 @@ ISAR_APT_RETRIES ??= "${@'10' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAP
ISAR_APT_DELAY_MAX ??= "${@'600' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''}"
ISAR_APT_SNAPSHOT_TIMESTAMP ??= "${SOURCE_DATE_EPOCH}"
+# Rootless build execution
+ISAR_ROOTLESS ??= "0"
+ISAR_CHROOT_MODE ??= "${@'unshare' if bb.utils.to_boolean(d.getVar('ISAR_ROOTLESS')) else 'schroot'}"
+
# Default parallelism and resource usage for xz
XZ_MEMLIMIT ?= "50%"
XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
@@ -207,6 +211,7 @@ CCACHE_DEBUG ?= "0"
# Variables for tasks marking
# Long term TODO: get rid of sudo marked tasks
TASK_USE_NETWORK = "1"
+# nested namespacing requires this as well
TASK_USE_SUDO = "1"
TASK_USE_NETWORK_AND_SUDO = "1"
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index cf6c355c..4d102ed6 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -161,6 +161,8 @@ do_bootstrap() {
line="[trusted=yes] ${line}"
fi
echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list"
+ echo > ${WORKDIR}/mmtmpdir
+ chmod 666 ${WORKDIR}/mmtmpdir
# no need to sync /var/cache/apt/archives if base-apt used
syncin='echo skip sync-in'
@@ -177,12 +179,14 @@ do_bootstrap() {
mkdir -p \$1/base-apt && \
mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \
chroot \$1 apt-get update -y \
- -o APT::Update::Error-Mode=any && \
+ -o APT::Update::Error-Mode=any \
+ ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} && \
chroot \$1 apt-get install -y dpkg && \
umount \$1/base-apt && \
- umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \
- umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp"
+ umount \$1/$base_apt_tmp && \
+ umount $base_apt_tmp && rmdir \$1/$base_apt_tmp"
else
+ # prepare dl_dir for access from both sides (local and rootfs)
deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list"
@@ -202,6 +206,7 @@ do_bootstrap() {
-o Dir::State="$1/var/lib/apt" \
-o Dir::Etc="$1/etc/apt" \
-o Dir::Cache="$1/var/cache/apt" \
+ ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \
-o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}" \
${@get_apt_opts(d, '-o')}'
extra_essential="$extra_essential && $syncout"
@@ -225,7 +230,8 @@ do_bootstrap() {
+ ${@'' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else 'run_privileged'} \
+ TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
$arch_param \
--mode=unshare \
${MMHOOKS} \
@@ -244,6 +250,7 @@ do_bootstrap() {
--customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \
--customize-hook='chroot "$1" /usr/sbin/locale-gen' \
--customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \
+ ${@'--skip=output/dev' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \
--skip=cleanup/apt \
--skip=download/empty \
${MMOPTS} \
@@ -258,7 +265,8 @@ do_bootstrap() {
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
- run_privileged rm -rf --one-file-system "${WORKDIR}/dl_dir"
+ run_privileged find ${WORKDIR}/dl_dir -maxdepth 1 -mindepth 1 -exec rm -rf --one-file-system "{}" \;
+ rmdir ${WORKDIR}/dl_dir
index aa62b324..054d7fc2 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -66,8 +66,28 @@ ROOTFS_POSTPROCESS_COMMAND:remove = "rootfs_cleanup_base_apt"
DEPLOY_SCHROOT = "${@d.getVar('SCHROOT_' + d.getVar('SBUILD_VARIANT').upper() + '_DIR')}${SBUILD_SCHROOT_SUFFIX}"
-do_sbuildchroot_deploy[dirs] = "${DEPLOY_DIR}/schroot-${SBUILD_VARIANT}"
-do_sbuildchroot_deploy() {
+sbuildchroot_deploy_tree() {
ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_SCHROOT}"
}
+sbuildchroot_deploy_tar() {
+ lopts="--one-file-system --exclude=var/cache/apt/archives --exclude=isar-apt"
+ # we cannot use pzstd, as this results in a different magic
+ # (zstd skippable frame) which is not detected by sbuild
+ # https://salsa.debian.org/debian/sbuild/-/blob/d975d388a98627a0d7d112791e441c27a6d529df/lib/Sbuild/ChrootUnshare.pm#L608
+ ZSTD="zstd -${SSTATE_ZSTD_CLEVEL} -T${ZSTD_THREADS}"
+ run_privileged \
+ tar -C ${ROOTFSDIR} -cpS $lopts ${ROOTFS_TAR_ATTR_FLAGS} . \
+ | $ZSTD > ${DEPLOY_SCHROOT}.tar.zst
+ # cleanup extracted rootfs
+
+do_sbuildchroot_deploy[network] = "${TASK_USE_SUDO}"
+do_sbuildchroot_deploy[dirs] += "${DEPLOY_DIR}/schroot-${SBUILD_VARIANT}"
+python do_sbuildchroot_deploy() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ bb.build.exec_func('sbuildchroot_deploy_tar', d)
+ else:
+ bb.build.exec_func('sbuildchroot_deploy_tree', d)
+}
addtask sbuildchroot_deploy before do_build after do_rootfs
--
2.53.0
where mounting file systems is possible. This has proven problematic
for security reasons, both when running in a privileged container or
locally.
To solve this, we implement fully rootless builds that rely on the
unshare syscall which allows us to avoid sudo and instead operate in
temporary kernel namespaces as a user that is just privileged within
that namespace. This comes with some challenges regarding the handling
of mounts (they are cleared when leaving the namespace), as well as
cross namespace deployments (the outer user might not be able to access
the inner data). For that, we rework the handling of mounts and artifact
passing to make it compatible with both chroot modes (schroot and
unshare).
Kconfig | 2 +-
for security reasons, both when running in a privileged container or
locally.
To solve this, we implement fully rootless builds that rely on the
unshare syscall which allows us to avoid sudo and instead operate in
temporary kernel namespaces as a user that is just privileged within
that namespace. This comes with some challenges regarding the handling
of mounts (they are cleared when leaving the namespace), as well as
cross namespace deployments (the outer user might not be able to access
the inner data). For that, we rework the handling of mounts and artifact
passing to make it compatible with both chroot modes (schroot and
unshare).
RECIPE-API-CHANGELOG.md | 21 +++++
doc/user_manual.md | 2 +
kas/isar.yaml | 2 +-
meta/classes-global/base.bbclass | 86 ++++++++++++++++++-
kas/isar.yaml | 2 +-
meta/classes-recipe/deb-dl-dir.bbclass | 9 +-
meta/classes-recipe/dpkg-base.bbclass | 22 ++++-
meta/classes-recipe/dpkg.bbclass | 17 ++--
.../image-locales-extension.bbclass | 9 +-
.../image-tools-extension.bbclass | 84 ++++++++++++++++++
meta/classes-recipe/image.bbclass | 7 +-
.../imagetypes_container.bbclass | 4 +-
meta/classes-recipe/imagetypes_wic.bbclass | 6 +-
meta/classes-recipe/rootfs.bbclass | 52 ++++++++---
meta/classes-recipe/sbuild.bbclass | 24 +++++-
meta/classes-recipe/sdk.bbclass | 10 ++-
meta/conf/bitbake.conf | 7 +-
.../isar-mmdebstrap/isar-mmdebstrap.inc | 18 ++--
.../sbuild-chroot/sbuild-chroot.inc | 24 +++++-
19 files changed, 362 insertions(+), 44 deletions(-)
diff --git a/Kconfig b/Kconfig
index 86a4aac3..66dd4112 100644
--- a/Kconfig
+++ b/Kconfig
@@ -14,7 +14,7 @@ config KAS_INCLUDE_MAIN
config KAS_BUILD_SYSTEM
string
- default "isar"
+ default "isar-rootless"
source "kas/machine/Kconfig"
source "kas/distro/Kconfig"
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index d0aa6e1a..27d14dc4 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1077,3 +1077,24 @@ specifies the rootfs path.
Using these helpers instead of direct `sudo` invocations centralizes platform-specific
privileged execution logic in `base.bbclass`. Direct use of `sudo` is discouraged
in downstream layers.
+
+### Rootless isar execution
+
+Isar is able to run without the need for `sudo` in an environment that
+allows unprivileged users to unshare the kernels `user namespace`. Further,
+a sufficiently large set of sub ids needs to be configured in `/etc/subuid` / `etc/subgid`.
+This range should be `> 65536`, but smaller ranges might work as well, depending on the
+ids used in the rootfs.
+
+A simple check if rootless is supported can be done by running:
+
+```bash
+mmdebstrap --unshare-helper /bin/echo "rootless supported" || echo "rootless not supported"
+```
+
+To enable rootless builds, set the bitbake variable `ISAR_ROOTLESS = "1"`.
+This internally switches the chroot mode from `schroot` to `unshare`.
+
+When using kas, the `build_system` needs to be set to `isar-rootless`, but the final
+interfaces still need to be clarified. Further, kas patches are needed (for details,
+check the kas mailing list).
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 69e8dfef..26041f9a 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -74,6 +74,7 @@ Building `debian-trixie` requires host system >= bookworm.
Install the following packages:
```
apt install \
+ acl \
binfmt-support \
bubblewrap \
bzip2 \
@@ -88,6 +89,7 @@ apt install \
qemu-user-static \
reprepro \
sudo \
+ uidmap \
unzip \
xz-utils \
git-buildpackage \
diff --git a/kas/isar.yaml b/kas/isar.yaml
index 16ce8b42..3cfc4f96 100644
--- a/kas/isar.yaml
+++ b/kas/isar.yaml
@@ -4,7 +4,7 @@
header:
version: 14
-build_system: isar
+build_system: isar-rootless
repos:
isar:
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index 90e4525e..7167cbb1 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -141,7 +141,9 @@ root_cleandirs() {
die "Could not remove $i, because subdir is mounted"
done
for i in $ROOT_CLEANDIRS_DIRS; do
- run_privileged rm -rf --one-file-system "$TMPDIR$i"
done
for i in $ROOT_CLEANDIRS_DIRS; do
+ [ -d "$TMPDIR$i" ] || continue
+ find "$TMPDIR$i" \( ! -user "$(whoami)" -type d -prune \) -exec ${RUN_PRIVILEGED_CMD} rm -rf --one-file-system {} \;
+ rm -rf --one-file-system "$TMPDIR$i"
mkdir -p "$TMPDIR$i"
done
}
@@ -380,7 +382,28 @@ def deb_list_beautify(d, varname):
# shall be used outside of this class.
def insert_isar_mounts(d, rootfs, mounts):
+ """
+ In unshare mode, all mounts must be created after unsharing the
+ mount namespace. As needs to happen within the unshared session,
+ we implement it as a code generator. Note, that the random and urandom
+ mounts are needed for DDI images.
+ """
lines = []
+ to_touch = ['/dev/null', '/dev/random', '/dev/urandom']
+ to_mkdir = ['/dev/pts', '/dev/shm']
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ lines.append('touch ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_touch]))
+ lines.append('mkdir -p ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_mkdir]))
+ lines.append('mount -o bind,private,mode=666 /dev/null {}/dev/null'.format(rootfs))
+ lines.append('mount -t devpts -o noexec,nosuid,uid=5,mode=620,ptmxmode=666 none {}/dev/pts'.format(rootfs))
+ lines.append('( cd {}/dev; ln -sf pts/ptmx . )'.format(rootfs))
+ lines.append('mount -t tmpfs none {}/dev/shm'.format(rootfs))
+ lines.append('mount -o bind /dev/random {}/dev/random'.format(rootfs))
+ lines.append('mount -o bind /dev/urandom {}/dev/urandom'.format(rootfs))
+ lines.append('mount -t proc none {}/proc'.format(rootfs))
+ # we do not unshare the network namespace, so we cannot create a sysfs, hence bind-mount
+ lines.append('mount -o rbind /sys {}/sys'.format(rootfs))
+
for m in mounts.split():
for m in mounts.split():
host, inner = m.split(':') if ':' in m else (m, m)
inner_full = os.path.join(rootfs, inner[1:])
@@ -389,7 +412,18 @@ def insert_isar_mounts(d, rootfs, mounts):
return '\n'.join(lines)
def insert_isar_umounts(d, rootfs, mounts):
+ """
+ In unshare mount we don't unmount the system mounts but just
+ remove the mountpoints.
+ """
lines = []
+ to_unlink = ['/dev/null', '/dev/random', '/dev/urandom', '/dev/ptmx']
+ to_rmdir = ['/dev/pts', '/dev/shm']
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ lines.append('rm -f ' + ' '.join(['{}/{}'.format(rootfs, f) for f in to_unlink]))
+ for d in ['{}/{}'.format(rootfs, _d) for _d in to_rmdir]:
+ lines.append('[ -d {} ] && rmdir {}'.format(d, d))
+
for m in mounts.split():
for m in mounts.split():
host, inner = m.split(':') if ':' in m else (m, m)
mp = '{}/{}'.format(rootfs, inner)
@@ -397,11 +431,52 @@ def insert_isar_umounts(d, rootfs, mounts):
lines.append('[ -d {} ] && rmdir --ignore-fail-on-non-empty {}'.format(mp, mp))
return '\n'.join(lines)
+def get_subid_range(idmap, d):
+ import getpass
+ with open(idmap, 'r') as f:
+ entries = f.readlines()
+ for e in entries:
+ user, base, cnt = e.split(':')
+ if user == os.getuid() or user == getpass.getuser():
+ return int(base), int(cnt)
+ bb.error("No sub-id range specified in %s" % idmap)
+
def run_privileged_cmd(d):
- cmd = 'sudo -E'
+ """
+ In unshare mode we need to map the rootfs uid/gid range into the
+ subuid/subgid range of the parent namespace. As we usually only
+ get 65534 ids, we cannot map the whole range, as two ids are already
+ used by the calling environment (root and builder user). Hence, map
+ as much as we can but also map the highest id (nobody / nogroup) as
+ these are used within the rootfs. It would be easier to use
+ mmdebstrap --unshare-helper as command (which is also internally used
+ by sbuild), but this only maps linear ranges, hence it cannot map the
+ nobody / nogroup on the default subid range. By that, we have to avoid
+ the nobody / nogroup when building packages in this case.
+ """
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ nobody_id = 65534
+ uid_base, uid_cnt = get_subid_range('/etc/subuid', d)
+ nobody_subid = uid_base + uid_cnt - 1
+ gid_base, gid_cnt = get_subid_range('/etc/subgid', d)
+ nogroup_subid = gid_base + gid_cnt - 1
+ cmd = 'unshare --mount --pid --uts --ipc --user' \
+ ' --kill-child' \
+ ' --setuid 0 --setgid 0 --fork' \
+ f' --map-users 1:{uid_base+1}:{uid_cnt-2}' \
+ f' --map-groups 1:{gid_base+1}:{gid_cnt-2}'
+ if uid_cnt < nobody_id:
+ cmd += f' --map-users {nobody_id}:{nobody_subid}:1'
+ if gid_cnt < nobody_id:
+ cmd += f' --map-groups {nobody_id}:{nogroup_subid}:1'
+ cmd += " --map-root-user"
+ else:
+ cmd = 'sudo -E'
bb.debug(1, "privileged cmd: %s" % cmd)
return cmd
+UNSHARE_SUBUID_BASE := "${@get_subid_range('/etc/subuid', d)[0] if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '0'}"
+# store in variable to only compute once and make available to fetcher
RUN_PRIVILEGED_CMD := "${@run_privileged_cmd(d)}"
run_privileged() {
@@ -415,5 +490,10 @@ run_privileged_heredoc() {
run_in_chroot() {
rootfs="$1"
shift
- ${RUN_PRIVILEGED_CMD} chroot "$rootfs" "$@"
+
+ rootfs=$rootfs run_privileged_heredoc <<'EORIC' "$@"
+ set -e
+ ${@insert_isar_mounts(d, '$rootfs', '')}
+ chroot "$rootfs" "$@"
+EORIC
}
diff --git a/meta/classes-recipe/deb-dl-dir.bbclass b/meta/classes-recipe/deb-dl-dir.bbclass
index 04fd6414..0e268f06 100644
--- a/meta/classes-recipe/deb-dl-dir.bbclass
+++ b/meta/classes-recipe/deb-dl-dir.bbclass
@@ -123,8 +123,13 @@ deb_dl_dir_import() {
# let our unprivileged user place downloaded packages in /var/cache/apt/archives/
- mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
- chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
+ if [ "${ISAR_CHROOT_MODE}" = "unshare" ]; then
+ mkdir -p "${rootfs}"/var/cache/apt/archives
+ chmod 777 "${rootfs}"/var/cache/apt/archives
+ else
+ mkdir -p "${rootfs}"/var/cache/apt/archives/partial/
+ chown -R ${uid}:${gid} "${rootfs}"/var/cache/apt/archives/
+ fi
EOSUDO
# nothing to copy if download directory does not exist just yet
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index e8721c79..a0d4fd05 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -168,12 +168,30 @@ dpkg_schroot_create_configs() {
EOSUDO
}
+dpkg_chroot_prepare() {
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ dpkg_schroot_create_configs
+ fi
+}
+
+dpkg_chroot_finalize() {
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ schroot_delete_configs
+ fi
+}
+
+dpkg_prepare_unshare_ccache() {
+ mkdir -p "${CCACHE_DIR}"
+ # sbuild id from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1110942
+ setfacl -m u:${UNSHARE_SUBUID_BASE}:rwX -m u:${@int(d.getVar('UNSHARE_SUBUID_BASE')) + 999}:rwx "${CCACHE_DIR}"
+}
+
python do_dpkg_build() {
- bb.build.exec_func('dpkg_schroot_create_configs', d)
+ bb.build.exec_func('dpkg_chroot_prepare', d)
try:
bb.build.exec_func("dpkg_runbuild", d)
finally:
- bb.build.exec_func('schroot_delete_configs', d)
+ bb.build.exec_func('dpkg_chroot_finalize', d)
}
do_dpkg_build[network] = "${TASK_USE_NETWORK_AND_SUDO}"
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index dcdef487..57fe042b 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -85,7 +85,10 @@ dpkg_runbuild() {
ext_deb_dir="${ext_root}${deb_dir}"
if [ ${USE_CCACHE} -eq 1 ]; then
- schroot_configure_ccache
+ ${ISAR_CHROOT_MODE}_configure_ccache
+ fi
+ if [ "${ISAR_CHROOT_MODE}" = "unshare" ]; then
+ sbuild_add_unshare_mounts
fi
profiles="${@ isar_deb_build_profiles(d)}"
@@ -109,23 +112,27 @@ dpkg_runbuild() {
DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
- sbuild -A -n -c ${SBUILD_CHROOT} --chroot-mode=schroot \
+ sbuild -A -n -c ${SBUILD_CHROOT} \
+ --chroot-mode=${ISAR_CHROOT_MODE} \
--host=${PACKAGE_ARCH} --build=${BUILD_ARCH} ${profiles} \
--no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
--bd-uninstallable-explainer=apt \
--no-apt-update --apt-distupgrade \
--chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \
- --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
+ --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;${@'\nAPT::Sandbox::User root;' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}\" > /etc/apt/apt.conf.d/50isar-apt" \
--chroot-setup-commands="rm -f /var/log/dpkg.log" \
--chroot-setup-commands="mkdir -p ${deb_dir}" \
--chroot-setup-commands="find ${ext_deb_dir} -maxdepth 1 -name '*.deb' -exec ln -t ${deb_dir}/ -sf {} +" \
--chroot-setup-commands="apt-get update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"" \
--finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-*-dummy_*.deb" \
--finished-build-commands="find ${deb_dir} -maxdepth 1 -type f -name '*.deb' -print -exec cp ${CP_FLAGS} -t ${ext_deb_dir}/ {} +" \
- --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \
+ ${@ '--finished-build-commands="cp /var/log/dpkg.log $ext_root/dpkg_partial.log"' if d.getVar('ISAR_CHROOT_MODE') == 'schroot' else '' } \
--build-path="" --build-dir=${WORKDIR} --dist="${DEBDISTRONAME}" ${DSC_FILE}
- sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
+ # TODO: port to unshare backend
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
+ fi
deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
# Cleanup apt artifacts
diff --git a/meta/classes-recipe/image-locales-extension.bbclass b/meta/classes-recipe/image-locales-extension.bbclass
# Cleanup apt artifacts
index 029caec7..9bb43a8d 100644
--- a/meta/classes-recipe/image-locales-extension.bbclass
+++ b/meta/classes-recipe/image-locales-extension.bbclass
@@ -29,8 +29,12 @@ ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT += "image_install_localepurge_download"
image_install_localepurge_download[weight] = "40"
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
- run_in_chroot '${ROOTFSDIR}' \
image_install_localepurge_download[network] = "${TASK_USE_NETWORK_AND_SUDO}"
image_install_localepurge_download() {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS') if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else '')}
+ chroot ${ROOTFSDIR} \
/usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only localepurge
+EOF
}
ROOTFS_INSTALL_COMMAND += "image_install_localepurge_install"
@@ -62,6 +66,9 @@ __EOF__
# Install configuration into image:
run_privileged_heredoc <<'EOSUDO'
set -e
+
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), '')}
+
localepurge_state='i'
if chroot '${ROOTFSDIR}' dpkg -s localepurge 2>/dev/null >&2
then
if chroot '${ROOTFSDIR}' dpkg -s localepurge 2>/dev/null >&2
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index 766f386d..cc046fdb 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -16,7 +16,14 @@ do_image_tools[depends] += " \
SCHROOT_MOUNTS = "${WORKDIR}:${PP_WORK} ${IMAGE_ROOTFS}:${PP_ROOTFS} ${DEPLOY_DIR_IMAGE}:${PP_DEPLOY}"
SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
+# only used on unshare
+ROOTFS_IMAGETOOLS ?= "${WORKDIR}/rootfs-imgtools-${BB_CURRENTTASK}"
+
imager_run() {
+ imager_run_${ISAR_CHROOT_MODE} "$@"
+}
+
+imager_run_schroot() {
local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
@@ -103,3 +110,80 @@ generate_imager_sbom() {
--timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS} \
< ${WORKDIR}/imager.manifest
}
+
+imager_run_unshare() {
+ exec 3<&0
+
+ # ignore everything before '--'. If the remaining list is empty,
+ # assume a here document is passed via stdin
+ while [ "$#" -gt 0 ]; do
+ case "$1" in
+ --) shift 1; break ;;
+ *) shift 1 ;;
+ esac
+ done
+
+ if [ "$#" -eq 0 ]; then
+ set -- "$@" '/bin/bash' '-s'
+ fi
+
+ local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+
+ run_privileged_heredoc <<'EOF'
+ set -e
+ mkdir -p ${ROOTFS_IMAGETOOLS}
+ tar -xf "${SBUILD_CHROOT}" -C "${ROOTFS_IMAGETOOLS}"
+ mkdir -p ${ROOTFS_IMAGETOOLS}/isar-apt
+ cp -rL /etc/resolv.conf "${ROOTFS_IMAGETOOLS}/etc"
+EOF
+
+ # setting up error handler
+ imager_cleanup() {
+ run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
+ }
+ trap 'exit 1' INT HUP QUIT TERM ALRM USR1
+ trap 'imager_cleanup' EXIT
+
+ if [ -n "${local_install}" ]; then
+ echo "Installing imager deps: ${local_install}"
+
+ distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+ distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ fi
+
+ E="${@ isar_export_proxies(d)}"
+ deb_dl_dir_import ${ROOTFS_IMAGETOOLS} ${distro}
+ ${SCRIPTSDIR}/lockrun.py -r -f "${REPO_ISAR_DIR}/isar.lock" -s <<'EOAPT'
+ local_install=$local_install ${@run_privileged_cmd(d)} /bin/bash -s <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} apt-get update \
+ -o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \
+ -o Dir::Etc::SourceParts='-' \
+ -o APT::Get::List-Cleanup='0'
+ chroot ${ROOTFS_IMAGETOOLS} apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades --download-only install \
+ $local_install
+EOF
+EOAPT
+
+ deb_dl_dir_export ${ROOTFS_IMAGETOOLS} ${distro}
+ local_install=$local_install run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades install \
+ $local_install
+EOF
+ fi
+
+ run_privileged_heredoc <<'EOF' "$@"
+ set -e
+ mkdir -p ${ROOTFS_IMAGETOOLS}/${SCRIPTSDIR}
+ ${@insert_isar_mounts(d, d.getVar('ROOTFS_IMAGETOOLS'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${ROOTFS_IMAGETOOLS} "$@" <&3
+EOF
+
+ run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
+}
diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 9fcdda48..0fa15a87 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -189,6 +189,7 @@ SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --"
python() {
image_types = (d.getVar('IMAGE_FSTYPES') or '').split()
conversions = set(d.getVar('IMAGE_CONVERSIONS').split())
+ chroot_mode = d.getVar('ISAR_CHROOT_MODE')
basetypes = {}
typedeps = {}
@@ -264,7 +265,8 @@ python() {
if image_cmd:
localdata.setVar('type', bt)
cmds.append(localdata.expand(image_cmd))
- cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
+ if chroot_mode == 'schroot':
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}'))
else:
bb.fatal("No IMAGE_CMD for %s" % bt)
vardeps.add('IMAGE_CMD:' + bt_clean)
@@ -294,7 +296,8 @@ python() {
cmd = '\t' + localdata.getVar('CONVERSION_CMD:' + c)
if cmd not in cmds:
cmds.append(cmd)
- cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
+ if chroot_mode == 'schroot':
+ cmds.append(localdata.expand('\tsudo chown $(id -u):$(id -g) ${IMAGE_FILE_HOST}.%s' % c))
vardeps.add('CONVERSION_CMD:' + c)
for dep in (localdata.getVar('CONVERSION_DEPS:' + c) or '').split():
conversion_install.add(dep)
diff --git a/meta/classes-recipe/imagetypes_container.bbclass b/meta/classes-recipe/imagetypes_container.bbclass
index fb1d0cdf..a68438e9 100644
--- a/meta/classes-recipe/imagetypes_container.bbclass
+++ b/meta/classes-recipe/imagetypes_container.bbclass
@@ -68,7 +68,9 @@ do_containerize() {
run_privileged rm -rf "${oci_img_dir}_unpacked"
# no root needed anymore
- run_privileged chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
# no root needed anymore
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ run_privileged chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+ fi
}
convert_container() {
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 8b048dc7..3e261622 100644
convert_container() {
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -193,8 +193,10 @@ generate_wic_image() {
fi
EOIMAGER
- run_privileged chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
- run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+ if [ "${ISAR_CHROOT_MODE}" = "schroot" ]; then
+ run_privileged chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
+ run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
+ fi
+ run_privileged chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
rm -rf ${IMAGE_ROOTFS}/../pseudo
cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
cat ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.manifest \
index 7352a87c..f31964db 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -145,7 +145,12 @@ rootfs_cmd() {
}
rootfs_do_mounts[weight] = "3"
-rootfs_do_mounts() {
+python rootfs_do_mounts() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'schroot':
+ bb.build.exec_func('rootfs_do_mounts_priv', d)
+}
+
+rootfs_do_mounts_priv() {
run_privileged_heredoc <<'EOSUDO'
set -e
mountpoint -q '${ROOTFSDIR}/dev' || \
@@ -168,7 +173,13 @@ rootfs_do_mounts() {
set -e
mountpoint -q '${ROOTFSDIR}/dev' || \
EOSUDO
}
-rootfs_do_umounts() {
+python rootfs_do_umounts() {
+ # unconditionally run the unmount code as this ignores missing
+ # mountpoints but also does the cleanup of the directories
+ bb.build.exec_func('rootfs_do_umounts_priv', d)
+}
+
+rootfs_do_umounts_priv() {
run_privileged_heredoc <<'EOSUDO'
set -e
@@ -215,7 +226,11 @@ ROOTFS_EXTRA_IMPORTED := "${@rootfs_extra_import(d)}"
rootfs_prepare[weight] = "25"
rootfs_prepare(){
+ rm -rf ${ROOTFSDIR}
+ run_privileged_heredoc << 'EOF'
+ mkdir -p ${ROOTFSDIR}
+ tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console"
+EOF
# setup chroot
run_privileged "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}"
@@ -285,10 +300,14 @@ rootfs_install_pkgs_update[weight] = "5"
rootfs_install_pkgs_update[isar-apt-lock] = "acquire-before"
rootfs_install_pkgs_update[network] = "${TASK_USE_NETWORK_AND_SUDO}"
rootfs_install_pkgs_update() {
- run_in_chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
rootfs_install_pkgs_update[network] = "${TASK_USE_NETWORK_AND_SUDO}"
rootfs_install_pkgs_update() {
- -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
- -o Dir::Etc::SourceParts="-" \
- -o APT::Get::List-Cleanup="0"
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
+ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
+ -o Dir::Etc::SourceParts="-" \
+ -o APT::Get::List-Cleanup="0"
+EOF
}
ROOTFS_INSTALL_COMMAND += "rootfs_install_resolvconf"
@@ -316,9 +335,12 @@ rootfs_install_pkgs_download[isar-apt-lock] = "release-after"
rootfs_install_pkgs_download[network] = "${TASK_USE_NETWORK}"
rootfs_install_pkgs_download() {
rootfs_install_pkgs_download() {
# download packages using apt in a non-privileged namespace
- rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
- ${ROOTFSDIR} \
- -- /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot ${ROOTFSDIR} \
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} -oDebug::NoLocking=1 --download-only ${ROOTFS_PACKAGES}
+EOF
}
ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= ""
@@ -345,8 +367,12 @@ rootfs_install_pkgs_install[weight] = "8000"
rootfs_install_pkgs_install[progress] = "custom:rootfs_progress.PkgsInstallProgressHandler"
rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
rootfs_install_pkgs_install() {
- run_in_chroot "${ROOTFSDIR}" \
rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
rootfs_install_pkgs_install() {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ chroot "${ROOTFSDIR}" \
/usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
+EOF
}
ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
@@ -653,8 +679,10 @@ rootfs_install_sstate_finalize() {
# - after building the rootfs, the tar won't be there, but we also don't need to unpack
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
+ run_privileged_heredoc <<'EOF'
# - after restoring from cache, there will be a tar which we unpack and then delete
if [ -f rootfs.tar ]; then
mkdir -p ${ROOTFSDIR}
- run_privileged tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} < rootfs.tar
+ tar -C ${ROOTFSDIR} -xp ${SSTATE_TAR_ATTR_FLAGS} -f rootfs.tar
+EOF
rm rootfs.tar
fi
}
diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass
index d9ccce7f..8ca66138 100644
fi
}
diff --git a/meta/classes-recipe/sbuild.bbclass b/meta/classes-recipe/sbuild.bbclass
--- a/meta/classes-recipe/sbuild.bbclass
+++ b/meta/classes-recipe/sbuild.bbclass
@@ -7,7 +7,8 @@ SCHROOT_MOUNTS ?= ""
inherit crossvars
-SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
+SBUILD_CHROOT:unshare ?= "${SCHROOT_DIR}.tar.zst"
+SBUILD_CHROOT:schroot ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}"
SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}"
SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
@@ -144,6 +145,13 @@ END
EOSUDO
}
+unshare_configure_ccache() {
+ # ccache must be below /build for file permissions to work properly
+ cat <<'EOF' >> ${SBUILD_CONFIG}
+$path = "/usr/lib/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games";
+EOF
+}
+
sbuild_dpkg_log_export() {
export dpkg_partial_log="${1}"
@@ -152,3 +160,17 @@ sbuild_dpkg_log_export() {
cat ${dpkg_partial_log} >> ${SCHROOT_DIR}/tmp/dpkg_common.log
) 9>"${SCHROOT_DIR}/tmp/dpkg_common.log.lock"
}
+
+# additional mounts managed by sbuild
+sbuild_add_unshare_mounts() {
+ dpkg_prepare_unshare_ccache
+
+ cat <<'EOF' >> ${SBUILD_CONFIG}
+$unshare_bind_mounts = [
+ { directory => '${WORKDIR}/rootfs', mountpoint => '${PP}/rootfs' },
+ { directory => '${WORKDIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt/${DISTRO}', mountpoint => '/isar-apt' },
+ { directory => '${REPO_BASE_DIR}', mountpoint => '/base-apt' },
+ { directory => "${CCACHE_DIR}", mountpoint => "/ccache" }
+];
+EOF
+}
diff --git a/meta/classes-recipe/sdk.bbclass b/meta/classes-recipe/sdk.bbclass
index 16165792..7a8d5ff4 100644
--- a/meta/classes-recipe/sdk.bbclass
+++ b/meta/classes-recipe/sdk.bbclass
@@ -74,13 +74,17 @@ rootfs_configure_isar_apt_dir() {
ROOTFS_POSTPROCESS_COMMAND:prepend:class-sdk = "sdkchroot_configscript "
sdkchroot_configscript () {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ ${@insert_isar_mounts(d, d.getVar('ROOTFSDIR'), d.getVar('ROOTFS_MOUNTS')) if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''}
+ cp -rL /etc/resolv.conf '${ROOTFSDIR}/etc'
+ chroot ${ROOTFSDIR} /configscript.sh ${DISTRO_ARCH}
+EOF
}
ROOTFS_POSTPROCESS_COMMAND:append:class-sdk = " sdkchroot_finalize"
sdkchroot_finalize() {
ROOTFS_POSTPROCESS_COMMAND:append:class-sdk = " sdkchroot_finalize"
-
- rootfs_do_umounts
+ rootfs_do_umounts_priv
# Remove setup scripts
run_privileged rm -f ${ROOTFSDIR}/chroot-setup.sh ${ROOTFSDIR}/configscript.sh
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 5f339d40..4d3fd62e 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -73,7 +73,7 @@ KERNEL_FILE:arm64 ?= "vmlinux"
MACHINEOVERRIDES ?= "${MACHINE}"
DISTROOVERRIDES ?= "${DISTRO}"
-OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}:forcevariable"
+OVERRIDES = "${PACKAGE_ARCH}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${BASE_DISTRO_CODENAME}:${ISAR_CHROOT_MODE}:forcevariable"
FILESOVERRIDES = "${PACKAGE_ARCH}:${MACHINE}"
# Setting default QEMU_ARCH variables for different DISTRO_ARCH:
@@ -152,6 +152,10 @@ ISAR_APT_RETRIES ??= "${@'10' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAP
ISAR_APT_DELAY_MAX ??= "${@'600' if bb.utils.to_boolean(d.getVar('ISAR_USE_APT_SNAPSHOT')) else ''}"
ISAR_APT_SNAPSHOT_TIMESTAMP ??= "${SOURCE_DATE_EPOCH}"
+# Rootless build execution
+ISAR_ROOTLESS ??= "0"
+ISAR_CHROOT_MODE ??= "${@'unshare' if bb.utils.to_boolean(d.getVar('ISAR_ROOTLESS')) else 'schroot'}"
+
# Default parallelism and resource usage for xz
XZ_MEMLIMIT ?= "50%"
XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
@@ -207,6 +211,7 @@ CCACHE_DEBUG ?= "0"
# Variables for tasks marking
# Long term TODO: get rid of sudo marked tasks
TASK_USE_NETWORK = "1"
+# nested namespacing requires this as well
TASK_USE_SUDO = "1"
TASK_USE_NETWORK_AND_SUDO = "1"
diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
index cf6c355c..4d102ed6 100644
--- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
+++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
@@ -161,6 +161,8 @@ do_bootstrap() {
line="[trusted=yes] ${line}"
fi
echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list"
+ echo > ${WORKDIR}/mmtmpdir
+ chmod 666 ${WORKDIR}/mmtmpdir
# no need to sync /var/cache/apt/archives if base-apt used
syncin='echo skip sync-in'
@@ -177,12 +179,14 @@ do_bootstrap() {
mkdir -p \$1/base-apt && \
mount -o bind,private '${REPO_BASE_DIR}' \$1/base-apt && \
chroot \$1 apt-get update -y \
- -o APT::Update::Error-Mode=any && \
+ -o APT::Update::Error-Mode=any \
+ ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} && \
chroot \$1 apt-get install -y dpkg && \
umount \$1/base-apt && \
- umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \
- umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp"
+ umount \$1/$base_apt_tmp && \
+ umount $base_apt_tmp && rmdir \$1/$base_apt_tmp"
else
+ # prepare dl_dir for access from both sides (local and rootfs)
deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list"
@@ -202,6 +206,7 @@ do_bootstrap() {
-o Dir::State="$1/var/lib/apt" \
-o Dir::Etc="$1/etc/apt" \
-o Dir::Cache="$1/var/cache/apt" \
+ ${@'-o APT::Sandbox::User=root' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \
-o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}" \
${@get_apt_opts(d, '-o')}'
extra_essential="$extra_essential && $syncout"
@@ -225,7 +230,8 @@ do_bootstrap() {
mkdir -p ${DEBDIR}
touch ${DEB_DL_LOCK}
- run_privileged TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
touch ${DEB_DL_LOCK}
+ ${@'' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else 'run_privileged'} \
+ TMPDIR="${BOOTSTRAP_TMPDIR}" mmdebstrap $bootstrap_args \
$arch_param \
--mode=unshare \
${MMHOOKS} \
@@ -244,6 +250,7 @@ do_bootstrap() {
--customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \
--customize-hook='chroot "$1" /usr/sbin/locale-gen' \
--customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \
+ ${@'--skip=output/dev' if d.getVar('ISAR_CHROOT_MODE') == 'unshare' else ''} \
--skip=cleanup/apt \
--skip=download/empty \
${MMOPTS} \
@@ -258,7 +265,8 @@ do_bootstrap() {
if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then
deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ run_privileged find ${WORKDIR}/dl_dir -maxdepth 1 -mindepth 1 -exec rm -rf --one-file-system "{}" \;
+ rmdir ${WORKDIR}/dl_dir
fi
}
addtask bootstrap before do_build after do_generate_keyrings
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
}
addtask bootstrap before do_build after do_generate_keyrings
index aa62b324..054d7fc2 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -66,8 +66,28 @@ ROOTFS_POSTPROCESS_COMMAND:remove = "rootfs_cleanup_base_apt"
DEPLOY_SCHROOT = "${@d.getVar('SCHROOT_' + d.getVar('SBUILD_VARIANT').upper() + '_DIR')}${SBUILD_SCHROOT_SUFFIX}"
-do_sbuildchroot_deploy[dirs] = "${DEPLOY_DIR}/schroot-${SBUILD_VARIANT}"
-do_sbuildchroot_deploy() {
+sbuildchroot_deploy_tree() {
ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_SCHROOT}"
}
+sbuildchroot_deploy_tar() {
+ lopts="--one-file-system --exclude=var/cache/apt/archives --exclude=isar-apt"
+ # we cannot use pzstd, as this results in a different magic
+ # (zstd skippable frame) which is not detected by sbuild
+ # https://salsa.debian.org/debian/sbuild/-/blob/d975d388a98627a0d7d112791e441c27a6d529df/lib/Sbuild/ChrootUnshare.pm#L608
+ ZSTD="zstd -${SSTATE_ZSTD_CLEVEL} -T${ZSTD_THREADS}"
+ run_privileged \
+ tar -C ${ROOTFSDIR} -cpS $lopts ${ROOTFS_TAR_ATTR_FLAGS} . \
+ | $ZSTD > ${DEPLOY_SCHROOT}.tar.zst
+ # cleanup extracted rootfs
+ run_privileged rm -rf ${ROOTFSDIR}
+}
+
+do_sbuildchroot_deploy[network] = "${TASK_USE_SUDO}"
+do_sbuildchroot_deploy[dirs] += "${DEPLOY_DIR}/schroot-${SBUILD_VARIANT}"
+python do_sbuildchroot_deploy() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ bb.build.exec_func('sbuildchroot_deploy_tar', d)
+ else:
+ bb.build.exec_func('sbuildchroot_deploy_tree', d)
+}
addtask sbuildchroot_deploy before do_build after do_rootfs
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
meta/lib/aptsrc_fetcher.py | 75 ++++++++++++++++++++++++++++++++++++++
1 file changed, 75 insertions(+)
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index 1d133aae..933480ea 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -7,10 +7,13 @@ from bb.fetch2 import FetchError
from bb.fetch2 import FetchMethod
class AptSrc(FetchMethod):
@classmethod
def create(cls, d):
return AptSrcSchroot()
def supports(self, ud, d):
@@ -92,3 +95,75 @@ class AptSrcSchroot(AptSrc):
+class AptSrcUnshare(AptSrc):
+ def _setup_chroot(self, rootfsdir, d):
+ sbuild_chroot = d.getVar('SBUILD_CHROOT')
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+
+ runfetchcmd(
+ f'''
+{unshare_cmd} /bin/bash -s <<EOF
+ mkdir -p {rootfsdir}
+ tar -xf {sbuild_chroot} -C {rootfsdir}
+ cp /etc/resolv.conf {os.path.join(rootfsdir, 'etc/resolv.conf')}
+EOF
+ ''', d)
+ logger.info(f'rootfs extracted to: {rootfsdir}')
+
+ def _teardown_chroot(self, rootfsdir, d):
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+ runfetchcmd(f'{unshare_cmd} rm -rf {rootfsdir}', d)
+
+ def download(self, ud, d):
+ bb.utils.exec_flat_python_func('isar_export_proxies', d)
+
+ workdir = d.getVar('WORKDIR')
+ rootfsdir = os.path.join(workdir, 'rootfs-fetcher')
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+
+ if not os.path.exists(os.path.join(rootfsdir, 'etc')):
+ self._setup_chroot(rootfsdir, d)
+
+ repo_isar_dir = d.getVar('REPO_ISAR_DIR')
+ lockfile = bb.utils.lockfile(f'{repo_isar_dir}/isar.lock')
+ os.makedirs(self.localpath(ud, d))
+
+ try:
+ runfetchcmd(f'''
+set -e
+{unshare_cmd} /bin/bash -s <<'EOF' | tar -C {self.localpath(ud, d)} -x
+ chroot {rootfsdir} /bin/bash -c '
+ set -e
+ TMPDIR=$(mktemp -d)
+ mkdir -p $TMPDIR/{ud.localfile}
+ cd $TMPDIR/{ud.localfile}
+ apt-get -y -oDebug::NoLocking=1 --download-only --only-source source {ud.src_package} >/dev/null;
+ tar -c --owner=0 --group=0 --numeric-owner .
+ '
+EOF
+ ''', d)
+ except (OSError, FetchError):
+ raise
+ finally:
+ bb.utils.unlockfile(lockfile)
+ self._teardown_chroot(rootfsdir, d)
+
+ def unpack(self, ud, rootdir, d):
+ workdir = d.getVar('WORKDIR')
+ rootfsdir = os.path.join(workdir, 'rootfs-fetcher')
+ extractto = f'{d.getVar("S")}.dpkg'
+ bb.utils.remove(extractto, recurse=True)
+
+ try:
+ runfetchcmd(f'''
+ set -e
+ find {self.localpath(ud, d)} -print -type f -name '*.dsc' -exec dpkg-source -su -x {{}} {extractto} \\;
+ find {extractto} -mindepth 1 -maxdepth 1 -exec mv {{}} {d.getVar('S')}/ \\;
+ ''', d)
+ except (OSError, FetchError):
+ raise
+ finally:
+ bb.utils.remove(extractto, recurse=True)
+ self._teardown_chroot(rootfsdir, d)
--
2.53.0
1 file changed, 75 insertions(+)
diff --git a/meta/lib/aptsrc_fetcher.py b/meta/lib/aptsrc_fetcher.py
index 1d133aae..933480ea 100644
--- a/meta/lib/aptsrc_fetcher.py
+++ b/meta/lib/aptsrc_fetcher.py
@@ -7,10 +7,13 @@ from bb.fetch2 import FetchError
from bb.fetch2 import FetchMethod
from bb.fetch2 import logger
from bb.fetch2 import runfetchcmd
+import os
from bb.fetch2 import runfetchcmd
class AptSrc(FetchMethod):
@classmethod
def create(cls, d):
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ return AptSrcUnshare()
return AptSrcSchroot()
def supports(self, ud, d):
@@ -92,3 +95,75 @@ class AptSrcSchroot(AptSrc):
finally:
runfetchcmd(f'schroot -q -f -e -c {session_id}', d)
runfetchcmd(f'schroot -q -f -e -c {session_id}', d)
bb.build.exec_func('schroot_delete_configs', d)
+
+
+
+class AptSrcUnshare(AptSrc):
+ def _setup_chroot(self, rootfsdir, d):
+ sbuild_chroot = d.getVar('SBUILD_CHROOT')
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+
+ runfetchcmd(
+ f'''
+{unshare_cmd} /bin/bash -s <<EOF
+ mkdir -p {rootfsdir}
+ tar -xf {sbuild_chroot} -C {rootfsdir}
+ cp /etc/resolv.conf {os.path.join(rootfsdir, 'etc/resolv.conf')}
+EOF
+ ''', d)
+ logger.info(f'rootfs extracted to: {rootfsdir}')
+
+ def _teardown_chroot(self, rootfsdir, d):
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+ runfetchcmd(f'{unshare_cmd} rm -rf {rootfsdir}', d)
+
+ def download(self, ud, d):
+ bb.utils.exec_flat_python_func('isar_export_proxies', d)
+
+ workdir = d.getVar('WORKDIR')
+ rootfsdir = os.path.join(workdir, 'rootfs-fetcher')
+ unshare_cmd = d.getVar('RUN_PRIVILEGED_CMD')
+
+ if not os.path.exists(os.path.join(rootfsdir, 'etc')):
+ self._setup_chroot(rootfsdir, d)
+
+ repo_isar_dir = d.getVar('REPO_ISAR_DIR')
+ lockfile = bb.utils.lockfile(f'{repo_isar_dir}/isar.lock')
+ os.makedirs(self.localpath(ud, d))
+
+ try:
+ runfetchcmd(f'''
+set -e
+{unshare_cmd} /bin/bash -s <<'EOF' | tar -C {self.localpath(ud, d)} -x
+ chroot {rootfsdir} /bin/bash -c '
+ set -e
+ TMPDIR=$(mktemp -d)
+ mkdir -p $TMPDIR/{ud.localfile}
+ cd $TMPDIR/{ud.localfile}
+ apt-get -y -oDebug::NoLocking=1 --download-only --only-source source {ud.src_package} >/dev/null;
+ tar -c --owner=0 --group=0 --numeric-owner .
+ '
+EOF
+ ''', d)
+ except (OSError, FetchError):
+ raise
+ finally:
+ bb.utils.unlockfile(lockfile)
+ self._teardown_chroot(rootfsdir, d)
+
+ def unpack(self, ud, rootdir, d):
+ workdir = d.getVar('WORKDIR')
+ rootfsdir = os.path.join(workdir, 'rootfs-fetcher')
+ extractto = f'{d.getVar("S")}.dpkg'
+ bb.utils.remove(extractto, recurse=True)
+
+ try:
+ runfetchcmd(f'''
+ set -e
+ find {self.localpath(ud, d)} -print -type f -name '*.dsc' -exec dpkg-source -su -x {{}} {extractto} \\;
+ find {extractto} -mindepth 1 -maxdepth 1 -exec mv {{}} {d.getVar('S')}/ \\;
+ ''', d)
+ except (OSError, FetchError):
+ raise
+ finally:
+ bb.utils.remove(extractto, recurse=True)
+ self._teardown_chroot(rootfsdir, d)
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
The fetching of a common source package needs to happen in the
chroot. Previously we only had an implementation for the schroot
backend, but we also need one for unshare, which is added here.
meta/classes-recipe/dpkg-source.bbclass | 38 ++++++++++++++++++++++---
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index 629796d6..a1848473 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -52,10 +52,7 @@ do_dpkg_build[depends] += "${BPN}:do_deploy_source"
SCHROOT_MOUNTS = "${WORKDIR}:/work ${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
-do_fetch_common_source[depends] += "${SCHROOT_DEP} ${BPN}:do_deploy_source"
-do_fetch_common_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
-do_fetch_common_source[network] = "${TASK_USE_SUDO}"
-do_fetch_common_source() {
+fetch_common_source_schroot() {
schroot_create_configs
insert_mounts
@@ -83,6 +80,39 @@ do_fetch_common_source() {
remove_mounts
schroot_delete_configs
}
+
+UNSHARE_DPKG_SOURCE_CHROOT = "${WORKDIR}/dpkg-source-chroot"
+fetch_common_source_unshare() {
+ tar -xf "${SBUILD_CHROOT}" -C ${UNSHARE_DPKG_SOURCE_CHROOT}
+
+ ${@insert_isar_mounts(d, d.getVar('UNSHARE_DPKG_SOURCE_CHROOT'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${UNSHARE_DPKG_SOURCE_CHROOT} /bin/bash -s <<'EOAPT'
+ set -e
+ apt-get update \
+ apt-get -y --download-only --only-source \
+ -o Debug::NoLocking=1 -o Acquire::Source-Symlinks="false" \
+ source ${DEBIAN_SOURCE}
+EOAPT
+EOF
+
+ # run cleanup in separate session to ensure nothing is mounted
+ run_privileged rm -rf ${UNSHARE_DPKG_SOURCE_CHROOT}
+}
+
+do_fetch_common_source[depends] += "${SCHROOT_DEP} ${BPN}:do_deploy_source"
+do_fetch_common_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
+do_fetch_common_source[network] = "${TASK_USE_SUDO}"
+do_fetch_common_source() {
+ fetch_common_source_${ISAR_CHROOT_MODE}
+}
addtask fetch_common_source
do_dpkg_build[depends] += "${@'${PN}:do_dpkg_source' if '${PN}' == '${BPN}' else '${PN}:do_fetch_common_source'}"
--
2.53.0
chroot. Previously we only had an implementation for the schroot
backend, but we also need one for unshare, which is added here.
meta/classes-recipe/dpkg-source.bbclass | 38 ++++++++++++++++++++++---
1 file changed, 34 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index 629796d6..a1848473 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -52,10 +52,7 @@ do_dpkg_build[depends] += "${BPN}:do_deploy_source"
SCHROOT_MOUNTS = "${WORKDIR}:/work ${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
-do_fetch_common_source[depends] += "${SCHROOT_DEP} ${BPN}:do_deploy_source"
-do_fetch_common_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
-do_fetch_common_source[network] = "${TASK_USE_SUDO}"
-do_fetch_common_source() {
+fetch_common_source_schroot() {
schroot_create_configs
insert_mounts
@@ -83,6 +80,39 @@ do_fetch_common_source() {
remove_mounts
schroot_delete_configs
}
+
+UNSHARE_DPKG_SOURCE_CHROOT = "${WORKDIR}/dpkg-source-chroot"
+fetch_common_source_unshare() {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ mkdir -p ${UNSHARE_DPKG_SOURCE_CHROOT}
+ set -e
+ tar -xf "${SBUILD_CHROOT}" -C ${UNSHARE_DPKG_SOURCE_CHROOT}
+
+ ${@insert_isar_mounts(d, d.getVar('UNSHARE_DPKG_SOURCE_CHROOT'), d.getVar('SCHROOT_MOUNTS'))}
+ chroot ${UNSHARE_DPKG_SOURCE_CHROOT} /bin/bash -s <<'EOAPT'
+ set -e
+ apt-get update \
+ -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
+ -o Dir::Etc::SourceParts="-" \
+ -o APT::Get::List-Cleanup="0"
+
+ cd /work
+ -o Dir::Etc::SourceParts="-" \
+ -o APT::Get::List-Cleanup="0"
+
+ apt-get -y --download-only --only-source \
+ -o Debug::NoLocking=1 -o Acquire::Source-Symlinks="false" \
+ source ${DEBIAN_SOURCE}
+EOAPT
+EOF
+
+ # run cleanup in separate session to ensure nothing is mounted
+ run_privileged rm -rf ${UNSHARE_DPKG_SOURCE_CHROOT}
+}
+
+do_fetch_common_source[depends] += "${SCHROOT_DEP} ${BPN}:do_deploy_source"
+do_fetch_common_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
+do_fetch_common_source[network] = "${TASK_USE_SUDO}"
+do_fetch_common_source() {
+ fetch_common_source_${ISAR_CHROOT_MODE}
+}
addtask fetch_common_source
do_dpkg_build[depends] += "${@'${PN}:do_dpkg_source' if '${PN}' == '${BPN}' else '${PN}:do_fetch_common_source'}"
--
2.53.0
Felix Moessbauer
10:23 AM (2 hours ago) 10:23 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
We previously used the same sbom-chroot for generating the sbom of
different root filesystems. This required to have a live copy of the
sbom-chroot in the deploy dir, on which also was operated on. Further,
this copy was left behind in the deploy dir.
We improve this by just storing a minimized tarball of the sbom-chroot
in the deploy dir and extract that into the workdir of the rootfs.
With the new logic in place, we also enable the sbom generation in
unshare mode again.
.../image-tools-extension.bbclass | 27 +++++++++++++++---
meta/classes-recipe/imagetypes_wic.bbclass | 4 ++-
meta/classes/sbom.bbclass | 28 ++++++++++++++++---
.../sbom-chroot/sbom-chroot.bb | 11 +++++++-
4 files changed, 60 insertions(+), 10 deletions(-)
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index cc046fdb..c75025ca 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -82,7 +82,7 @@ EOAPT
dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
${WORKDIR}/imager.manifest
- ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom', '', d)}
+ ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom $schroot_dir', '', d)}
fi
schroot -e -c ${session_id}
@@ -91,14 +91,18 @@ EOAPT
schroot_delete_configs
}
-generate_imager_sbom() {
+generate_imager_sbom_in_chroot() {
+ run_privileged mkdir -p \
+ ${SBOM_CHROOT_LOCAL}/mnt/rootfs \
+ ${SBOM_CHROOT_LOCAL}/mnt/deploy-dir
+
bwrap \
--unshare-user \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
- --bind $schroot_dir /mnt/rootfs \
+ --bind ${SBOM_CHROOT_LOCAL} / \
+ --bind $1 /mnt/rootfs \
--bind ${WORKDIR} /mnt/deploy-dir \
-- debsbom -vv generate ${SBOM_DEBSBOM_TYPE_ARGS} \
--from-pkglist -r /mnt/rootfs -o /mnt/deploy-dir/imager \
@@ -128,6 +132,7 @@ imager_run_unshare() {
fi
local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+ local_bom="${@(d.getVar("BOM_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
run_privileged_heredoc <<'EOF'
set -e
@@ -185,5 +190,19 @@ EOF
+ if [ -n "${local_bom}" ]; then
+ run_in_chroot ${ROOTFS_IMAGETOOLS} \
+ dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
+ ${WORKDIR}/imager.manifest
+
+ ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom {}'.format(d.getVar('ROOTFS_IMAGETOOLS')), '', d)}
+ fi
+
run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
}
+
+generate_imager_sbom() {
+ prepare_sbom_chroot
+ trap 'cleanup_sbom_chroot' EXIT
+ generate_imager_sbom_in_chroot "$1"
+}
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 3e261622..3c65ed0d 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -205,9 +205,11 @@ EOIMAGER
| sort | uniq > "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.manifest"
if ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'true', 'false', d)} ; then
+ prepare_sbom_chroot
for bomtype in ${SBOM_TYPES}; do
merge_wic_sbom $bomtype
done
+ cleanup_sbom_chroot
fi
}
@@ -227,7 +229,7 @@ merge_wic_sbom() {
bwrap \
--unshare-user \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
+ --bind ${SBOM_CHROOT_LOCAL} / \
-- debsbom -v merge -t $BOMTYPE \
--distro-name '${SBOM_DISTRO_NAME}-Image' --distro-supplier '${SBOM_DISTRO_SUPPLIER}' \
--distro-version '${SBOM_DISTRO_VERSION}' --base-distro-vendor '${SBOM_BASE_DISTRO_VENDOR}' \
diff --git a/meta/classes/sbom.bbclass b/meta/classes/sbom.bbclass
index b4fcddaa..2e6d579f 100644
--- a/meta/classes/sbom.bbclass
+++ b/meta/classes/sbom.bbclass
@@ -23,7 +23,8 @@ SBOM_SPDX_NAMESPACE_PREFIX ?= "https://spdx.org/spdxdocs"
DEPLOY_DIR_SBOM = "${DEPLOY_DIR_IMAGE}"
SBOM_DIR = "${DEPLOY_DIR}/sbom"
-SBOM_CHROOT = "${SBOM_DIR}/sbom-chroot"
+SBOM_CHROOT = "${SBOM_DIR}/sbom-chroot.tar.zst"
+SBOM_CHROOT_LOCAL = "${WORKDIR}/sbom-chroot"
# adapted from the isar-cip-core image_uuid.bbclass
def generate_document_uuid(d, warn_not_repr=True):
@@ -40,14 +41,24 @@ def sbom_doc_uuid(d):
if not d.getVar("SBOM_DOCUMENT_UUID"):
d.setVar("SBOM_DOCUMENT_UUID", generate_document_uuid(d))
+prepare_sbom_chroot() {
+ tar -xf ${SBOM_CHROOT} -C ${SBOM_CHROOT_LOCAL}
+EOF
+}
+
generate_sbom() {
- run_privileged mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
+ run_privileged mkdir -p \
+ ${SBOM_CHROOT_LOCAL}/mnt/rootfs \
+ ${SBOM_CHROOT_LOCAL}/mnt/deploy-dir
TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
bwrap \
--unshare-user \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
+ --bind ${SBOM_CHROOT_LOCAL} / \
--bind ${ROOTFSDIR} /mnt/rootfs \
--bind ${DEPLOY_DIR_SBOM} /mnt/deploy-dir \
-- debsbom -v generate ${SBOM_DEBSBOM_TYPE_ARGS} -r /mnt/rootfs -o /mnt/deploy-dir/'${ROOTFS_PACKAGE_SUFFIX}' \
@@ -59,8 +70,17 @@ generate_sbom() {
--timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS}
}
+cleanup_sbom_chroot() {
+ run_privileged rm -rf ${SBOM_CHROOT_LOCAL}
+}
+
do_generate_sbom[dirs] += "${DEPLOY_DIR_SBOM}"
+do_generate_sbom[network] = "${TASK_USE_SUDO}"
python do_generate_sbom() {
sbom_doc_uuid(d)
- bb.build.exec_func("generate_sbom", d)
+ try:
+ bb.build.exec_func("prepare_sbom_chroot", d)
+ bb.build.exec_func("generate_sbom", d)
+ finally:
+ bb.build.exec_func("cleanup_sbom_chroot", d)
}
diff --git a/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb b/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
index 182432a0..f347327b 100644
--- a/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
+++ b/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
@@ -27,7 +27,16 @@ ROOTFSDIR = "${WORKDIR}/rootfs"
ROOTFS_PACKAGES = "${SBOM_IMAGE_INSTALL}"
do_sbomchroot_deploy[dirs] = "${SBOM_DIR}"
+do_sbomchroot_deploy[network] = "${TASK_USE_SUDO}"
do_sbomchroot_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${SBOM_CHROOT}"
+ # deploy with empty var to make it smaller
+ lopts="--one-file-system --exclude=var/*"
}
addtask do_sbomchroot_deploy before do_build after do_rootfs
--
2.53.0
different root filesystems. This required to have a live copy of the
sbom-chroot in the deploy dir, on which also was operated on. Further,
this copy was left behind in the deploy dir.
We improve this by just storing a minimized tarball of the sbom-chroot
in the deploy dir and extract that into the workdir of the rootfs.
With the new logic in place, we also enable the sbom generation in
unshare mode again.
.../image-tools-extension.bbclass | 27 +++++++++++++++---
meta/classes-recipe/imagetypes_wic.bbclass | 4 ++-
meta/classes/sbom.bbclass | 28 ++++++++++++++++---
.../sbom-chroot/sbom-chroot.bb | 11 +++++++-
4 files changed, 60 insertions(+), 10 deletions(-)
diff --git a/meta/classes-recipe/image-tools-extension.bbclass b/meta/classes-recipe/image-tools-extension.bbclass
index cc046fdb..c75025ca 100644
--- a/meta/classes-recipe/image-tools-extension.bbclass
+++ b/meta/classes-recipe/image-tools-extension.bbclass
@@ -82,7 +82,7 @@ EOAPT
dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
${WORKDIR}/imager.manifest
- ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom', '', d)}
+ ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom $schroot_dir', '', d)}
fi
schroot -e -c ${session_id}
schroot_delete_configs
}
-generate_imager_sbom() {
+generate_imager_sbom_in_chroot() {
+ run_privileged mkdir -p \
+ ${SBOM_CHROOT_LOCAL}/mnt/rootfs \
+ ${SBOM_CHROOT_LOCAL}/mnt/deploy-dir
+
TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
sbom_document_uuid="${@d.getVar('SBOM_DOCUMENT_UUID') or generate_document_uuid(d, False)}"
bwrap \
--unshare-user \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
- --bind $schroot_dir /mnt/rootfs \
+ --bind ${SBOM_CHROOT_LOCAL} / \
+ --bind $1 /mnt/rootfs \
--bind ${WORKDIR} /mnt/deploy-dir \
-- debsbom -vv generate ${SBOM_DEBSBOM_TYPE_ARGS} \
--from-pkglist -r /mnt/rootfs -o /mnt/deploy-dir/imager \
@@ -128,6 +132,7 @@ imager_run_unshare() {
fi
local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
run_privileged_heredoc <<'EOF'
set -e
@@ -185,5 +190,19 @@ EOF
chroot ${ROOTFS_IMAGETOOLS} "$@" <&3
EOF
+ if [ -n "${local_bom}" ]; then
+ run_in_chroot ${ROOTFS_IMAGETOOLS} \
+ dpkg-query -W -f='${source:Package}|${source:Version}|${Package}:${Architecture}|${Version}\n' ${local_bom} > \
+ ${WORKDIR}/imager.manifest
+
+ ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'generate_imager_sbom {}'.format(d.getVar('ROOTFS_IMAGETOOLS')), '', d)}
+ fi
+
run_privileged rm -rf ${ROOTFS_IMAGETOOLS}
}
+
+generate_imager_sbom() {
+ prepare_sbom_chroot
+ trap 'cleanup_sbom_chroot' EXIT
+ generate_imager_sbom_in_chroot "$1"
+}
diff --git a/meta/classes-recipe/imagetypes_wic.bbclass b/meta/classes-recipe/imagetypes_wic.bbclass
index 3e261622..3c65ed0d 100644
--- a/meta/classes-recipe/imagetypes_wic.bbclass
+++ b/meta/classes-recipe/imagetypes_wic.bbclass
@@ -205,9 +205,11 @@ EOIMAGER
| sort | uniq > "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.manifest"
if ${@bb.utils.contains('ROOTFS_FEATURES', 'generate-sbom', 'true', 'false', d)} ; then
+ prepare_sbom_chroot
for bomtype in ${SBOM_TYPES}; do
merge_wic_sbom $bomtype
done
+ cleanup_sbom_chroot
fi
}
@@ -227,7 +229,7 @@ merge_wic_sbom() {
bwrap \
--unshare-user \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
+ --bind ${SBOM_CHROOT_LOCAL} / \
-- debsbom -v merge -t $BOMTYPE \
--distro-name '${SBOM_DISTRO_NAME}-Image' --distro-supplier '${SBOM_DISTRO_SUPPLIER}' \
--distro-version '${SBOM_DISTRO_VERSION}' --base-distro-vendor '${SBOM_BASE_DISTRO_VENDOR}' \
diff --git a/meta/classes/sbom.bbclass b/meta/classes/sbom.bbclass
index b4fcddaa..2e6d579f 100644
--- a/meta/classes/sbom.bbclass
+++ b/meta/classes/sbom.bbclass
@@ -23,7 +23,8 @@ SBOM_SPDX_NAMESPACE_PREFIX ?= "https://spdx.org/spdxdocs"
DEPLOY_DIR_SBOM = "${DEPLOY_DIR_IMAGE}"
SBOM_DIR = "${DEPLOY_DIR}/sbom"
-SBOM_CHROOT = "${SBOM_DIR}/sbom-chroot"
+SBOM_CHROOT = "${SBOM_DIR}/sbom-chroot.tar.zst"
+SBOM_CHROOT_LOCAL = "${WORKDIR}/sbom-chroot"
# adapted from the isar-cip-core image_uuid.bbclass
def generate_document_uuid(d, warn_not_repr=True):
@@ -40,14 +41,24 @@ def sbom_doc_uuid(d):
if not d.getVar("SBOM_DOCUMENT_UUID"):
d.setVar("SBOM_DOCUMENT_UUID", generate_document_uuid(d))
+prepare_sbom_chroot() {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ mkdir -p ${SBOM_CHROOT_LOCAL}
+ set -e
+ tar -xf ${SBOM_CHROOT} -C ${SBOM_CHROOT_LOCAL}
+EOF
+}
+
generate_sbom() {
- run_privileged mkdir -p ${SBOM_CHROOT}/mnt/rootfs ${SBOM_CHROOT}/mnt/deploy-dir
+ run_privileged mkdir -p \
+ ${SBOM_CHROOT_LOCAL}/mnt/rootfs \
+ ${SBOM_CHROOT_LOCAL}/mnt/deploy-dir
TIMESTAMP=$(date --iso-8601=s -d @${SOURCE_DATE_EPOCH})
bwrap \
--unshare-pid \
- --bind ${SBOM_CHROOT} / \
+ --bind ${SBOM_CHROOT_LOCAL} / \
--bind ${ROOTFSDIR} /mnt/rootfs \
--bind ${DEPLOY_DIR_SBOM} /mnt/deploy-dir \
-- debsbom -v generate ${SBOM_DEBSBOM_TYPE_ARGS} -r /mnt/rootfs -o /mnt/deploy-dir/'${ROOTFS_PACKAGE_SUFFIX}' \
@@ -59,8 +70,17 @@ generate_sbom() {
--timestamp $TIMESTAMP ${SBOM_DEBSBOM_EXTRA_ARGS}
}
+cleanup_sbom_chroot() {
+ run_privileged rm -rf ${SBOM_CHROOT_LOCAL}
+}
+
do_generate_sbom[dirs] += "${DEPLOY_DIR_SBOM}"
+do_generate_sbom[network] = "${TASK_USE_SUDO}"
python do_generate_sbom() {
sbom_doc_uuid(d)
- bb.build.exec_func("generate_sbom", d)
+ try:
+ bb.build.exec_func("prepare_sbom_chroot", d)
+ bb.build.exec_func("generate_sbom", d)
+ finally:
+ bb.build.exec_func("cleanup_sbom_chroot", d)
}
diff --git a/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb b/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
index 182432a0..f347327b 100644
--- a/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
+++ b/meta/recipes-devtools/sbom-chroot/sbom-chroot.bb
@@ -27,7 +27,16 @@ ROOTFSDIR = "${WORKDIR}/rootfs"
ROOTFS_PACKAGES = "${SBOM_IMAGE_INSTALL}"
do_sbomchroot_deploy[dirs] = "${SBOM_DIR}"
+do_sbomchroot_deploy[network] = "${TASK_USE_SUDO}"
do_sbomchroot_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${SBOM_CHROOT}"
+ # deploy with empty var to make it smaller
+ lopts="--one-file-system --exclude=var/*"
+ ZSTD="zstd -${SSTATE_ZSTD_CLEVEL} -T${ZSTD_THREADS}"
+
+ run_privileged \
+ tar -C ${ROOTFSDIR} -cpS $lopts ${ROOTFS_TAR_ATTR_FLAGS} . \
+ | $ZSTD > ${SBOM_CHROOT}
+
+ run_privileged \
+ tar -C ${ROOTFSDIR} -cpS $lopts ${ROOTFS_TAR_ATTR_FLAGS} . \
+ # cleanup extracted rootfs
+ run_privileged rm -rf ${ROOTFSDIR}
}
addtask do_sbomchroot_deploy before do_build after do_rootfs
--
2.53.0
Felix Moessbauer
10:24 AM (2 hours ago) 10:24 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
meta/classes-recipe/dpkg-base.bbclass | 68 ++++++++++++++++++++++-----
1 file changed, 56 insertions(+), 12 deletions(-)
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index a0d4fd05..b3583373 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -253,13 +253,11 @@ do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
do_deploy_deb[dirs] = "${S}"
python do_devshell() {
isar_export_proxies(d)
isar_export_ccache(d)
isar_export_build_settings(d)
- if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
- bb.build.exec_func('schroot_configure_ccache', d)
+
+ bb.build.exec_func('devshell_chroot_prepare', d)
schroot = d.getVar('SBUILD_CHROOT')
pkg_arch = d.getVar('PACKAGE_ARCH')
@@ -271,21 +269,39 @@ python do_devshell() {
-t \"apt-get -y -q -o Debug::pkgProblemResolver=yes --no-install-recommends --allow-downgrades\" \
debian/control"
- termcmd = "schroot -d / -c {0} -u root -- sh -c ' \
- cd {1}; \
+ termcmd = "cd {0}; \
apt-get -y -q update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"; \
apt-get -y upgrade; \
- {2}; \
+ {1}; \
if [ -n \"$PATH_PREPEND\" ]; then export PATH=$PATH_PREPEND:$PATH; fi; \
- $SHELL -i \
- '"
- oe_terminal(termcmd.format(schroot, pp_pps, install_deps), "Isar devshell", d)
-
- bb.build.exec_func('schroot_delete_configs', d)
+ $SHELL -i".format(pp_pps, install_deps)
+
+ mounts += ' {}:/home/builder/{}'.format(d.getVar('WORKDIR'), d.getVar('BPN'))
+
+ if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
+ bb.build.exec_func('dpkg_prepare_unshare_ccache', d)
+ mounts += ' {}:/ccache'.format(d.getVar('CCACHE_DIR'))
+
+ termcmd = """{0} \
+sh -c "{1};cp /etc/resolv.conf {2}/etc;chroot {2} sh -c '{3}'"
+""".format(
+ run_privileged_cmd(d),
+ insert_isar_mounts(d, d.getVar('DEVSHELL_UNSHARE_ROOTFS'), mounts),
+ d.getVar('DEVSHELL_UNSHARE_ROOTFS'),
+ termcmd.replace('"', "\\\""))
+ else:
+ termcmd = "schroot -d / -c {0} -u root -- sh -c '{1}'".format(schroot, termcmd)
+ bb.warn(termcmd)
+ oe_terminal(termcmd, "Isar devshell", d)
+
+ bb.build.exec_func('devshell_chroot_finalize', d)
}
addtask devshell after do_local_isarapt do_prepare_build
DEVSHELL_STARTDIR ?= "${S}"
+DEVSHELL_UNSHARE_ROOTFS ?= "${WORKDIR}/rootfs-devshell"
do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell[nostamp] = "1"
do_devshell[network] = "${TASK_USE_SUDO}"
@@ -299,3 +315,31 @@ addtask devshell_nodeps after do_local_isarapt do_prepare_build
do_devshell_nodeps[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell_nodeps[nostamp] = "1"
do_devshell_nodeps[network] = "${TASK_USE_SUDO}"
+
+devshell_prepare_unshare_chroot() {
+ tar -xf ${SBUILD_CHROOT} -C ${DEVSHELL_UNSHARE_ROOTFS}
+EOF
+}
+
+devshell_cleanup_unshare_chroot() {
+ run_privileged rm -rf ${DEVSHELL_UNSHARE_ROOTFS}
+}
+
+python devshell_chroot_prepare() {
+ else:
+ bb.build.exec_func('dpkg_schroot_create_configs', d)
+ if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
+ bb.build.exec_func('schroot_configure_ccache', d)
+}
+
+python devshell_chroot_finalize() {
+ else:
+ bb.build.exec_func('schroot_delete_configs', d)
+}
--
2.53.0
1 file changed, 56 insertions(+), 12 deletions(-)
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index a0d4fd05..b3583373 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -253,13 +253,11 @@ do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
do_deploy_deb[dirs] = "${S}"
python do_devshell() {
- bb.build.exec_func('dpkg_schroot_create_configs', d)
-
isar_export_proxies(d)
isar_export_ccache(d)
isar_export_build_settings(d)
- if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
- bb.build.exec_func('schroot_configure_ccache', d)
+
+ bb.build.exec_func('devshell_chroot_prepare', d)
schroot = d.getVar('SBUILD_CHROOT')
pkg_arch = d.getVar('PACKAGE_ARCH')
@@ -271,21 +269,39 @@ python do_devshell() {
-t \"apt-get -y -q -o Debug::pkgProblemResolver=yes --no-install-recommends --allow-downgrades\" \
debian/control"
- termcmd = "schroot -d / -c {0} -u root -- sh -c ' \
- cd {1}; \
+ termcmd = "cd {0}; \
apt-get -y -q update -o Dir::Etc::SourceList=\"sources.list.d/isar-apt.list\" -o Dir::Etc::SourceParts=\"-\" -o APT::Get::List-Cleanup=\"0\"; \
apt-get -y upgrade; \
- {2}; \
+ {1}; \
if [ -n \"$PATH_PREPEND\" ]; then export PATH=$PATH_PREPEND:$PATH; fi; \
- $SHELL -i \
- '"
- oe_terminal(termcmd.format(schroot, pp_pps, install_deps), "Isar devshell", d)
-
- bb.build.exec_func('schroot_delete_configs', d)
+ $SHELL -i".format(pp_pps, install_deps)
+
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ mounts = d.getVar('SCHROOT_MOUNTS')
+ mounts += ' {}:/home/builder/{}'.format(d.getVar('WORKDIR'), d.getVar('BPN'))
+
+ if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
+ bb.build.exec_func('dpkg_prepare_unshare_ccache', d)
+ mounts += ' {}:/ccache'.format(d.getVar('CCACHE_DIR'))
+
+ termcmd = """{0} \
+sh -c "{1};cp /etc/resolv.conf {2}/etc;chroot {2} sh -c '{3}'"
+""".format(
+ run_privileged_cmd(d),
+ insert_isar_mounts(d, d.getVar('DEVSHELL_UNSHARE_ROOTFS'), mounts),
+ d.getVar('DEVSHELL_UNSHARE_ROOTFS'),
+ termcmd.replace('"', "\\\""))
+ else:
+ termcmd = "schroot -d / -c {0} -u root -- sh -c '{1}'".format(schroot, termcmd)
+ bb.warn(termcmd)
+ oe_terminal(termcmd, "Isar devshell", d)
+
+ bb.build.exec_func('devshell_chroot_finalize', d)
}
addtask devshell after do_local_isarapt do_prepare_build
DEVSHELL_STARTDIR ?= "${S}"
+DEVSHELL_UNSHARE_ROOTFS ?= "${WORKDIR}/rootfs-devshell"
do_devshell[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell[nostamp] = "1"
do_devshell[network] = "${TASK_USE_SUDO}"
@@ -299,3 +315,31 @@ addtask devshell_nodeps after do_local_isarapt do_prepare_build
do_devshell_nodeps[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell_nodeps[nostamp] = "1"
do_devshell_nodeps[network] = "${TASK_USE_SUDO}"
+
+devshell_prepare_unshare_chroot() {
+ run_privileged_heredoc <<'EOF'
+ set -e
+ mkdir -p ${DEVSHELL_UNSHARE_ROOTFS}
+ set -e
+ tar -xf ${SBUILD_CHROOT} -C ${DEVSHELL_UNSHARE_ROOTFS}
+EOF
+}
+
+devshell_cleanup_unshare_chroot() {
+ run_privileged rm -rf ${DEVSHELL_UNSHARE_ROOTFS}
+}
+
+python devshell_chroot_prepare() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ bb.build.exec_func('devshell_prepare_unshare_chroot', d)
+ else:
+ bb.build.exec_func('dpkg_schroot_create_configs', d)
+ if bb.utils.to_boolean(d.getVar('USE_CCACHE')):
+ bb.build.exec_func('schroot_configure_ccache', d)
+}
+
+python devshell_chroot_finalize() {
+ if d.getVar('ISAR_CHROOT_MODE') == 'unshare':
+ bb.build.exec_func('devshell_cleanup_unshare_chroot', d)
+ else:
+ bb.build.exec_func('schroot_delete_configs', d)
+}
--
2.53.0
Felix Moessbauer
10:24 AM (2 hours ago) 10:24 AM
to isar-...@googlegroups.com, jan.k...@siemens.com, quirin.g...@siemens.com, Felix Moessbauer
While the build mode (schroot or unshare) should be transparent for the
user, we need to test both cases. For that, we add a parameter to the
testsuite to select which mode to test. Later on, more fine-grained
control over which tests run in which mode can be introduced. For now it
is important to get an overview at which things break and where code or
tests need to be adjusted.
testsuite/cibuilder.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index f9cca0c7..bdfcbeed 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -128,6 +128,7 @@ class CIBuilder(Test):
customizations=None,
generate_sbom=False,
lines=None,
+ rootless=False,
**kwargs,
):
# write configuration file and set bitbake_args
@@ -140,6 +141,9 @@ class CIBuilder(Test):
if not sstate:
sstate = bool(int(self.params.get('sstate', default=0)))
+ if not rootless:
+ rootless = bool(int(self.params.get('rootless', default=0)))
+
# set those to "" to not set dir value but use system default
if dl_dir is None:
dl_dir = os.getenv('DL_DIR')
@@ -279,6 +283,8 @@ class CIBuilder(Test):
)
if generate_sbom is False:
f.write('ROOTFS_FEATURES:remove = "generate-sbom"\n')
+ if rootless:
+ f.write('ISAR_ROOTLESS = "1"')
if lines is not None:
f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines)
--
2.53.0
user, we need to test both cases. For that, we add a parameter to the
testsuite to select which mode to test. Later on, more fine-grained
control over which tests run in which mode can be introduced. For now it
is important to get an overview at which things break and where code or
tests need to be adjusted.
testsuite/cibuilder.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/testsuite/cibuilder.py b/testsuite/cibuilder.py
index f9cca0c7..bdfcbeed 100755
--- a/testsuite/cibuilder.py
+++ b/testsuite/cibuilder.py
@@ -128,6 +128,7 @@ class CIBuilder(Test):
customizations=None,
generate_sbom=False,
lines=None,
+ rootless=False,
**kwargs,
):
# write configuration file and set bitbake_args
@@ -140,6 +141,9 @@ class CIBuilder(Test):
if not sstate:
sstate = bool(int(self.params.get('sstate', default=0)))
+ if not rootless:
+ rootless = bool(int(self.params.get('rootless', default=0)))
+
# set those to "" to not set dir value but use system default
if dl_dir is None:
dl_dir = os.getenv('DL_DIR')
@@ -279,6 +283,8 @@ class CIBuilder(Test):
)
if generate_sbom is False:
f.write('ROOTFS_FEATURES:remove = "generate-sbom"\n')
+ if rootless:
+ f.write('ISAR_ROOTLESS = "1"')
if lines is not None:
f.writelines((line + '\n' if not line.endswith('\n') else line) for line in lines)
--
2.53.0
Reply all
Reply to author
Forward
0 new messages