[PATCH 0/3] Rootfs install race fix for isar-apt packages

[Anton Mikanovich]

After merging isar apt for different architecures in 50d641e it unlocks
a race during isar-apt packages installation into rootfs. If some
package performs do_deploy_deb task during do_rootfs_install execution
it can remove its deb in the moment when it read by apt-get.

This happens because our existing logic for precaching debs under
isar-apt lock was not working for isar-apt packages, because apt-get
just do not cache packages from local repos in /var/cache/apt/archives.

To fix this, introduce a new rootfs install command which obtains all
the packages not been cached and download them manually.

Also add "--no-download" option to the final apt-get install call to
prevent masking such an issues in the future.

Anton Mikanovich (3):
rootfs: Allow locking on single install command
rootfs: Download isar-apt packages under isar-apt lock
rootfs: Deny packages download during install

meta/classes-recipe/rootfs.bbclass | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)

--
2.34.1

[Anton Mikanovich]

It turns out apt-get install do not cache debs from local repos
(started with file:/) in /var/cache/apt/archives even with
--download-only option enabled. It results in isar-apt packages
download execution during rootfs_install_pkgs_install command, which is
not covered under isar-apt lock.
To put isar-apt packages in local cache under isar-apt lock new command
was introduced. It obtain all the packages not been cached and download
them manually.

This change fixes "file not found" issues during rootfs installation.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---
meta/classes-recipe/rootfs.bbclass | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index dc4e702c..81384340 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -363,6 +363,29 @@ rootfs_export_package_cache() {
deb_dl_dir_export ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
}

+ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_isar_download"
+rootfs_install_pkgs_isar_download[weight] = "50"
+rootfs_install_pkgs_isar_download[isar-apt-lock] = "acquire-before release-after"
+rootfs_install_pkgs_isar_download() {
+ mkdir -p "${WORKDIR}/dpkg"
+
+ # Use our own dpkg lock files rather than those in the rootfs since we are not root
+ # (this is safe as there are no concurrent apt/dpkg operations for that rootfs)
+ touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend"
+
+ # Command apt-get install do not cache packages from local repos
+ # We can obtain non cached package URIs by recalling install command here
+ # No need in export those files to dl_dir, so we can run it right after
+ rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
+ --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \
+ --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \
+ --chdir "/var/cache/apt/archives" \
+ ${ROOTFSDIR} \
+ -- /usr/bin/sh -c "apt-get ${ROOTFS_APT_ARGS} --print-uris ${ROOTFS_PACKAGES} | \
+ sed -n \"s|^.*/\\(.*\\)_[^_]*_[^_]*\\.deb'.*|\\1|p\" | \
+ xargs -r apt-get download"
+}
+
ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
rootfs_install_clean_files[weight] = "2"
rootfs_install_clean_files() {
--
2.34.1

[Anton Mikanovich]

In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
can require isar-apt locking. To allow this logic improve flag checking
to accept the list of codenames. Existing API and codenames
("acquire-before" and "release-after") were not changed.

So the usage of single task declaration now looks like:

rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

meta/classes-recipe/rootfs.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 8b502a50..dc4e702c 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -427,13 +427,13 @@ python do_rootfs_install() {
for cmd in cmds:
progress_reporter.next_stage()

- if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "acquire-before":
+ if "acquire-before" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
shared=True)

bb.build.exec_func(cmd, d)

- if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "release-after":
+ if "release-after" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
bb.utils.unlockfile(lock)
progress_reporter.finish()
finally:
--
2.34.1

[Anton Mikanovich]

During the command rootfs_install_pkgs_install no actuall packages
downloading should be done. All the debs must be precached and already
placed in /var/cache/apt/archives, so apt-get can reuse them.

Currently any downloads in this command are really succeed because we
still have "network" flag on for using sudo, which masks all the
caching issues in previous rootfs install commands.

To prevent such a cases add "--no-download" option to apt-get.

Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
---

meta/classes-recipe/rootfs.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 81384340..f4580421 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -402,7 +402,7 @@ rootfs_install_pkgs_install[progress] = "custom:rootfs_progress.PkgsInstallProgr
rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
rootfs_install_pkgs_install() {
sudo -E chroot "${ROOTFSDIR}" \
- /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
+ /usr/bin/apt-get ${ROOTFS_APT_ARGS} --no-download ${ROOTFS_PACKAGES}
}

ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
--
2.34.1

[MOESSBAUER, Felix]

On Wed, 2026-06-10 at 09:48 +0300, Anton Mikanovich wrote:
> In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
> can require isar-apt locking. To allow this logic improve flag checking
> to accept the list of codenames. Existing API and codenames
> ("acquire-before" and "release-after") were not changed.
>
> So the usage of single task declaration now looks like:
>
> rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"

Hi, does it even make sense to do an acquire release in two different
commands? To me this looks like a source for deadlocks.

Felix

>
> Signed-off-by: Anton Mikanovich <ami...@ilbers.de>
> ---
> meta/classes-recipe/rootfs.bbclass | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
> index 8b502a50..dc4e702c 100644
> --- a/meta/classes-recipe/rootfs.bbclass
> +++ b/meta/classes-recipe/rootfs.bbclass
> @@ -427,13 +427,13 @@ python do_rootfs_install() {
> for cmd in cmds:
> progress_reporter.next_stage()
>
> - if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "acquire-before":
> + if "acquire-before" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
> lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
> shared=True)
>
> bb.build.exec_func(cmd, d)
>
> - if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "release-after":
> + if "release-after" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
> bb.utils.unlockfile(lock)
> progress_reporter.finish()
> finally:
> --
> 2.34.1
>

> --
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-2-amikan%40ilbers.de.

[Anton Mikanovich]

10.06.2026 11:23, MOESSBAUER, Felix wrote:
> On Wed, 2026-06-10 at 09:48 +0300, Anton Mikanovich wrote:
>> In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
>> can require isar-apt locking. To allow this logic improve flag checking
>> to accept the list of codenames. Existing API and codenames
>> ("acquire-before" and "release-after") were not changed.
>>
>> So the usage of single task declaration now looks like:
>>
>> rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"
> Hi, does it even make sense to do an acquire release in two different
> commands? To me this looks like a source for deadlocks.
>
> Felix

Current implementation have 4 tasks combined in one lock section.
This is done to be sure isar-apt will not changed between "apt-get update"
and "apt-get install" commands. This probably can be reordered and
optimized,
but it will be out of scope for the current patchset.

[MOESSBAUER, Felix]

Makes sense. Then the patchset is fine.

Reviewed-by: Felix Moessbauer <felix.mo...@siemens.com>

[Anton Mikanovich]

Applied to next.