[PATCH] image: copy .list files from SRC_URI to the image

[Cedric Hombourger]

There are use-cases where the .list files used by Isar / apt during the
build would not be the same as the initial .list files presented to the
end-user of the image (e.g. build from base-apt or from a local mirror).
If any .list files are listed in the SRC_URI of an image recipe, they
will be copied to /etc/apt/sources.list.d/ verbatim and in lieu of the
bootstrap sources that were used by Isar.

Signed-off-by: Cedric Hombourger <cedric.h...@siemens.com>
---
doc/user_manual.md | 14 ++++++++++++++
meta/classes/image.bbclass | 18 ++++++++++++++++++
2 files changed, 32 insertions(+)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 847f4b07..89c3a293 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -769,6 +769,20 @@ inherit image

```

+If the resulting image should not ship apt sources used during the build but custom ones (e.g. for end-users to point
+to an external or simply different server when they "apt-get update", custom list files may be listed in `SRC_URI`:
+Isar will copy them to `/etc/apt/sources.list.d/` and omit bootstrap sources. Possible use-cases:
+
+ * image built from base-apt (which is by definition local to the build host)
+
+ * image built from an internal mirror, not reachable by devices running the produced image
+
+ * ship template list files for the end-user to edit (e.g. letting him uncomment `deb` or `deb-src` entries)
+
+It should be noted that Isar will not validate or even load supplied list files: they are simply copied verbatim to
+the root file-system just before creating an image out of it (loading sources from the network would make the build
+non-reproducible).
+
### Additional Notes

The distribution selected via the `DISTRO` variable may need to run a post-configuration script after the root file-system
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index c29d9e26..0c162aa3 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -392,6 +392,18 @@ python do_deploy() {
}
addtask deploy before do_build after do_image

+def apt_list_files(d):
+ lists = []
+ sources = d.getVar("SRC_URI").split()
+ for s in sources:
+ _, _, local, _, _, parm = bb.fetch.decodeurl(s)
+ base, ext = os.path.splitext(os.path.basename(local))
+ if ext and ext in (".list"):
+ lists.append(local)
+ return lists
+
+IMAGE_LISTS = "${@ ' '.join(apt_list_files(d)) }"
+
do_rootfs_finalize() {
sudo -s <<'EOSUDO'
set -e
@@ -429,6 +441,12 @@ do_rootfs_finalize() {
mv "${ROOTFSDIR}/etc/apt/sources-list" \
"${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
fi
+ if [ -n "${IMAGE_LISTS}" ]; then
+ rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
+ for l in ${IMAGE_LISTS}; do
+ cp "${WORKDIR}"/${l} "${ROOTFSDIR}/etc/apt/sources.list.d/"
+ done
+ fi

rm -f "${ROOTFSDIR}/run/blkid/blkid.tab"
rm -f "${ROOTFSDIR}/run/blkid/blkid.tab.old"
--
2.39.2

[Jan Kiszka]

On 23.09.24 16:27, 'Cedric Hombourger' via isar-users wrote:
> There are use-cases where the .list files used by Isar / apt during the
> build would not be the same as the initial .list files presented to the
> end-user of the image (e.g. build from base-apt or from a local mirror).
> If any .list files are listed in the SRC_URI of an image recipe, they
> will be copied to /etc/apt/sources.list.d/ verbatim and in lieu of the
> bootstrap sources that were used by Isar.

How well does this interact with the list swappings we do already, e.g.
for using snapshots during build but then upstream on the target?

Jan

Siemens AG, Technology
Linux Expert Center

[cedric.h...@siemens.com]

On Wed, 2024-09-25 at 18:04 +0200, Jan Kiszka wrote:
> On 23.09.24 16:27, 'Cedric Hombourger' via isar-users wrote:
> > There are use-cases where the .list files used by Isar / apt during
> > the
> > build would not be the same as the initial .list files presented to
> > the
> > end-user of the image (e.g. build from base-apt or from a local
> > mirror).
> > If any .list files are listed in the SRC_URI of an image recipe,
> > they
> > will be copied to /etc/apt/sources.list.d/ verbatim and in lieu of
> > the
> > bootstrap sources that were used by Isar.
>
> How well does this interact with the list swappings we do already,
> e.g.
> for using snapshots during build but then upstream on the target?

Rechecked to be sure. Lines above my changes are responsible for the
swap (/etc/apt/sources-list => /etc/apt/sources.list.d/bootstrap.list)

head of /etc/apt/sources.list.d/bootstrap.list before
do_rootfs_finalize()

deb [check-valid-until=no]
http://snapshot.debian.org/archive/debian/20240925T180156Z/ bookworm
main contrib non-free-firmware non-free

head of /etc/apt/sources-list before do_rootfs_finalize()

deb http://deb.debian.org/debian bookworm main contrib non-free-
firmware non-free

After do_rootfs_finalize(), sources-list is gone and bootstrap.list is
no lomger pointing to the selected snapshot

With my changes and with user-supplied .list files in SRC_URI
bootstrap.list is removed and only user-supplied .list files are
present in /etc/apt/sources.list.d/

Hope this clarifies
Cedric

Cedric Hombourger
Siemens AG
www.siemens.com

[Uladzimir Bely]

On Mon, 2024-09-23 at 16:27 +0200, 'Cedric Hombourger' via isar-users
wrote:

Hello all.

If there are no objection and futher discussion, we are ready to merge
the patch.

--
Best regards,
Uladzimir.

[Uladzimir Bely]

On Mon, 2024-09-23 at 16:27 +0200, 'Cedric Hombourger' via isar-users
wrote:

Applied to next, thanks.

--
Best regards,
Uladzimir.