Bug#1062641: live-build Removes User Packages Installed via Hooks

[Arszilla]

Package: live-build
Version: 1:20230502+kali3

When users install `.deb` packages that are not available in Debian via a `.chroot` hook (such as `1password`), the `./live/filesystem.packages-remove` file in the generated ISO uninstalls the packages installed via `.chroot` after the system is installed.

This was not the case until this action (which was added 12 years ago) became active a year ago:
- https://salsa.debian.org/installer-team/live-installer/-/commit/ad0ebaad
- https://salsa.debian.org/installer-team/live-installer/-/commit/ca1e1706757ecc9a4cf1fa5c637d5a9b513acee6

Because certain packages cannot be installed without `.chroot`
hooks, I recommend reverting this change. It was discussed that users should drop their `.deb` packages to the `packages.chroot` directory instead, as that is the intended way. However, certain programs such as `1Password`, `docker` (from Docker's repositories), ProtonVPN, etc. only use the `.deb` packages to add their repos to the system and not install packages, which require users to `sudo apt update && sudo apt install -y <package(s)>`.

This was detected during a Kali Linux live-build:
- https://gitlab.com/kalilinux/build-scripts/live-build-config/-/issues/61
- https://gitlab.com/kalilinux/build-scripts/live-build-config/-/merge_requests/33

[Arszilla]

Hi Roland,

First off, I'd like to let you know that your first email appeared (in my inbox). despite it not appearing on the website. So be rest assured :)

> I still think that removing all live-related packages in the installer is a good idea. The processing of 'live/filesystem.packages-remove' shows where the package management system has been circumvented.

I get that this is totally up to you. However, if people use `live-build` in the same manner as I do, they may face this issue and may be dissatisfied. I came up with a "tempfix" on my end by implementing a `.binary` hook that removes `filesystem.packages-remove` if it detects its presence on the ISO since the packages I install end up there.

I should mention that this issue is not 1Password-specific. We initially discovered the presence of this behavior in https://gitlab.com/kalilinux/build-scripts/live-build-config/-/issues/61 - where another user reported that their custom packages were being removed.

I think when I tried installing 1Password with the commands listed in 1Password's article, I had a similar result. I may have to check again. However, as per my previous statement, this issue affects more packages other than 1Password.

In my testing, I have noticed that this issue affects the following packages/programs I installed in my custom ISO:
- Docker (installed from Docker's own repositories)
- Tenable Nessus
- Insomnia (https://insomnia.rest)
- Spotify
- ProtonVPN
- Obsidian (https://obsidian.md)
- Visual Studio Code
- Discord

These are just some of the packages I can remember off of my head. A small excerpt from the `syslog` found on `/var/log/installer/` directory lists these in more detail:

> Jan 16 16:29:36 in-target: The following packages will be REMOVED:
> Jan 16 16:29:36 in-target: 1password* code* containerd.io* discord* docker-buildx-plugin* docker-ce*
> Jan 16 16:29:36 in-target: docker-ce-cli* docker-ce-rootless-extras* docker-compose-plugin*
> Jan 16 16:29:36 in-target: gir1.2-nm-1.0* gnupg2* insomnia* libcairo-script-interpreter2* libgtk-4-1*
> Jan 16 16:29:36 in-target: libgtk-4-bin* libgtk-4-common* libgtk-4-media-gstreamer* libnma-gtk4-0*
> Jan 16 16:29:36 in-target: libslirp0* libvulkan1* mesa-vulkan-drivers* multiviewer-for-f1* nessus*
> Jan 16 16:29:36 in-target: network-manager-openvpn* network-manager-openvpn-gnome* pigz*
> Jan 16 16:29:36 in-target: proton-vpn-gnome-desktop* proton-vpn-gtk-app* protonvpn-stable-release*
> Jan 16 16:29:36 in-target: python3-jaraco.classes* python3-jeepney* python3-keyring*
> Jan 16 16:29:36 in-target: python3-proton-core* python3-proton-keyring-linux*
> Jan 16 16:29:36 in-target: python3-proton-keyring-linux-secretservice* python3-proton-vpn-api-core*
> Jan 16 16:29:36 in-target: python3-proton-vpn-connection* python3-proton-vpn-killswitch*
> Jan 16 16:29:36 in-target: python3-proton-vpn-killswitch-network-manager* python3-proton-vpn-logger*
> Jan 16 16:29:36 in-target: python3-proton-vpn-network-manager*
> Jan 16 16:29:36 in-target: python3-proton-vpn-network-manager-openvpn* python3-proton-vpn-session*
> Jan 16 16:29:36 in-target: python3-secretstorage* python3-shtab* slirp4netns* spotify-client*
> Jan 16 16:29:37 in-target: 0 upgraded, 0 newly installed, 47 to remove and 0 not upgraded.

One of the packages I installed but not affected by this is Tailscale, which is installed by the following script based on Tailscale's own install script:

> TRACK="stable"
> OS="debian"
> VERSION="bullseye"
> mkdir -p --mode=0755 /usr/share/keyrings
> curl -fsSL "https://pkgs.tailscale.com/$TRACK/$OS/$VERSION.noarmor.gpg" | tee /usr/share/keyrings/tailscale-archive-keyring.gpg >/dev/null
> curl -fsSL "https://pkgs.tailscale.com/$TRACK/$OS/$VERSION.tailscale-keyring.list" | tee /etc/apt/sources.list.d/tailscale.list
> apt-get update
> apt-get install -y tailscale tailscale-archive-keyring
> systemctl enable tailscaled

Regardless, this issue affects various popular programs. Hence, I am raising this issue to see if there could be a better way of addressing the effect desired by this change. Because IMO the last thing anyone using live-build to "cook" a custom ISO with their desired changes wants to see is the programs they desire to be removed "unknowingly" once they install their ISO and having to manually fix this issue, defeating the whole purpose of them using `live-build`.

---

> The bug report was based on a Kali version of live-build, so I assume you know better than me how to do so.

I don't know if there are "major" differences between the live-build version of Debian and Kali, but according to https://pkg.kali.org/pkg/live-build / https://gitlab.com/kalilinux/packages/live-build/-/blob/kali/master/debian/changelog?ref_type=heads there are only minor adjustments to the version in Debian and Kali aimed at addressing some firmware or GRUB related issues/differences.

> Please add such command to the bug report, so I can update the live-manual to address such use case.

I don't really understand what you meant with this statement. If you could elaborate a bit further, I'd sincerely appreciate it.

Kind regards,
Arszilla




On Sunday, February 4th, 2024 at 18:59, Roland Clobus <rcl...@rclobus.nl> wrote:

> On 04/02/2024 17:41, Roland Clobus wrote:
> ...
>
> > echo 'deb [arch=amd64
> > signed-by=/usr/share/keyrings/1password-archive-keyring.gpg]
> > https://downloads.1password.com/linux/debian/amd64 stable main' >
> > config/includes.chroot_before_packages/etc/apt/sources.list.d/1password.list
>
>
> And I'm certain that there is a more secure way, that ensures that only
> the package called '1password' will come from this repository.
> The bug report was based on a kali version of live-build, so I assume
> you know better than me how to do so.
> Please add such command to the bug report, so I can update the
> live-manual to address such use case.
>
> With kind regards,
> Roland Clobus

[Arszilla]

Hi again,

I forgot to mention that I have been installing 1Password using the commands that 1Password provided on their support page instead of the Debian Package. However, I realized I had missed the debsig-verify step. As a result, I modified my hook to look like the following:

> #!/bin/bash
> set -e
> # Create a temporary user, because otherwise 1Password will break the Live System:
> # Refer to: https://gitlab.com/kalilinux/build-scripts/live-build-config/-/issues/63
> useradd --no-create-home --user-group --uid 1000 tmpuser
> # Add 1Password's GPG key and repository:
> curl -sS https://downloads.1password.com/linux/keys/1password.asc | gpg --dearmor -o /usr/share/keyrings/1password-archive-keyring.gpg
> echo 'deb [arch=amd64 signed-by=/usr/share/keyrings/1password-archive-keyring.gpg] https://downloads.1password.com/linux/debian/amd64 stable main' | tee /etc/apt/sources.list.d/1password.list
> # Add the "debsig-verify" policy:
> mkdir -p /etc/debsig/policies/AC2D62742012EA22/
> curl -sS https://downloads.1password.com/linux/debian/debsig/1password.pol | tee /etc/debsig/policies/AC2D62742012EA22/1password.pol
> mkdir -p /usr/share/debsig/keyrings/AC2D62742012EA22
> curl -sS https://downloads.1password.com/linux/keys/1password.asc | gpg --dearmor --output /usr/share/debsig/keyrings/AC2D62742012EA22/debsig.gpg
> # Install 1Password:
> apt-get update
> apt-get install -y 1password
> # Delete the temporary user:
> userdel tmpuser

Despite this, I can see that `1password` is listed for removal in the `filesystem.packages-remove` in the brand new ISO I have generated with this change (alongside some of the other packages I have listed). This can be seen in the following paste: https://paste.debian.net/1306346/

I have also realized I have provided a wrongful statement about Tailscale upon examining that `filesystem.packages-remove`: It is indeed written for removal. Thus, I kindly ask you to disregard my previous statement regarding Tailscale.

Kind regards,
Arszilla