Rebuilding Debian Live gnome image fails

[Matthijs]

Hi all,

I'm trying to build a custom Debian live USB image. To do so, I start
trying to rebuild an original non-modified live image - specifically
from
"https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid",
the debian-live-10.8.0-amd64-gnome.iso image.

Following the Debian Live manual on using a predefined
configuration(https://live-team.pages.debian.net/live-manual/html/live-manual/managing-a-configuration.en.html#333):
$ mkdir live-images && cd live-images
$ lb config --config
https://salsa.debian.org/live-team/live-images.git::debian
$ cd images/standard
$ sudo lb build

...this works and I get an ISO image in that directory.

But, instead doing this:
$ mkdir live-images && cd live-images
$ lb config --config
https://salsa.debian.org/live-team/live-images.git::debian
$ cd images/gnome-desktop
$ sudo lb build
(note the change in the 'cd' command)
...this does NOT work. The build command starts doing a lot of work,
fetching & installing stuff, but then simply stops at this point
(copy/paste of ~17 lines of build.log):

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
The following NEW packages will be installed:
dctrl-tools
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 104 kB of archives.
After this operation, 413 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 dctrl-tools amd64
2.24-2+b1 [104 kB]
Fetched 104 kB in 0s (1481 kB/s)
Selecting previously unselected package dctrl-tools.
(Reading database ... 9125 files and directories currently installed.)
Preparing to unpack .../dctrl-tools_2.24-2+b1_amd64.deb ...
Unpacking dctrl-tools (2.24-2+b1) ...
Setting up dctrl-tools (2.24-2+b1) ...
P: Begin unmounting filesystems...
P: Saving caches...
Reading package lists...
Building dependency tree...
Reading state information...
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

No further output, it just quits. Same for cinnamon-desktop, kde-desktop
and lxde-desktop. Comparing build.log from gnome-desktop with standard
doesn't give me a clue to why it stops.

In case it is relevant: I'm building on a Debian Buster 10.8 AMD64
machine, without any desktop (headless machine with ssh access only).

Anyone here with a suggestion how to make "lb build" work for a
gnome-desktop Debian config?

Thanks!
Matthijs

[John Crawley]

On 09/02/2021 21:40, Matthijs wrote:
> Following the Debian Live manual on using a predefined configuration(https://live-team.pages.debian.net/live-manual/html/live-manual/managing-a-configuration.en.html#333):
> $ mkdir live-images && cd live-images
> $ lb config --config https://salsa.debian.org/live-team/live-images.git::debian
> $ cd images/standard
> $ sudo lb build
>
> ...this works and I get an ISO image in that directory.
>
> But, instead doing this:
> $ mkdir live-images && cd live-images
> $ lb config --config https://salsa.debian.org/live-team/live-images.git::debian
> $ cd images/gnome-desktop
> $ sudo lb build
> (note the change in the 'cd' command)
> ...this does NOT work. The build command starts doing a lot of work, fetching & installing stuff, but then simply stops at this point (copy/paste of ~17 lines of build.log):

It's not mentioned in those docs, but I'm pretty sure you'll need to run 'lb config' one more time after moving to the config directory (and possibly editing the files inside), before 'lb build'.

$ mkdir live-images && cd live-images
$ lb config --config https://salsa.debian.org/live-team/live-images.git::debian
$ cd images/gnome-desktop

( edit auto/config or other files to taste )
$ lb config
$ sudo lb build

---
I've never used that '--config' option to pull a git repo, but if you have git installed you could also do:
$ git clone https://salsa.debian.org/live-team/live-images
$ git checkout debian
You then have the various configs available in images/. Possibly copy one of those directories elsewhere before building, to keep the git repo clean.

--
John

[Matthijs]

On 12-02-2021 03:12, John Crawley wrote:
> On 09/02/2021 21:40, Matthijs wrote:
>> Following the Debian Live manual on using a predefined
>> configuration(https://live-team.pages.debian.net/live-manual/html/live-manual/managing-a-configuration.en.html#333):
>> $ mkdir live-images && cd live-images
>> $ lb config --config
>> https://salsa.debian.org/live-team/live-images.git::debian
>> $ cd images/standard
>> $ sudo lb build
>>
>> ...this works and I get an ISO image in that directory.
>>
>> But, instead doing this:
>> $ mkdir live-images && cd live-images
>> $ lb config --config
>> https://salsa.debian.org/live-team/live-images.git::debian
>> $ cd images/gnome-desktop
>> $ sudo lb build
>> (note the change in the 'cd' command)
>> ...this does NOT work. The build command starts doing a lot of work,
>> fetching & installing stuff, but then simply stops at this point
>> (copy/paste of ~17 lines of build.log):
> It's not mentioned in those docs, but I'm pretty sure you'll need to
> run 'lb config' one more time after moving to the config directory
> (and possibly editing the files inside), before 'lb build'.

Interesting. I've tried it, but it doesn't make much of a difference -
stops at roughly the same point (well, I get two lines extra, but not
much information from that).

I did manage to trace the root cause back to lines 75/76 in
/usr/lib/live/build/chroot_package-lists:
Expand_packagelist "$(basename ${LIST})" "config/package-lists" \
| grep -v '^#' >> chroot/root/packages.chroot

It seems that at some point, with the config I'm using,
"Expand_packagelist" does not return anything and the script then simply
stops. When I replace it with:
My_list=$(Expand_packagelist "$(basename ${LIST})"
"config/package-lists")
echo ${My_list} | grep -v '^#' >> chroot/root/packages.chroot

...the build continues and delivers a bootable image. So, my issue seems
to be solved with that, or at least I have a workaround. Perhaps I'll
dive a bit further, because right now I don't fully understand why the
build would stop originally: I would expect, if "expand_packagelist"
doesn't return anything, that the following "grep" would hang
indefinitely.

Nevertheless, an additional "lb config" doesn't seem to be necessary, as
the modified chrot_package-list script works correctly with the original
procedure.

--
Matthijs

[John Crawley]

This might be the issue that was fixed with a commit on Jan. 12th:
https://salsa.debian.org/live-team/live-build/-/commit/f13273368a16df271e4317d413d9da564f541dea
You could try that fix of appending '|| true' to the line.

> Nevertheless, an additional "lb config" doesn't seem to be necessary, as the modified chrot_package-list script works correctly with the original procedure.

You're right in this case, because auto/config holds exactly the same configuration for all the desktop versions. I still think, however, that if you edited auto/config inside the gnome (or whatever) directory, that you'd have to run 'lb config' inside the directory to pick the changes up.

Just in the interest of science, I have a local git clone of live-images, so I copied images/gnome-desktop to a virtual machine and ran the same live-build commands there. The build failed at exactly the same point you recorded.

--
John

[debian-...@vanaalten.net]

That's exactly the same issue! And indeed that fix works as well. Hmmm,
I looked through the bug report on packages.debian.org, but didn't think
to look at that page you mentioned. But, good to have it solved. Thanks!

>> Nevertheless, an additional "lb config" doesn't seem to be necessary,
>> as the modified chrot_package-list script works correctly with the
>> original procedure.
>
> You're right in this case, because auto/config holds exactly the same
> configuration for all the desktop versions. I still think, however,
> that if you edited auto/config inside the gnome (or whatever)
> directory, that you'd have to run 'lb config' inside the directory to
> pick the changes up.

Might be. I'll keep it in mind. For now, I'm just playing with the
possibilities, trying to get a squashfs-content as much as possible
equal to the official Debian release, then make sure I can reproduce it
in a scripted way. Then the fun start, to make the small changes that I
need in my personalized version.

--

Matthijs