Appliance not working moving from Leap 15.3 to Leap 15.4

[glid...@googlemail.com]

I've been using kiwi for many years to build a Mythfrontend appliance ISO, which I then either serve using AOE or write to local SSD. I have several times before run into problems when trying to upgrade to a new version of opensuse. It's rarely (perhaps never) been a problem with kiwi, but there are many experts in this group in the linux install and boot process, and I've always been given good advice that's led to a solution. I know very little in this area. It would be brilliant if someone can help me out again.

My latest problem comes when I just change all the repos referenced in config.xml from the 15.3 versions to the 15.4 versions. The applicance struggles to boot: it seems to get as far as running X but in a non-working state. It runs like an overloaded system - like it's stuck swapping (although it has no swap space allocated). There are also some changes in the logs.

Logs for working and non-working are here: https://download.glidos.net/kiwi-problems/

Where the working system has:

Sep 05 14:24:41 localhost kernel: On node 0 totalpages: 1006685
Sep 05 14:24:41 localhost kernel:   DMA zone: 64 pages used for memmap
Sep 05 14:24:41 localhost kernel:   DMA zone: 23 pages reserved
Sep 05 14:24:41 localhost kernel:   DMA zone: 3995 pages, LIFO batch:0
Sep 05 14:24:41 localhost kernel:   DMA32 zone: 7476 pages used for memmap
Sep 05 14:24:41 localhost kernel:   DMA32 zone: 478402 pages, LIFO batch:63
Sep 05 14:24:41 localhost kernel:   Normal zone: 8192 pages used for memmap
Sep 05 14:24:41 localhost kernel:   Normal zone: 524288 pages, LIFO batch:63

the non-working system has:

Sep 05 15:47:44 localhost kernel: On node 0, zone DMA: 1 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA: 1 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA: 1 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA: 98 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA32: 8529 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA32: 11585 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone DMA32: 172 pages in unavailable ranges
Sep 05 15:47:44 localhost kernel: On node 0, zone Normal: 21504 pages in unavailable ranges

Also the non-working system has:

Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop0.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop0: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/sda.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/sda: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/etherd/e0.1.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/etherd/e0.1: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/mmcblk0.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/mmcblk0: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop1.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop1: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/sda1.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/sda1: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/etherd/e0.1p1.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/etherd/e0.1p1: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop2.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop2: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/sda2.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/sda2: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/etherd/e0.1p2.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/etherd/e0.1p2: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop3.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop3: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop4.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop4: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop5.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop5: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop6.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop6: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/loop7.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/loop7: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/mmcblk0boot0.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/mmcblk0boot0: Failed to get external handle [udev].
Sep 05 15:48:50 slurp lvm[1702]:   Udev database has incomplete information about device /dev/mmcblk0boot1.
Sep 05 15:48:50 slurp lvm[1702]:   /dev/mmcblk0boot1: Failed to get external handle [udev].

One other thing, I wondered about: the ISO image has gone up from about 700MB to 800MB. Maybe that's a problem?

Any advice would be greatly appreciated, thanks.

[Marcus Schäfer]

Hi,

> struggles to boot: it seems to get as far as running X but in a
> non-working state. It runs like an overloaded system - like it's stuck
> swapping (although it has no swap space allocated). There are also some
> changes in the logs.

Unfortunately there are always quite some behavior changes of
several components in a Linux system from one major version to
another. I also suffer from this every time.

* From a booting perspective I don't see an issue. The good/bad
logs shows that it booted up into systemd and seems like
operational from a console perspective

Sep 05 15:49:12 slurp systemd[1]: Reached target Login Prompts.
Sep 05 15:49:12 slurp systemd-logind[1828]: New session c1 of user mythfrontend

* Regarding swap I see the exact same messages from the logs.
Thus cannot say anything useful about the swap setup

* Looking for errors in the logs the good one has more than the bad one :)

I'm afraid according to the logs I did not see an issue that
looks familiar. Both logs ends with the same service startup

Sep 05 15:49:23 slurp su[2348]: (to root) mythfrontend on pts/0
Sep 05 15:49:23 slurp su[2348]: pam_unix(su:session): session opened for user root by mythfrontend(uid=500)
Sep 05 15:49:25 slurp systemd[2250]: Started D-Bus User Message Bus.

From my perspective the system is stuck in "myth" itself somewhere.
From a debugging perspective I would concentrate on this part and
maybe establish a debug ssh access if not already possible.

Sorry if I wasn't that helpful

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer Brunnenweg 18
Tel: +49 7562 905437 D-88260 Argenbühl
Germany
-------------------------------------------------------

[glid...@googlemail.com]

That is actually very helpful, thank you. It's very useful to know that the error messages I singled out don't look to you to be signs of anything untoward. I was already starting to suspect the problems might be with mythfrontend: I build the packages for mythtv myself and this is the only test I've made of the 15.4 version of mythfrontend. My current plan is to temporarily sidestep kiwi and use a standard network-based install on the machine and experiment with that. The great thing about using kiwi is I'll still be able to boot my working image via AOE, leaving the experimental SSD install in place for further tests. I've read there are some problems with Mesa working with Intel graphics. It may turn out to be related to that.

[glid...@googlemail.com]

Solved.

As a test, rather than use kiwi, I performed a standard network install of Leap 15.4 and then added my MythTv packages. It worked perfectly. Checking the X logs, I noticed that xf86-video-intel package hadn't been installed. I was still getting hardware accelerated video playback via the vaapi driver, and hardware accelerated Open GL (2D at least) via something called galmoregl, so performance was fine. So I removed the intel driver package also from my kiwi config, and that also works fine now too. Apparently the intel driver can work with glamorgl, but there are complications (one's I have yet to understand). Thankfully I don't seem to need the intel driver at all.

[Marcus Schäfer]

Hi,

> Solved.

great :)

removing stuff and fixing issues with it, is one of the
most satisfactory workflows ;)

Thanks for sharing the details