kiwi-ng or yast/autoyast for suse elemental

[Gabriel Machado]

Hey Marcus,

In our company, we plan to deploy a rancher infrastructure with physical servers.

First, we'll make a custom suse microOS iso with some packages/tools used by our company to deploy rancher nodes.

Next we'll probably use elemental to customize/update/deploy these rancher nodes.

Initially, we were thinking of doing the first iso with kiwi-ng. But I read this discussion : https://groups.google.com/g/kiwi-images/c/54m803HufGI

So I would like to know if we should use kiwi-ng or yast/autoyast for ours needs ?

[Marcus Schäfer]

Hi,

> In our company, we plan to deploy a rancher infrastructure with
> physical servers.

ok

> First, we'll make a custom suse microOS iso with some packages/tools
> used by our company to deploy rancher nodes.

ok

> Next we'll probably use elemental to customize/update/deploy these
> rancher nodes.

ok

> Initially, we were thinking of doing the first iso with kiwi-ng. But I
> read this discussion
> : https://groups.google.com/g/kiwi-images/c/54m803HufGI
> So I would like to know if we should use kiwi-ng or yast/autoyast for
> ours needs ?

Good question. So at SUSE there are two ways to produce products.
The, I name it classical product ISO, and the appliance(image) based
approach.

The classical product ISO is produced without using kiwi, inhouse
with special tools, some magic and the glue for all this is yast.
So when the classical product ISO boots it starts yast. Such an
ISO kiwi cannot produce.

You mentioned microOS. Micro OS is a 100% appliance based product
offering. This means Micro OS is an image produced through a kiwi
image description and this is also true for the install ISO that
is created. The base Micro OS image works without yast components.

To answer your question:

> ... if we should use kiwi-ng or yast/autoyast for ours needs ?

With regards to Micro OS you should use kiwi. You said you want to
modify microOS and add some packages/tools. For this purpose you
would need to take the microOS kiwi image description and change
it according to your needs. If you do this within the buildservice
it would mean:

osc copypac SUSE:ALP:Products:Marble:6.0 SLE-Micro YOUR-PROJECT
==> modify it, done

In this way of working there is no yast involved.

I hope this helps with your next steps

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer

[Gabriel Machado]

Thank you for your reply Marcus.

Must i use OBS to build this appliance ? If yes, for my appliance be private (not viewable by everyone) must I create my own OBS instance ?

I tried to build kiwi opensuse microOS image from tumbleweed and tumbleweed-microOS hosts but it didn't work. I think there is host constraints problems ?

I only succeed build an image with an universal boxbuild vm. Is it normal ?

osc co openSUSE:Factory/openSUSE-MicroOS && cd openSUSE\:Factory/openSUSE-MicroOS/

(I modified the repos in the kiwi project)

kiwi-ng --profile=OpenStack-Cloud --debug system boxbuild --box universal -- --description . --target-dir /opt/kiwi/Tumbleweed-MicroOS

Regards,
Gabriel.

[Marcus Schäfer]

Hi,

> Must i use OBS to build this appliance ? If yes, for my appliance be
> private (not viewable by everyone) must I create my own OBS instance ?

From a kiwi perspective, no. You can build any image without OBS.
Key to success here is the availability of the source repositories.
If you want to build this for SLES, this task can be pretty complicated.
That's because SLES repos are not publicly available, but in the buildservice
they are available. As a SLES customer you bought a SLES subscription
which provides you access to the SLES repositories also outside of OBS.
On such a system with access to the SLES repos you can also use
kiwi to build the image.

Actually this is really an issue of how to get access to the repos and
from the kiwi side we can't do much as this is determined by the
business of the companies. If you have setup your image build for
local builds and it does not work I'm happy to assist with the
errors you might run into

> I tried to build kiwi opensuse microOS image from tumbleweed and
> tumbleweed-microOS hosts but it didn't work. I think there is host
> constraints problems ?

Building on your local system for TW should be no issue, but all tools
to build images needs to be installed. I need to know what errors you
got to tell you more though

> I only succeed build an image with an universal boxbuild vm. Is it
> normal ?

boxbuild is a good choice to ensure the host brings everything that
is required to build images. I encourage people to use boxbuild because
you can build as a standard user, your system is not tainted with extra
tools only needed to build images and you can be sure we provide an
environment ready to build. That this succeeds for you is a good sign ;)

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer

-------------------------------------------------------
Marcus Schäfer Brunnenweg 18
Tel: +49 7562 905437 D-88260 Argenbühl
Germany
-------------------------------------------------------

[Gabriel Machado]

Thank you very much for your help.

Please tell me what dependencies are missing.

From a tumbleweed host I have these errors :

[ DEBUG   ]: 20:58:48 | EXEC: [chroot /opt/kiwi/Tumbleweed-MicroOS/build/image-root zypper --non-interactive --gpg-auto-import-keys --pkg-cache-dir /var/cache/kiwi/packages --reposd-dir /var/cache/kiwi/zypper/repos --solv-cache-dir /var/cache/kiwi/zypper/solv --cache-dir /var/cache/kiwi/zypper --raw-cache-dir /var/cache/kiwi/zypper/raw --config /kiwi_rkbv6esr install --download in-advance --auto-agree-with-licenses --no-recommends -- combustion >= 1.2 cryptsetup device-mapper dracut-kiwi-oem-dump dracut-kiwi-oem-repart growpart-generator ignition-dracut jeos-firstboot kernel-default kernel-firmware-all live-add-yast-repos patterns-base-bootloader patterns-microos-base-zypper patterns-microos-basesystem patterns-microos-defaults patterns-microos-hardware patterns-microos-selinux]
[ DEBUG   ]: 20:58:48 | system: Loading repository data...
[ DEBUG   ]: 20:58:49 | system: Reading installed packages...
[ DEBUG   ]: 20:58:49 | system: 'ignition-dracut' not found in package names. Trying capabilities.
[ DEBUG   ]: 20:58:49 | system: Resolving package dependencies...
[ DEBUG   ]: 20:58:49 | system: Problem: the to be installed patterns-microos-base-zypper-5.0-85.1.x86_64 requires 'pattern() = microos_base', but this requirement cannot be provided
[ DEBUG   ]: 20:58:49 | system:   not installable providers: patterns-microos-base-5.0-85.1.x86_64[tumbleweed-oss]
[ DEBUG   ]: 20:58:49 | system:  Solution 1: Following actions will be done:
[ DEBUG   ]: 20:58:49 | system:   do not install patterns-microos-base-zypper-5.0-85.1.x86_64
[ DEBUG   ]: 20:58:49 | system:   do not install patterns-microos-basesystem-5.0-85.1.x86_64
[ DEBUG   ]: 20:58:49 | system:   do not install patterns-microos-defaults-5.0-85.1.x86_64
[ DEBUG   ]: 20:58:49 | system:  Solution 2: deinstallation of busybox-diffutils-1.36.1-32.1.noarch
[ DEBUG   ]: 20:58:49 | system:  Solution 3: break patterns-microos-base-zypper-5.0-85.1.x86_64 by ignoring some of its dependencies
[ DEBUG   ]: 20:58:49 | system: Choose from above solutions by number or cancel [1/2/3/c/d/?] (c): c
[ ERROR   ]: 20:58:49 | KiwiInstallPhaseFailed: System package installation failed:
[ INFO    ]: 20:58:49 | Cleaning up SystemPrepare instance

From an opensuse microOS host :

(venv_kiwi) pc-77:/opt/kiwi/projets/openSUSE:Factory/openSUSE-MicroOS # kiwi-ng --profile=SelfInstall system build --description . --target-dir /opt/kiwi/Tumbleweed-MicroOS
[ INFO    ]: 20:07:59 | Loading XML description
[ INFO    ]: 20:07:59 | Support for XML markup available
[ INFO    ]: 20:07:59 | --> loaded ./openSUSE-MicroOS.kiwi
[ INFO    ]: 20:07:59 | --> Selected build type: oem
[ INFO    ]: 20:07:59 | --> Selected profiles: SelfInstall
[ INFO    ]: 20:07:59 | Preparing new root system
[ INFO    ]: 20:07:59 | Setup root directory: /opt/kiwi/Tumbleweed-MicroOS/build/image-root
[ ERROR   ]: 20:07:59 | KiwiCommandError: rpm: stderr: error: can't create transaction lock on /opt/kiwi/Tumbleweed-MicroOS/build/image-root/usr/lib/sysimage/rpm/.rpm.lock (Permission denied)
, stdout: (no output on stdout)

Regards,
Gabriel.

[Roger Oberholtzer]

On Mon, Mar 11, 2024 at 4:19 PM Marcus Schäfer <marcus....@gmail.com> wrote:

boxbuild is a good choice to ensure the host brings everything that
is required to build images. I encourage people to use boxbuild because
you can build as a standard user, your system is not tainted with extra
tools only needed to build images and you can be sure we provide an
environment ready to build. That this succeeds for you is a good sign ;)

I can confirm that I build 15.5 things on Tumbleweed using boxbuild as the environment. Works great.

I've been curious which openSUSE/SUSE OS versions are supported in boxbuild.

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Please tell me what dependencies are missing.

> [ DEBUG ]: 20:58:49 | system: Problem: the to be installed
> patterns-microos-base-zypper-5.0-85.1.x86_64 requires 'pattern() =
> microos_base', but this requirement cannot be provided
> [ DEBUG ]: 20:58:49 | system: not installable providers:
> patterns-microos-base-5.0-85.1.x86_64[tumbleweed-oss]

You did not share your repo setup from the kiwi file with us.
Thus I can't tell you if it's correct or not. The mentioned pattern
package patterns-microos-base exists in the tumbleweed repo here:

https://download.opensuse.org/tumbleweed/repo/oss/x86_64/patterns-microos-base-5.0-85.1.x86_64.rpm

So if you use "https://download.opensuse.org/tumbleweed/repo/oss"
as your repo you should be able to build it

> [ ERROR ]: 20:07:59 | KiwiCommandError: rpm: stderr: error: can't
> create transaction lock on
> /opt/kiwi/Tumbleweed-MicroOS/build/image-root/usr/lib/sysimage/rpm/.rpm
> .lock (Permission denied)

You get this when you try to build images on an selinux enabled system.
Your security policy seems to not allow what is needed to build images.
You can solve this by either setting selinux to permissive or use
kiwi boxbuild

[Gabriel Machado]

For host tumbleweed microOS you were right. I set selinux to permissive and I've not this error.

Now on tumbleweed microOS, I have the same error as with tumbleweed.

I attached my kiwi file as requested.

I use the url repos you mentioned but I can't install microos_base pattern.

pc-75:/etc/zypp/repos.d # cat /etc/os-release
NAME="openSUSE Tumbleweed"
# VERSION="20240304"
ID="opensuse-tumbleweed"
ID_LIKE="opensuse suse"
VERSION_ID="20240304"
PRETTY_NAME="openSUSE Tumbleweed"
ANSI_COLOR="0;32"
# CPE 2.3 format, boo#1217921
CPE_NAME="cpe:2.3:o:opensuse:tumbleweed:20240304:*:*:*:*:*:*:*"
#CPE 2.2 format
#CPE_NAME="cpe:/o:opensuse:tumbleweed:20240304"
BUG_REPORT_URL="https://bugzilla.opensuse.org"
SUPPORT_URL="https://bugs.opensuse.org"
HOME_URL="https://www.opensuse.org"
DOCUMENTATION_URL="https://en.opensuse.org/Portal:Tumbleweed"
LOGO="distributor-logo-Tumbleweed"
pc-75:/etc/zypp/repos.d # cat -v *
[download.opensuse.org-non-oss]
name=DM-CM-)pM-CM-4t principal (NON-OSS)
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/tumbleweed/repo/non-oss/
path=/
type=rpm-md
keeppackages=0
[download.opensuse.org-oss]
name=DM-CM-)pM-CM-4t principal (OSS)
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/tumbleweed/repo/oss/
path=/
type=rpm-md
keeppackages=0
[download.opensuse.org-tumbleweed]
name=DM-CM-)pM-CM-4t principal de mise M-CM-  jour
enabled=1
autorefresh=1
baseurl=http://download.opensuse.org/update/tumbleweed/
path=/
type=rpm-md
keeppackages=0
[kiwi-appliance-builder]
enabled=1
autorefresh=0
baseurl=http://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_Tumbleweed/
[openSUSE-20240214-0]
name=openSUSE-20240214-0
enabled=0
autorefresh=0
baseurl=cd:/?devices=/dev/disk/by-id/ata-QEMU_DVD-ROM_QM00001
path=/
type=rpm-md
keeppackages=0
[repo-debug]
name=openSUSE-Tumbleweed-Debug
enabled=0
autorefresh=1
baseurl=http://download.opensuse.org/debug/tumbleweed/repo/oss/
path=/
keeppackages=0
[repo-openh264]
name=Open H.264 Codec (openSUSE Tumbleweed)
enabled=1
autorefresh=1
baseurl=http://codecs.opensuse.org/openh264/openSUSE_Tumbleweed
path=/
type=rpm-md
keeppackages=0
[repo-source]
name=openSUSE-Tumbleweed-Source
enabled=0
autorefresh=1
baseurl=http://download.opensuse.org/source/tumbleweed/repo/oss/
path=/
keeppackages=0

pc-75:/etc/zypp/repos.d # zypper install -t pattern microos_base
Retrieving repository 'Dépôt principal (NON-OSS)' metadata .............................................................................................................................................................................[done]
Building repository 'Dépôt principal (NON-OSS)' cache ..................................................................................................................................................................................[done]
Retrieving repository 'Dépôt principal (OSS)' metadata .................................................................................................................................................................................[done]
Building repository 'Dépôt principal (OSS)' cache ......................................................................................................................................................................................[done]
Loading repository data...
Reading installed packages...
Resolving package dependencies...

Problem: the to be installed pattern:microos_base-5.0-85.1.x86_64 requires 'patterns-microos-base', but this requirement cannot be provided
  not installable providers: patterns-microos-base-5.0-85.1.x86_64[download.opensuse.org-oss]

 Solution 1: Following actions will be done:

  deinstallation of product:openSUSE-20240304-0.x86_64
  deinstallation of systemd-presets-branding-openSUSE-12.2-23.2.noarch
  deinstallation of product:openSUSE-20240304-0.x86_64
 Solution 2: do not install pattern:microos_base-5.0-85.1.x86_64
 Solution 3: break pattern:microos_base-5.0-85.1.x86_64 by ignoring some of its dependencies

What's wrong ?

Regards,
Gabriel.

--
You received this message because you are subscribed to a topic in the Google Groups "kiwi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kiwi-images/FptmmheOT7g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kiwi-images...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiwi-images/ZfAkxRvCIxXjL3XF%40asterix.fritz.box.

[Marcus Schäfer]

Hi,

> For host tumbleweed microOS you were right. I set selinux to permissive
> and I've not this error.

ok

> Now on tumbleweed microOS, I have the same error as with tumbleweed.
> I attached my kiwi file as requested.

Thanks. I can reproduce the solver issue. The problem was that
the package snapper requires diffutils but busybox-diffutils was
preferred by zypper for whatever reason. That causes snapper to
be no longer resolvable for patterns-microos-base and that caused
the error you saw.

This is a tricky one and I find it particulary problematic that
the buildservice resolves this differently than zypper. The
solution though is easy:

--- openSUSE-MicroOS.kiwi.orig 2024-03-13 11:11:35.699361079 +0100
+++ openSUSE-MicroOS.kiwi 2024-03-13 11:10:53.795402140 +0100
@@ -794,6 +794,7 @@
</packages>
<packages type="bootstrap">
<!-- Avoid that zypper picks the busybox-foo variant for those, that would fail later -->
+ <package name="diffutils"/>
<package name="coreutils"/>
<package name="gawk"/>
<package name="gzip"/>


Please apply this patch and the build will work.

Explicitly telling that we want diffutils and not busybox-diffutils
fixes the issue

It would be nice if you can report this also to the Micro OS team
because this is a bug in MicroOS

Thanks