Missing/Removed? obs://Virtualization:Appliances:Builder/xUbuntu_22.04

[mac junior]

I have been attempting to use Kiwi NG to build an Ubuntu image for my company. I have been using this config as an example to work from https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86_64/ubuntu-jammy/config.xml. This example config uses the Appliance Builder xUbuntu_22.04. This has been working fine for the past couple of weeks, until this morning. This morning it returns a 404 when it is looking for "xUbuntu_22.04". From what I understand the url of "obs://Virtualization:Appliances:Builder/xUbuntu_22.04" refernces this repo https://build.opensuse.org/project/show/Virtualization:Appliances:Builder. When I go here I only see a xUbuntu_23.04 version and the xUbuntu_22.04 does not seem to exist anymore.

This brings up a couple of questions for me

1. Is there a way to clone what I need to build an ISO from this OBS so that we are not reliant on the public repo at https://build.opensuse.org. Mainly we need to guarantee that we can rebuild an ISO and if things change on the public repo as I am seeing this morning it can break the flow. What is the best solution for managing something like obs in house and would we clone it from https://build.opensuse.org
2. Will xUbuntu_22.04 be available on https://build.opensuse.org again? 22 is LTS and 23 is only a 9 month supported distribution so I was surprised that xUbuntu_22.04 was missing this morning.
3. I did notice that the kiwi-ng binary was updated overnight. I did a fresh install this morning on Ubuntu and it pulled v10.0.2. I am noticing that the config options set in https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86_64/ubuntu-jammy/config.xml are no longer support by v10. Everything was working fine with v9.25.22 last week, along side the xUbuntu_22.04 appliance. Is there a plan to fix the templates or a guide on what config options have change from v9 to v10?

Thank you for any help!

-Mac

[Marcus Schäfer]

Hi Mac,

> I have been attempting to use Kiwi NG to build an Ubuntu image for my
> company. I have been using this config as an example to work from
> https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86_64/u
> buntu-jammy/config.xml. This example config uses the Appliance Builder
> xUbuntu_22.04. This has been working fine for the past couple of weeks,
> until this morning. This morning it returns a 404 when it is looking
> for "xUbuntu_22.04". From what I understand the url of
> "obs://Virtualization:Appliances:Builder/xUbuntu_22.04" refernces this
> repo

You are right. The Virtualization:Appliances:Builder project keeps
builds for the two latest versions of the distributions. For Ubuntu
there is 23.04 and 23.10 and 22.04 fell out of the matrix.

But this was not the main reason for us to drop the 22.04 target
this time. The main reason was that we also released a new major
version of kiwi (v10.x.x) for which we decided to let go some
compatibility settings. Also see https://github.com/OSInside/kiwi/issues/2369

The current main code stream still runs on 22.04 and also down
to python v3.6 but we decided to not strictly be bound on these
targets to allow further development of kiwi towards newer
distributions.

I'm sorry that this hit you in your work. To unblock you and maybe
others I have created Virtualization:Appliances:Builder:Deprecated
which allows us to build the v9 kiwi version for older targets.

https://build.opensuse.org/project/show/Virtualization:Appliances:Builder:Deprecated

I updated the ubuntu-jammy description to reference this repo.
That should bring you back on the former state.

However, please note if possible you should switch to the latest
kiwi code base. The reason why the current package does not build
for Ubuntu 22.04 LTS is a too old version of poetry. If you can manage
to provide an updated poetry for 22.04 you can easily build kiwi v10
for it.

> This brings up a couple of questions for me

> 1. Is there a way to clone what I need to build an ISO from this OBS

> so that we are not reliant on the public repo at
> https://build.opensuse.org. Mainly we need to guarantee that we can
> rebuild an ISO and if things change on the public repo as I am
> seeing this morning it can break the flow. What is the best
> solution for managing something like obs in house and would we
> clone it from https://build.opensuse.org

I think the easiest way is if you create your own project on build.opensuse.org
Then copy the version of kiwi which you don't want to see changed by

osc coppac Virtualization:Appliances:Builder python-kiwi YOUR-PROJECT

In your image build reference YOUR-PROJECT instead of ours and that
decouples you from the development.

> 2. Will xUbuntu_22.04 be available on https://build.opensuse.org

> again? 22 is LTS and 23 is only a 9 month supported distribution so
> I was surprised that xUbuntu_22.04 was missing this morning.

Again sorry. As I wrote I provided Virtualization:Appliances:Builder:Deprecated
which brings it back. But I will not update this kiwi version there. v9
is a stable fix release.

> 3. I did notice that the kiwi-ng binary was updated overnight. I did a

> fresh install this morning on Ubuntu and it pulled v10.0.2. I am
> noticing that the config options set in
> https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86_
> 64/ubuntu-jammy/config.xml are no longer support by v10. Everything
> was working fine with v9.25.22 last week, along side the
> xUbuntu_22.04 appliance. Is there a plan to fix the templates or a
> guide on what config options have change from v9 to v10?

That I don't see:

kiwi-ng --version
KIWI (next generation) version 10.0.3

kiwi-ng image info --description /home/ms/Project/kiwi-descriptions/ubuntu/x86_64/ubuntu-jammy

[ INFO ]: 15:15:18 | Reading runtime config file: '/etc/kiwi.yml'
[ INFO ]: 15:15:19 | Loading XML description
[ INFO ]: 15:15:19 | Support for multiple markup descriptions available
[ INFO ]: 15:15:19 | --> loaded /home/ms/Project/kiwi-descriptions/ubuntu/x86_64/ubuntu-jammy/config.xml
[ INFO ]: 15:15:19 | --> Selected build type: iso
[ INFO ]: 15:15:19 | --> Selected profiles: Live
{
"image": "Ubuntu-22.04_appliance"
}

So I can build this with v9 and v10 given the repo setup is correct.
Do you have more details which options are not valid anymore ?

Let me know if this helps

Best regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer Brunnenweg 18
Tel: +49 7562 905437 D-88260 Argenbühl
Germany
-------------------------------------------------------

[mac junior]

Marcus,

Thank you for your response, I really appreciate it. 

Thank you for hosting the Virtualization:Appliances:Builder:Depracated section as well, that will be helpful. I understand with only supporting the last two releases of Ubuntu but 22.04 is LTS while the current 2 versions are not LTS. I did a test and switched our appliance config to use "xUbuntu_23.04" and when I deploy the ISO to test it it shows that it is running 22.04, which is what I am wanting but I'm not understanding how the version of Ubuntu is dictated since I used the xUbuntu_23.04 Virtualization:Appliances:Builder?

One other question that I have (I could have about 5 more to learn and really understand what's happening) is how do I know what base packages are installed with Virtualization:Appliances:Builder:xUbuntu_23.04? Lets say I want the ISO to end up like I selected the Ubuntu Server install method (as selected below). It seems like the example here https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86_64/ubuntu-jammy/config.xml will build the ISO with the "Ubuntu Server (minimized)" option below.

If you don't mind answering this question that would help me out a lot as well. Is there a way for me to create my own repo/clone of Virtualization:Appliances:Builder:Depracated:xUbuntu_22.04. If I do that is there a lot of overhead maintenance for keeping the packages up to date?

Thank you very much!

-Mac

[Marcus Schäfer]

Hi,

> ...current 2 versions are not LTS.

LTS is Ubuntu's long term service plan like it exists for other
distributions e.g SLES. These long term service agreements targets
customers in the enterprise area and last for 10-15 years. What
you see on Virt:App:Builder is kiwi's latest and greatest upstream
code base and there is no commitment to stay compatible with
distributions that can last for decades ;) As I work for SUSE
there is of course a kiwi version also for SLES, but this is an
extra maintenance effort from an extra branch. Regarding Ubuntu
LTS and kiwi, this would be a decision of Canonical if and how
they want to support applications. kiwi is availabe in Ubuntu with
Lunar for the first time:

https://packages.ubuntu.com/lunar/otherosfs/kiwi

I doubt it will ever exist for jammy (22.04) LTS

> I did a test and switched our appliance
> config to use "xUbuntu_23.04" and when I deploy the ISO to test it it
> shows that it is running 22.04, which is what I am wanting but I'm not
> understanding how the version of Ubuntu is dictated since I used the
> xUbuntu_23.04 Virtualization:Appliances:Builder?

I'm not sure what exactly you changed but my assumption is that
the host system which builds your image is 23.04 and the image
you are building is for 22.04, which is fine

> One other question that I have (I could have about 5 more to learn and
> really understand what's happening) is how do I know what base packages
> are installed with Virtualization:Appliances:Builder:xUbuntu_23.04?
> Lets say I want the ISO to end up like I selected the Ubuntu Server
> install method (as selected below). It seems like the example
> here https://github.com/OSInside/kiwi-descriptions/blob/main/ubuntu/x86
> _64/ubuntu-jammy/config.xml will build the ISO with the "Ubuntu Server
> (minimized)" option below.

I'm not sure if I understand the question correctly, but if you want
to know what packages would be installed to build the image you can
resolve this as follows:

kiwi-ng image info --description /path/to/image/description --resolve-package-list

Make sure you have python3-solv installed

> Is there a way for me to create my own repo/clone of
> Virtualization:Appliances:Builder:Depracated:xUbuntu_22.04. If I do
> that is there a lot of overhead maintenance for keeping the packages up
> to date?

If you have an Open Build Service account you can create your own
project and then copy or branch it. To do this you need the "osc"
tool installed on your system and then you can get more information
about it via:

osc branch --help
osc copypac --help

Hope this helps

Regards,

[mac junior]

Thanks for the quick response! I have a couple more questions 🙂

1. What I was asking with the screen shot above is that when installing Ubuntu Server with the ISO (from the Ubuntu website) a user is presented with a couple install options: "Ubuntu Server" "Ubuntu Server (minimized)" and the "additional options". It seems like if I build the example appliance here that the minimized version of Ubuntu is installed. How can I installed the non minimized version? Do I have to figure out what packages are included with the non-minimized version and explicitly add them as individual packages or I do see in the documentation that there is a concept of namedCollection, does there happen to be a namedCollection for the non-minimized version of Ubuntu Server?
2. I'm having a hard time understanding from the docs but when I run a build process for Kiwi using the the xUbuntu_22.04 Appliance image as defined here, what does that image all include in it? Is it the core Ubuntu 22.04 binaries and then Kiwi uses the host repositories to install the additional packages specified within the config.xml. Is the process on how the Appliance xUbuntu_22.04 is made documented or opened source anywhere? That might help me understand how the base image of it works.

As for the Ubuntu_22.04 / xUbuntu_23.04 mix up that I mentioned I will have to do some testing and see what the results are again to make sure I was building/configured it correctly.

Thank again, I really appreciate the help

-Mac

[Marcus Schäfer]

Hi,

> 1. What I was asking with the screen shot above is that when

> installing Ubuntu Server with the ISO (from the Ubuntu website) a
> user is presented with a couple install options: "Ubuntu Server"
> "Ubuntu Server (minimized)" and the "additional options". It seems

> like if I build the example appliance [1]here that the minimized

> version of Ubuntu is installed. How can I installed the non
> minimized version? Do I have to figure out what packages are
> included with the non-minimized version and explicitly add them as
> individual packages or I do see in the documentation that there is

> a concept of [2]namedCollection, does there happen to be a

> namedCollection for the non-minimized version of Ubuntu Server?

Ah ok now I understand the question. So kiwi uses apt (like Ubuntu does)
to install the packages. In order to replicate the non-minimized variant
you need to find out the main components that this install variant uses.

For distributions that implements a concept of patters or collections or
groups provided as packages kiwi supports them. For Debian based
distributions I'm not aware that they have something like that. I think
their collection handling is tightly coupled with the individual tooling
(the installer) and not exposed in .deb packages. I might be wrong on
that though.

Thus the only way I know is to look at an already installed
non-minimized variant or install one into a VM and fetch the package
list. Problem here is that you get a fully resolved package list and
that can be a long list which you don't need because a lot of stuff
is pulled in by dependencies. So you surely can reduce the list a lot, e.g
library packages and such don't have to be explicitly listed.

I also like to encourage you to ask such questions on the Ubuntu
mailing list. The people there surely knows what makes up an Ubuntu
server or a better way to nail it down to the main package names that
you then can use in kiwi

> 2. I'm having a hard time understanding [3]from the docs but when I

> run a build process for Kiwi using the the xUbuntu_22.04 Appliance

> image as defined [4]here, what does that image all include in it?

> Is it the core Ubuntu 22.04 binaries and then Kiwi uses the host
> repositories to install the additional packages specified within
> the config.xml. Is the process on how the Appliance xUbuntu_22.04
> is made documented or opened source anywhere? That might help me
> understand how the base image of it works.

The workflow is documented here:

https://osinside.github.io/kiwi/concept_and_workflow.html

What has been installed is provided to you as part of the results.
What results are produced is documented here:

https://osinside.github.io/kiwi/image_types_and_results.html#image-results

So the .packages file contains the information you are looking for.

In short kiwi works with repositories. It bootstraps a system
(in case of Debian based it uses debootstrap or a prebuilt-binary).
It then installs the packages that you provided (in case of Debian via apt).
That package manager call resolves the given list and installs it.
The same action as if you would call "apt install something" just in
a new root.

To isolate the image build from your host, some tools needs to be called
to build an image, we recommend:

https://osinside.github.io/kiwi/plugins/self_contained.html