Building Disk Update Driver ISOs with Kiwi

[Daniel Juárez González]

Hello,

We are checking ways of creating Disk Update Driver disks (DUDs), which are basically built by:

mkdir -p dd/rpms/$basearch

echo -e "Driver Update Disk version 3\c" > ./dd/rhdd3

pushd dd/rpms/$basearch

# Download rpms specified by a list

popd

mkisofs -quiet -lR -V OEMDRV -input-charset utf8 -o ${dd-name}.iso ./dd

ref. https://gitlab.cern.ch/djuarezg/testing_dud_building/-/blob/master/mkdd.sh

We would like to use Kiwi to do the same. Is it even possible? From what I have seen we could place that specific struct inside the Kiwi built ISO file, but there may be extra stuff in the ISO we do not want.

For context,  Koji image building features will start including Kiwi: https://pagure.io/koji/pull-request/3069#. We were investigating if this tool is actually useful for building also such DUD ISOs.

Cheers,

Daniel

[Neal Gompa]

Kiwi typically creates bootable media. Judging by the script you've
got there, it sounds like DUDs aren't supposed to be bootable Linux
systems, and instead are supposed to be more or less data disks.

(This makes some sense, since DUDs were originally floppy diskettes...)

Kiwi (and most appliance creation tools) would probably be overkill
for this, though I suppose you could use KIWI to make installation
media ISOs that also have DUD content preloaded? That might be
interesting...


--
真実はいつも一つ！/ Always, there's only one truth!

[Marcus Schäfer]

Hi,

> > Hello,
> > We are checking ways of creating Disk Update Driver disks (DUDs), which are basically built by:
> >
> > mkdir -p dd/rpms/$basearch
> > echo -e "Driver Update Disk version 3\c" > ./dd/rhdd3
> > pushd dd/rpms/$basearch
> > # Download rpms specified by a list
> > popd
> > mkisofs -quiet -lR -V OEMDRV -input-charset utf8 -o ${dd-name}.iso ./dd
> >
> > ref. https://gitlab.cern.ch/djuarezg/testing_dud_building/-/blob/master/mkdd.sh
> >
> > We would like to use Kiwi to do the same. Is it even possible? From what I have seen we could place that specific struct inside the Kiwi built ISO file, but there may be extra stuff in the ISO we do not want.
> >
> > For context, Koji image building features will start including Kiwi: https://pagure.io/koji/pull-request/3069#. We were investigating if this tool is actually useful for building also such DUD ISOs.
> >
>
> Kiwi typically creates bootable media. Judging by the script you've
> got there, it sounds like DUDs aren't supposed to be bootable Linux
> systems, and instead are supposed to be more or less data disks.
>
> (This makes some sense, since DUDs were originally floppy diskettes...)
>
> Kiwi (and most appliance creation tools) would probably be overkill
> for this,

I agree to this estimation. From an appliance building perspective
the result is always a system... which you don't need if I got it right.
So 95% of the workflow of building an appliance is probably not of interest
with regards to creating a "Driver Update Disk"

> though I suppose you could use KIWI to make installation
> media ISOs that also have DUD content preloaded? That might be
> interesting...

We had a similar conversation a while back. An installation media
is usually a bootable ISO that runs an installation process. There
was an evaluation to use a live image ISO which comes with content
suitable to drive an installation or update, etc... process.

The particular test case was very easy and consisting out of
the following components:

- kiwi image description building a live ISO
- included in the ISO, a local repository with some update packages
- included in the ISO, a super simple update script called via
a systemd unit and capable to see the installed system

The result ISO was dumped on a stick, booted up and started
the process to update the packages from the embedded repository.
At the end it kexec'ed the installed system with the updates.

The biggest advantages here were

1. the presence of a full featured Linux system (the live ISO)
such that you could also run complex update processes.

2. All that can be done without interaction even in cloud environments
(grub loopback boot)

I can imagine the process that needs to run for DUDs could run in
such an environment too. But I also think this particular update procedure
is well included into the capabilities of the installed OS and
therefore driving it through a live ISO is probably overkill
as well.

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Software Solutions Germany GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Daniel Juárez González]

> But I also think this particular update procedure is well included into the capabilities of the installed OS and therefore driving it through a live ISO is probably overkill as well.

Unless I am misunderstanding you mean that the package updates can be done on the running OS, don't you? The key feature for the DUDs being in an ISO is that they can be used during a later kickstart installation, adding a pointer to their URLs as a kernel parameter / boot option.

This way the rpms are installed during the initial automated installation, which would help for the compatibility of for example deprecated drivers that are needed during the initial installation.

From https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/performing_an_advanced_rhel_installation/updating-drivers-during-installation_installing-rhel-as-an-experienced-user: "A driver update should only be performed if an unsupported driver prevents the installation from completing. Updating drivers during installation is typically only required to support a particular configuration. For example, installing drivers for a storage adapter card that provides access to your system’s storage devices."

Cheers,

Daniel

[Daniel Juárez González]

In a related matter, I am curious if using https://osinside.github.io/kiwi/building_images/build_kis.html would allow us to install the drivers during that PXE image building. Not sure if this would then provide a vmlinuz/kernel and initrd that already has our deprecated drivers inside, ready to be used for network installations.

Any idea if what I just said makes sense at all?

Cheers,

Daniel

[Neal Gompa]

On Fri, Oct 22, 2021 at 11:28 AM Daniel Juárez González
<juarezgonz...@gmail.com> wrote:
>
> In a related matter, I am curious if using https://osinside.github.io/kiwi/building_images/build_kis.html would allow us to install the drivers during that PXE image building. Not sure if this would then provide a vmlinuz/kernel and initrd that already has our deprecated drivers inside, ready to be used for network installations.
>
> Any idea if what I just said makes sense at all?

You could do that, yes. If you include the kmod packages in the image
build, then it would be possible.

[Marcus Schäfer]

Hi,

> Unless I am misunderstanding you mean that the package updates can be
> done on the running OS, don't you?

Partially, let me explain what I think is different.

In the current way to run the update process, you said:

> The key feature for the DUDs being
> in an ISO is that they can be used during a later kickstart
> installation, adding a pointer to their URLs as a kernel parameter /
> boot option.

So this means and correct me when wrong:

- You create an ISO which contains drivers/packages... something to update
and probably a kickstart directive to actually install those.
This ISO doesn't need to be bootable

- You have a running system that needs these updates

- You reboot that system and pass along parameters such that
the source location of the ISO is known to the system and that
kickstart procedure can run

- maybe you reboot the system after the changes were applied


Is this roughly correct ?


If so let's make the switch to an image based process. If you
think a bootable procedure based on an image is useful the process
would basically look like this:

- You create a kiwi image description which builds you a
let's call it "update system". This appliance would be a live ISO
type. You can boot this system from stick, CD, DVD, grub-loopback,
remote via PXE...

- In that image description you place a process which runs at boottime
of the appliance. This means you would need to add the drivers/packages
and also the tools and metadata for kickstart, meaning kickstart
itself and the kickstart file and maybe a systemd unit to start it.
The best choice for the appliance OS in your case would be probably
Fedora or RHEL because that's were kickstart is a supported component

- In that image description you also need a little bit of code that
can identify and mount the system that should receive these updates.
I already wrote that for use in another project
https://github.com/SUSE/suse-migration-services/blob/master/suse_migration_services/units/mount_system.py
So you could consider to re-use parts of it

- Now everything is in one place, in your kiwi image description

- Now you would build the image.. call kiwi-ng ...

- The result is an ISO image which is also a system that you can boot
and which is designed to perform an upgrade process.

- To make this "update system" start you could use kexec or a grub
loopback entry which can boot off from an ISO. This also means
your running system will be offline as long as the update runs
and will come back online as soon as the update is done

- The "update system" live ISO can also boot from remote via PXE


You see implementing it through an image concept is imho more complex
and for the relatively simple DUD approach via kickstart on the target
machine probably overkill.

I have implemented this concept to run distribution upgrades from
one major release to another which cannot be done while the target
machine is still running. I have also done it that way to ensure
a non-interactive process which is required when ugrading machines
in the cloud. So my use case was somewhat different than yours.

But in theory this is the same thing

Sorry for the long mail and I hope this makes sense to you