custom kernel and install image in kiwi 8

[Santosh]

Hi Marcus

I have a need to use custom kernel for building the appliance image. (video through serial is supported in the custom kernel for my special hardware)

The kernel parameters that I am passing in config.kiwi is getting passed in the grub config and in the isolinux config file but these config files use linux.vmx not my custom kernel.

I have to use my custom kernel and initrd for display to be sent to tty.

how to make this work?

grub.cfg:

------------

menuentry "enterprise_msm [ OEM ]" --class os --unrestricted {

    echo Loading kernel...

    set gfxpayload=800x600

    $linux ($root)/boot/linux.vmx selinux=1 console=uart8250,mmio32,0x9222e000,115200n8 earlyprintk=serial console=ttyS2,115200 console=tty0

    echo Loading initrd...

    $initrd ($root)/boot/initrd.vmx

}

ls -al /mnt/boot

-rw-------.  1 root root 25201171 Jun  9 17:54 initramfs-3.10.0-123.20.1.el7.local.santoshJune2.x86_64.img

-rw-r--r--.  1 root root   588799 Jun  9 17:53 initrd-plymouth.img

-rw-r--r--.  1 root root 62294428 Jun  9 17:58 initrd.vmx

-rwxr-xr-x.  1 root root  4907312 Jun  9 18:01 linux.vmx

-rw-r--r--.  1 root root       11 Jun  9 17:58 mbrid

-rw-r--r--.  1 root root   176500 Jun 10  2014 memtest86+-4.20

-rw-r--r--.  1 root root   228632 Jun  2 12:44 symvers-3.10.0-123.20.1.el7.local.santoshJune2.x86_64.gz

-rw-r--r--.  1 root root  2560080 Jun  9 18:01 unicode.pf2

-rwxr-xr-x.  1 root root  4907312 Jun  2 12:40 vmlinuz-3.10.0-123.20.1.el7.local.santoshJune2.x86_64

Thanks a lot.

[Marcus Schäfer]

Hi,

> I have to use my custom kernel and initrd for display to be sent to

You can use a custom kernel as follows

1. place the kernel package in the system XML description as a bootinclude

<package name="custom-kernel-package" bootinclude="true"/>

2. select the 'custom' kernel profile in the system XML description

<type .... bootkernel="custom" .../>

That way the custom kernel package is used in all places, for the
system and also for the initrd
--
Public Key available via: https://keybase.io
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Santosh]

Hi Marcus

I did this (mentioned below) but I don't see my kernel in the config files:

        <preferences profiles="msm_hardware">

                <type image="oem" boot="oemboot/rhel-07.0" filesystem="ext3" installiso="true" bootkernel="custom" bootloader="grub2"

                                        kernelcmdline="selinux=1 console=ttyS0,115200 console=tty0" firmware="efi" hybrid="true">

                        <oemconfig>

                                <oem-systemsize>24550</oem-systemsize>

                                <oem-swap>true</oem-swap>

                                <oem-swapsize>8192</oem-swapsize>

        <packages type="oem">

                <package name="grub2-efi" bootinclude="true"/>

                <package name="grub2-efi-modules" bootinclude="true"/>

                <package name="kernel" bootinclude="true"/>

        </packages>

        <packages type="image" profiles="custom">

                <package name="kernel" bootinclude="true"/>

        </packages>

and this is grub.cfg from enterprise_msm.raw file:

--------------------------------------------------------------------

menuentry "enterprise_msm [ OEM ]" --class os --unrestricted {

    echo Loading kernel...

    set gfxpayload=800x600

    $linux ($root)/boot/linux.vmx selinux=1 console=uart8250,mmio32,0x9222e000,115200n8 earlyprintk=serial console=ttyS2 console=ttyS0,115200 console=tty0

    echo Loading initrd...

    $initrd ($root)/boot/initrd.vmx

}

I also looked at this : https://github.com/SUSE/kiwi/blob/6161e202c32363b3a203a7a4ebd28b7f07278411/kiwi/boot/arch/x86_64/oemboot/rhel-07.0/config.xml

but couldn't figure out how to make it work !!

[Santosh]

And If I do like this:

        <preferences profiles="msm_hardware">

                <type image="oem" boot="oemboot/rhel-07.0" filesystem="ext3" installiso="true" bootkernel="msm_hardware" bootloader="grub2"

                                        kernelcmdline="selinux=1 console=ttyS0,115200 console=tty0" firmware="efi" hybrid="true">

                        <oemconfig>

                                <oem-systemsize>24550</oem-systemsize>

                                <oem-swap>true</oem-swap>

                                <oem-swapsize>8192</oem-swapsize>

                                <!--

        <packages type="image" profiles="msm_hardware">

                <package name="kernel" bootinclude="true"/>

        </packages>

Then I get errors like this and image creation aborts.

Failed to get D-Bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

background info:

--------------------

If I install custom kernel rpm on a running system (fortunately my current hardware has both VGA and serial but VGA will go away soon), serial starts working fine.

For this, I first install iso without any modification and then install custom kernel. On subsequent boot, I see serial output.

Now, I want to create an iso that can be installed through serial.

[Marcus Schäfer]

Hi,

The name of the kernel file in this origin raw disk configuration
is always the same if the kiwi initrd is used. It takes the installed
kernel file and creates a copy in your case named: linux.vmx. On first boot
dracut and/or mkinitrd is called which replaces all traces of the kiwi
first boot code.

Thus the kernel file linux.vmx should be identical to your installed
kernel provided by the package <package name="kernel" bootinclude="true"/>

After first boot your grub.cfg should look different (no linux.vmx)

The idea behind this is to allow a kernel with different capabilities
only used for the very first boot.

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc

[Santosh]

thanks for the explanation and I tried to build the image with just this entry:

<packages type="image">

..........

<package name="kernel" bootinclude="true"/>

...........

</packages>

My kernel gets picked (see that in the log) but installer won't work..(will go into hardware discovery mode and console would start showing error with some stack)

So, with the above mentioned changes, do I need to have bootkernel="xxx" as well?

[Marcus Schäfer]

Hi,

> thanks for the explanation and I tried to build the image with just
> this entry:
> <packages type="image">
> ..........
> <package name="kernel" bootinclude="true"/>
> ...........
> </packages>
> My kernel gets picked (see that in the log) but installer won't
> work..(will go into hardware discovery mode and console would start
> showing error with some stack)
> So, with the above mentioned changes, do I need to have
> bootkernel="xxx" as well?

yes

bootkernel="custom"

must be set in your <type> definition in addition to the changes you
already have done

[Santosh]

Sorry but I am a bit confused about the "custom" label in bootkernel as I don't have any profile named "custom". Based on https://github.com/SUSE/kiwi/blob/6161e202c32363b3a203a7a4ebd28b7f07278411/kiwi/boot/arch/x86_64/oemboot/rhel-07.0/config.xml,

"custom" needs to be a profile. 

My config.kiwi is like this:

        <profiles>

                <profile name="CentOS" description="Build for CentOS" import="true"/>

                <profile name="ovf" description="Builds OVF image" />

                <profile name="msm_hardware" description="Builds an image for deployment on MSM hardware" />

        </profiles>

        <preferences profiles="msm_hardware">

                <type image="oem" boot="oemboot/rhel-07.0" filesystem="ext3" installiso="true" bootprofile="default" bootloader="grub2"

                                        kernelcmdline="selinux=1 console=uart8250,mmio32,0x9222e000,115200n8 console=ttyS2,115200 console=tty0" firmware="efi" hybrid="true">

                        <oemconfig>

                                <oem-systemsize>24550</oem-systemsize>

                                <oem-swap>true</oem-swap>

                                <oem-swapsize>8192</oem-swapsize>

                                <oem-skip-verify>true</oem-skip-verify>

                        </oemconfig>

                        <!--size unit="G">10</size-->

                </type>

        </preferences>

    <packages type="image"> 
    .......... 
    <package name="kernel" bootinclude="true"/> 
    ........... 
    </packages> 

Now, without having any "custom" profile, what's the significance of adding "custom" in bootkernel ?

Or, can I just put bootkernel="custom" in type for msm_hardware profile and expect it to work?

Thanks a lot.

[Santosh]

I used this:

 <packages type="image">

                <package name="grub2"/>

                <package name="kernel" bootinclude="true"/>

 </packages>

and

<type image="oem" boot="oemboot/rhel-07.0" filesystem="ext3" installiso="true"  bootkernel="custom" bootloader="grub2"

and image got created fine. I thought of dealing with .raw file first (skipping iso to narrow down the issue) so cloned a disk with enterprise_msm.raw image. grub menu is like this:

and installation went into some wierd state: :-(

[Santosh]

To further narrow down the issue, I modified the default grub.cfg of the enterprise_msm.raw to exclude serial configuration and then image installed fine. My guess is, I need custom initrd for this.. but not sure.

[Marcus Schäfer]

Hi,

>
> Sorry but I am a bit confused about the "custom" label in bootkernel as
> I don't have any profile named "custom".

you don't have that ? which kiwi version do you use ?

Introduced first in

v5.05.39

commit eb33c9ed6de3bfae7edb4b646f56daa6a66e0150
Author: Marcus Schäfer <m...@suse.de>
Date: Mon Oct 21 11:45:51 2013 +0200

- added custom bootkernel profile (bnc #846068)
* with the bootkernel="custom" profile one can prevent the
installation of a kernel in the kiwi boot image (initrd)
This makes sense if a non profiled kernel or a specific
version of a kernel is used as part of the system image
XML description. Examples:

<type ... bootkernel="custom"/>

==> version pinned kernel:
<package name="kernel-default-3.7.10-1.16.1" bootinclude="true"/>

==> non profiled/custom kernel:
<package name="kernel-bob" bootinclude="true"/>

If the custom bootkernel attribute is used it's required to
mark the kernel in the system image XML with bootinclude="true"
If this is not done the initrd will not have a kernel/modules
which is in most cases unwanted

Regards,
Marcus
--
Public Key available via: https://keybase.io

[Santosh]

thank you. Got that. I am able to build the with those 2 settings.. but it fails to boot. I put screenshots in my previous replies.

I think I need to put custom initrd as well and for that I used "initrd_system="dracut" in the <type..> but image creation failed with this error.

Please take a look at the screenshots and see if I need to put custom initrd for this. Thanks for your help.

[ INFO    ]: 12:50:48 | Creating image identifier: 0x66aeb68f

[ INFO    ]: 12:50:48 | Creating generic dracut initrd archive

[ ERROR   ]: 12:50:48 | Unexpected error:

Traceback (most recent call last):

  File "/usr/bin/kiwi", line 9, in <module>

    load_entry_point('kiwi==8.15.3', 'console_scripts', 'kiwi-ng')()

  File "/usr/lib/python3.4/site-packages/kiwi/kiwi.py", line 54, in main

    App()

  File "/usr/lib/python3.4/site-packages/kiwi/app.py", line 35, in __init__

    task_class().process()

  File "/usr/lib/python3.4/site-packages/kiwi/tasks/system_build.py", line 198, in process

    result = image_builder.create()

  File "/usr/lib/python3.4/site-packages/kiwi/builder/disk.py", line 364, in create

    self.boot_image.create_initrd(self.mbrid)

  File "/usr/lib/python3.4/site-packages/kiwi/boot/image/dracut.py", line 65, in create_initrd

    kernel_version = kernel_info.get_kernel().version

AttributeError: 'NoneType' object has no attribute 'version'

[ INFO    ]: 12:50:48 | Cleaning up InstallImageBuilder instance

[Santosh]

this is kiwi 8.

[Marcus Schäfer]

Hi,

yep, that one is fixed in master. I will release a new v8 version
with the fix this week

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc

[Santosh]

thank you. Will keep an eye on it.

I opened the initrd that is being used in the default configuration and find the serial driver files there. Not sure why install is going in some weird state.

[Santosh]

I was little impatient so I took changed files such as (dracut.py,...) from master to latest installed version. Build fails with "dracut not found" error. I have dracut package in my config.kiwi file.

[Santosh]

I found some interesting code in boot/functions.sh around tty. In my case, console entries are like this:

console=uart8250,mmio32,0x9222e000,115200n8 console=ttyS2,115200 console=tty0

Do you think it could be an issue here?

On the system where serial is working (custom kernel rpm is installed later on), I see this:

crw-rw----.  1 root dialout   4,  64 Jun  9 01:30 ttyS0

crw-rw----.  1 root dialout   4,  65 Jun  9 01:30 ttyS1

crw--w----.  1 root tty       4,  66 Jun  9 01:30 ttyS2

crw-rw----.  1 root dialout   4,  67 Jun  9 01:30 ttyS3

Thank you

[Marcus Schäfer]

Hi,

> I found some interesting code in [1]boot/functions.sh around tty. In my

> case, console entries are like this:
> console=uart8250,mmio32,0x9222e000,115200n8 console=ttyS2,115200
> console=tty0
> Do you think it could be an issue here?

Sorry I lost track with this thread

The console code is only active for sysV systems, actually I have droped
this code from the v8 version as all distributions supported with it
are systemd based.

[Santosh]

Sorry about that. 

Currently wanting to update to latest kiwi 8 and I am getting error:

 # zypper update python3-kiwi

Building repository 'Virtualization_Appliances' cache .......................................................................................................[done]

Error building the cache:

[|] Failed to cache repo (4).

Disabling repository 'Virtualization_Appliances' because of the above error.

Loading repository data...

Reading installed packages...

Resolving package dependencies...

Problem: nothing provides python3-future needed by python3-kiwi-8.18.0-1.1.x86_64

 Solution 1: do not install python3-kiwi-8.18.0-1.1.x86_64

 Solution 2: break python3-kiwi-8.18.0-1.1.x86_64 by ignoring some of its dependencies

[Marcus Schäfer]

Hi,

> Currently wanting to update to latest kiwi 8 and I am getting error:
> # zypper update python3-kiwi
> Building repository 'Virtualization_Appliances' cache
> .......................................................................
> ................................[done]
> Error building the cache:
> [|] Failed to cache repo (4).
> Disabling repository 'Virtualization_Appliances' because of the above
> error.
> Loading repository data...
> Reading installed packages...
> Resolving package dependencies...
> Problem: nothing provides python3-future needed by

My fault, sorry about that. the publish flag was not set on the package
The repo should be in good shape in a few minutes when the buildservice
has processed the change

[Santosh]

Thanks. I updated the python3-kiwi package and included dracut package in config.kiwi both ways (not at the same time):

<package name="dracut"/>

<package name="dracut" bootinclude="true"/>

and added initrd_system="dracut"  in <type ...> 

but appliance creation keeps failing with this error:

[ DEBUG   ]: 10:32:50 | EXEC: Failed with stderr: chroot: failed to run command ‘dracut’: No such file or directory

, stdout: (no output on stdout)

[ ERROR   ]: 10:32:50 | KiwiCommandError: chroot: stderr: chroot: failed to run command ‘dracut’: No such file or directory

, stdout: (no output on stdout)

[ INFO    ]: 10:32:50 | Cleaning up InstallImageBuilder instance

[ INFO    ]: 10:32:50 | Cleaning up BootImageDracut instance

[Santosh]

and I have dracut installed in the system:

 dracut --version

getopt: unrecognized option '--version'

Usage: /usr/bin/dracut [OPTION]... [<initramfs> [<kernel-version>]]

Version: 037-17.6.1

What am I missing here?

thanks

[Marcus Schäfer]

Hi,

> Thanks. I updated the python3-kiwi package and included dracut package
> in config.kiwi both ways (not at the same time):
> <package name="dracut"/>
> <package name="dracut" bootinclude="true"/>
> and added initrd_system="dracut" in <type ...>
> but appliance creation keeps failing with this error:
> [ DEBUG ]: 10:32:50 | EXEC: Failed with stderr: chroot: failed to run
> command ‘dracut’: No such file or directory
> , stdout: (no output on stdout)
> [ ERROR ]: 10:32:50 | KiwiCommandError: chroot: stderr: chroot:
> failed to run command ‘dracut’: No such file or directory

I have done several fixes in that area, python3-kiwi v8.18.3 should
have this problem fixed. If you are using that version and still see
the issue, could you send me the complete build log file ?

Thanks

[Santosh]

I see it now. So I was using this repo before:

http://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_13.2/

but I don't see openSUSE_13.2 now..

 Factory/                                   20-Jun-2016 09:51    -   
 SLE_12_SP2/                                20-Jun-2016 09:51    -   
 openSUSE_Factory_ARM/                      20-Jun-2016 09:55    -   
 openSUSE_Leap_42.1/                        20-Jun-2016 09:51    -   
 openSUSE_Tumbleweed/                       20-Jun-2016 09:49    -   

 What option do I have?

# cat /etc/*rel*

openSUSE 13.2 (x86_64)

VERSION = 13.2

CODENAME = Harlequin

# /etc/SuSE-release is deprecated and will be removed in the future, use /etc/os-release instead

NAME=openSUSE

VERSION="13.2 (Harlequin)"

VERSION_ID="13.2"

PRETTY_NAME="openSUSE 13.2 (Harlequin) (x86_64)"

ID=opensuse

ANSI_COLOR="0;32"

CPE_NAME="cpe:/o:opensuse:opensuse:13.2"

BUG_REPORT_URL="https://bugs.opensuse.org"

HOME_URL="https://opensuse.org/"

ID_LIKE="suse"

[Marcus Schäfer]

Hi,

> I see it now. So I was using this repo before:
> http://download.opensuse.org/repositories/Virtualization:/Appliances:/B
> uilder/openSUSE_13.2/
> but I don't see openSUSE_13.2 now..

The openSUSE_13.2 packages are still in building state, should be
done soon

However 13.2 will be gone with Leap 42.2 coming soon.

Regards,
Marcus
--
Public Key available via: https://keybase.io

[Marcus Schäfer]

Hi,

> > but I don't see openSUSE_13.2 now..
>
> The openSUSE_13.2 packages are still in building state, should be
> done soon

Ready now

http://download.opensuse.org/repositories/Virtualization:/Appliances:/Builder/openSUSE_13.2/

[Santosh]

My laptop had issues so couldn't test it earlier.

fails with this error:

[ DEBUG   ]: 14:33:01 | EXEC: [mksquashfs /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_install_squashfs.xz131dtj /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/enterprise_msm.raw.squashfs -noappend -comp xz]
[ DEBUG   ]: 14:35:09 | EXEC: [mv /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/enterprise_msm.raw.squashfs /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_install_media._wdlnzh4]
[ INFO    ]: 14:35:09 | Setting up install image bootloader configuration
[ WARNING ]: 14:35:09 | efi firmware needs a root device but no uuid was given
[ DEBUG   ]: 14:35:09 | EXEC: [mkdir -p /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_install_media._wdlnzh4/boot/x86_64/loader]
[ DEBUG   ]: 14:35:09 | EXEC: [rsync -z -a /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader/ /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_install_media._wdlnzh4/boot/x86_64/loader]
[ DEBUG   ]: 14:35:09 | EXEC: Failed with stderr: rsync: change_dir "/root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

, stdout: (no output on stdout)

[ ERROR   ]: 14:35:09 | KiwiCommandError: rsync: stderr: rsync: change_dir "/root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/output/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

, stdout: (no output on stdout)

[ INFO    ]: 14:35:09 | Cleaning up InstallImageBuilder instance

[Marcus Schäfer]

Hi,

> [ DEBUG ]: 14:35:09 | EXEC: [rsync -z -a
> /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/o
> utput/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader/
> /root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/o
> utput/oem/msm_hardware/kiwi_install_media._wdlnzh4/boot/x86_64/loader]
> [ DEBUG ]: 14:35:09 | EXEC: Failed with stderr: rsync: change_dir
> "/root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/
> output/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader" failed:
> No such file or directory (2)
> rsync error: some files/attrs were not transferred (see previous
> errors) (code 23) at main.c(1165) [sender=3.1.1]
> , stdout: (no output on stdout)
> [ ERROR ]: 14:35:09 | KiwiCommandError: rsync: stderr: rsync:
> change_dir
> "/root/santosh-kiwi8-conversion-final_06_13/appliance/target/appliance/
> output/oem/msm_hardware/kiwi_boot_root.7ewlslf1/image/loader" failed:
> No such file or directory (2)
> rsync error: some files/attrs were not transferred (see previous
> errors) (code 23) at main.c(1165) [sender=3.1.1]
> , stdout: (no output on stdout)

The install media builder in combination with initrd_system="dracut"
has the bug that it also tries to use the dracut initrd for the
installation, but for the installation the kiwi install code should
be used. I'm going to fix this

Stay tuned :)

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc

[Marcus Schäfer]

Hi,

> The install media builder in combination with initrd_system="dracut"
> has the bug that it also tries to use the dracut initrd for the
> installation, but for the installation the kiwi install code should
> be used. I'm going to fix this

done now, 8.18.5 should be your friend

[Santosh]

great. Waiting on the version to become available. Thanks again for your super quick response.

[Santosh]

yey. Image built fine. Installation went well and system came up fine.

Thanks