Can't build kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS as OEM

[Roger Oberholtzer]

I have this kiwi installed on an up-to-date Leap 42.3:

kiwi-man-pages-9.11.2-44.1.x86_64
kiwi-tools-9.11.2-44.1.x86_64
python2-kiwi-9.11.2-44.1.x86_64

I just taken a copy of the kiwi-descriptions.

When I run this in kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS:

mkdir ./image
kiwi-ng --type=oem --debug system build --description=. --target-dir=./image

It fails:


[ INFO ]: 12:34:36 | Creating ISO filesystem
[ DEBUG ]: 12:34:36 | EXEC: [isolinux-config --base
boot/x86_64/loader
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_install_media.YnP04b/boot/x86_64/loader/isolinux.bin]
[ DEBUG ]: 12:34:37 | EXEC: [/usr/bin/mkisofs -V "KIWI Installation
System" -A 0x81ac2604 -R -J -f -pad -joliet-long -sort
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/tmp/tmpe72jl_
-no-emul-boot -boot-load-size 4 -boot-info-table -hide
boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_install_media.YnP04b]
[ DEBUG ]: 12:34:38 | EXEC: [/usr/bin/isoinfo -R -l -i
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso]
[ DEBUG ]: 12:34:38 | Found ISO header_end id at offset:
[ DEBUG ]: 12:34:38 | --> 2k blocks: 7797
[ DEBUG ]: 12:34:38 | --> 512 byte blocks(isohybrid): 31188
[ DEBUG ]: 12:34:38 | --> bytes(loop mount): 15968256
[ DEBUG ]: 12:34:38 | EXEC: [/usr/bin/mkisofs -hide header_end
-hide-joliet header_end -V "KIWI Installation System" -A 0x81ac2604 -R
-J -f -pad -joliet-long -sort
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/tmp/tmpe72jl_
-no-emul-boot -boot-load-size 4 -boot-info-table -hide
boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_install_media.YnP04b]
[ DEBUG ]: 12:34:43 | EXEC: [isohybrid --offset 31188 --id
0x81ac2604 --type 0x83 --uefi
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso]
[ ERROR ]: 12:34:44 | KiwiCommandError: isohybrid:
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso:
warning: unable to find efi image

[ INFO ]: 12:34:44 | Cleaning up InstallImageBuilder instance
[ DEBUG ]: 12:34:44 | EXEC: [rm -r -f
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_install_media.YnP04b]
[ DEBUG ]: 12:34:45 | EXEC: [rm -r -f
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_install_squashfs.p7JwTN]
[ INFO ]: 12:34:45 | Cleaning up BootImageKiwi instance
[ DEBUG ]: 12:34:45 | EXEC: [rm -r -f
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_boot_root.vVaTAT]
[ DEBUG ]: 12:34:45 | EXEC: [rm -r -f
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/tmp/kiwi_boot_root_copy.ATDU6f]
[ INFO ]: 12:34:47 | Cleaning up BootImageKiwi instance
[ DEBUG ]: 12:34:47 | EXEC: [rm -r -f
/home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/kiwi_boot_root.te0M_Y]
[ INFO ]: 12:34:47 | Cleaning up BootImageDracut instance


I am trying to discover why my own build cannot be deployed (a
parallel discussion). So I thought I would (once again) start with a
sample image. Once that builds and deploys, I can add my bits to see
what causes the problem. But the sample must work before I can do all
that.

--
Roger Oberholtzer

[Roger Oberholtzer]

On Wed, Sep 13, 2017 at 12:48 PM, Roger Oberholtzer
<roger.ob...@gmail.com> wrote:

> [ ERROR ]: 12:34:44 | KiwiCommandError: isohybrid:
> /home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso:
> warning: unable to find efi image

Even though the ERROR above was given during the build, an ISO
installer was created. I tried that, and the results are the same as
in my other post about my own OEM:

It installs the image, modifies the partitions, does a dracut, boots
the system, and gets to the place where it starts Plymouth. Then
nothing.

So it is not just my OEM spec that is perhaps the problem.

Could it be something in kiwi-ng? Can I run a different (i.e., older
non-ng) kiwi with the same spec file?

--
Roger Oberholtzer

[Roger Oberholtzer]

I dd's the raw image to the qemu disk, and that seemed to work. The
JeOS sample does not do so much. But I think it is all there. The root
password was 'linux'.

qemu-img create target_disk 20g
dd if=LimeJeOS-Leap-42.3.x86_64-1.42.3.raw of=target_disk conv=notrunc
qemu-system-x86_64 -drive
file=target_disk,index=0,media=disk,format=raw -m 4096

The image was not extended or a dracut image made (which the installer
does before starting the installed image). So perhaps when the
installer does this something goes wrong?


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> [ DEBUG ]: 12:34:43 | EXEC: [isohybrid --offset 31188 --id
> 0x81ac2604 --type 0x83 --uefi
> /home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso]
> [ ERROR ]: 12:34:44 | KiwiCommandError: isohybrid:
> /home/roger/source/RSofT_Diskless/kiwi/kiwi-descriptions/suse/x86_64/suse-leap-42.3-JeOS/image/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso:
> warning: unable to find efi image

This fails because you are using mkisofs which does not produce
an ioshybrid capable iso with regards to the embedded efi loader.
I had that problem on my machine too and solved it with:

rpm -e mkisofs
zypper in cdrkit-cdrtools-compat

The /usr/bin/mkisofs provided by the compat tools works nicely

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Roger Oberholtzer]

I also had to remove:

k3b-lang
k3b
dvd+rw-tools
zisofs-tools
cdrecord

I am building an installer now. It takes so much time on my system.
Even with the RPMs cached...


--
Roger Oberholtzer

[Roger Oberholtzer]

After installing cdrkit-cdrtools-compat the build log does look a bit
different. But I still get the Error.


[ DEBUG ]: 16:24:46 | EXEC: [isolinux-config --base
boot/x86_64/loader
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/kiwi_install_media.MdmZGI/boot/x86_64/loader/isolinux.bin]
[ DEBUG ]: 16:24:46 | EXEC: [/usr/bin/mkisofs -V "KIWI Installation
System" -A 0x2e2f230c -R -J -f -pad -joliet-long -sort
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/tmpq6m0UG

-no-emul-boot -boot-load-size 4 -boot-info-table -hide
boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o

/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/kiwi_install_media.MdmZGI]
[ DEBUG ]: 16:25:00 | EXEC: [/usr/bin/isoinfo -R -l -i
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso]
[ DEBUG ]: 16:25:01 | Found ISO header_end id at offset:
[ DEBUG ]: 16:25:01 | --> 2k blocks: 7794
[ DEBUG ]: 16:25:01 | --> 512 byte blocks(isohybrid): 31176
[ DEBUG ]: 16:25:01 | --> bytes(loop mount): 15962112
[ DEBUG ]: 16:25:01 | EXEC: [/usr/bin/mkisofs -hide header_end
-hide-joliet header_end -V "KIWI Installation System" -A 0x2e2f230c -R

-J -f -pad -joliet-long -sort

/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/tmpq6m0UG

-no-emul-boot -boot-load-size 4 -boot-info-table -hide
boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o

/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/kiwi_install_media.MdmZGI]
[ DEBUG ]: 16:25:18 | Relocated boot catalog from sector 0x3ce5 to 0x14
[ DEBUG ]: 16:25:18 | EXEC: [isohybrid --offset 31176 --id
0x2e2f230c --type 0x83 --uefi
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso]
[ ERROR ]: 16:25:20 | KiwiCommandError: isohybrid:
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso:

warning: unable to find efi image

isohybrid: Warning: more than 1024 cylinders: 1826
isohybrid: Not all BIOSes will be able to boot this device

When I use the raw image in qemu), I get to a character login. It does
not accept my (or any I can think of) root password. I expected the
first boot YasT stuff to run, asking things like location, root
password and such. But that does not happen. So at least now the boot
does not freeze. But I cannot seem to do anything...

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> After installing cdrkit-cdrtools-compat the build log does look a bit
> different. But I still get the Error.
>

> [ ERROR ]: 16:25:20 | KiwiCommandError: isohybrid:
> /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso:
> warning: unable to find efi image
> isohybrid: Warning: more than 1024 cylinders: 1826
> isohybrid: Not all BIOSes will be able to boot this device

Pretty strange I can't reproduce any of these on my Leap42.2 build system
But I guess the hybrid problem is because this iso seems to be quite big ?
how big is it ? which iso-level is used in the mkisofs call (from the log)

I use:

cdrkit-cdrtools-compat-1.1.11-22.28.x86_64
syslinux-4.04-36.17.x86_64

> When I use the raw image in qemu), I get to a character login. It does
> not accept my (or any I can think of) root password. I expected the
> first boot YasT stuff to run, asking things like location, root
> password and such. But that does not happen. So at least now the boot
> does not freeze. But I cannot seem to do anything...

I can only imagine your image setup invalidates the root password
setup from kiwi somewhere down the line. You are saying this image
is configured to call some yast stuff at first boot ?

Have you tried if you can login when there is no other player in the game ?

[Roger Oberholtzer]

On Wed, Sep 13, 2017 at 7:50 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> After installing cdrkit-cdrtools-compat the build log does look a bit
>> different. But I still get the Error.
>>
>> [ ERROR ]: 16:25:20 | KiwiCommandError: isohybrid:
>> /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSofT-42.3-oem.x86_64-42.3.2.install.iso:
>> warning: unable to find efi image
>> isohybrid: Warning: more than 1024 cylinders: 1826
>> isohybrid: Not all BIOSes will be able to boot this device
>
> Pretty strange I can't reproduce any of these on my Leap42.2 build system
> But I guess the hybrid problem is because this iso seems to be quite big ?
> how big is it ? which iso-level is used in the mkisofs call (from the log)

The files are:

9962520576 RSL-OEM-Leap-42.3.x86_64-42.3.2.raw
1914699776 RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso

There the verification to the install drive does not produce an error.

The commands that are getting run are:

EXEC: [/usr/bin/mkisofs -hide header_end -hide-joliet header_end -V

"KIWI Installation System" -A 0x3d2abd97 -R -J -f -pad -joliet-long
-sort /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/tmpo3uPpV

-no-emul-boot -boot-load-size 4 -boot-info-table -hide
boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o

/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/kiwi_install_media.gjM3XV]

Relocated boot catalog from sector 0x3ce5 to 0x14

EXEC: [isohybrid --offset 31176 --id 0x3d2abd97 --type 0x83 --uefi
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]


> I use:
>
> cdrkit-cdrtools-compat-1.1.11-22.28.x86_64
> syslinux-4.04-36.17.x86_64

On Leap 42.3:

cdrkit-cdrtools-compat-1.1.11-24.16.x86_64
syslinux-4.04-38.17.x86_64


I get the exact same error when I build
kiwi-descriptions/suse/x86_64/LimeJeOS-Leap-42.3

And that is a smaller image. So I do not think size is causing this problem.

>> When I use the raw image in qemu), I get to a character login. It does
>> not accept my (or any I can think of) root password. I expected the
>> first boot YasT stuff to run, asking things like location, root
>> password and such. But that does not happen. So at least now the boot
>> does not freeze. But I cannot seem to do anything...
>
> I can only imagine your image setup invalidates the root password
> setup from kiwi somewhere down the line. You are saying this image
> is configured to call some yast stuff at first boot ?

I have a config-yast-firstboot.xml in the --description= directory
(not the root directory that is in the same directory). This is how I
have always done this. I looked in the kiwi ng docs to see if
something had changed. But I did not see mention of this file.

> Have you tried if you can login when there is no other player in the game ?

I can log in to the LimeJeOS-Leap-42.3 build. Not to my own.





--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> The commands that are getting run are:
>
> EXEC: [/usr/bin/mkisofs -hide header_end -hide-joliet header_end -V
> "KIWI Installation System" -A 0x3d2abd97 -R -J -f -pad -joliet-long
> -sort /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/tmpo3uPpV
> -no-emul-boot -boot-load-size 4 -boot-info-table -hide
> boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b
> boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
> -eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
> -boot-load-size 30720 -o

And from here you see both loaders "-b boot/x86_64/loader/isolinux.bin"
and "-b boot/x86_64/efi" are embedded whereas the efi loader is an
alternative one to the primary syslinux loader, all good to my
understanding.

> EXEC: [isohybrid --offset 31176 --id 0x3d2abd97 --type 0x83 --uefi
> /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]

Which firmware setup do you have in your image firmware="efi" or "uefi" ?

> > I use:
> >
> > cdrkit-cdrtools-compat-1.1.11-22.28.x86_64
> > syslinux-4.04-36.17.x86_64
>
> On Leap 42.3:
>
> cdrkit-cdrtools-compat-1.1.11-24.16.x86_64
> syslinux-4.04-38.17.x86_64
>
> I get the exact same error when I build
> kiwi-descriptions/suse/x86_64/LimeJeOS-Leap-42.3

yes that should be consistent because mkisofs and isohybrid are called
from the host. If possible from a dependency point of view you can try
to downgrade cdrkit-cdrtools-compat to the one from Leap42.2 and test
if that makes a difference.

Alternatively you can send along your image description and I'll
take a look

> And that is a smaller image. So I do not think size is causing this problem.

agreed

> I have a config-yast-firstboot.xml in the --description= directory
> (not the root directory that is in the same directory). This is how I
> have always done this. I looked in the kiwi ng docs to see if
> something had changed. But I did not see mention of this file.

yast has changed the way how they support a firstboot workflow so that
it was hard for us to provide a compatible and working system. Thus the
current way to do use is as follows

1. Add the package to your XML description

<package name="yast2-firstboot"/>

2. In cour config.sh script activate the service with the line:

# activate yast2 firstboot system
touch /var/lib/YaST2/reconfig_system

3. If you need a customized workflow as I think you do because you
have a custom config-yast-firstboot.xml, overwrite yast2-firstboot
provided firstboot.xml file with yours by adding the following
overlay file in your image description

root/etc/YaST2/firstboot.xml

Before you do that I would give it a test with the original
file because the format might have changed

All this applies to the information I have from the current
yast2-firstboot docs in:

file:///usr/share/doc/packages/yast2-firstboot/index.html

In case this does not work please contact the yast team

> > Have you tried if you can login when there is no other player in the game ?
>
> I can log in to the LimeJeOS-Leap-42.3 build. Not to my own.

On one hand that's good news, on the other I have no clue why you
can't login to the system. Have you tried to just copy and paste
the password hash from the suse-leap-42.3-JeOS/config.xml to your
XML description and see if you can login to the image after rebuild ?
root:linux. If that would work something is broken in the password
hash you created, if you still can't login something else in your
image description causes a problem

[Roger Oberholtzer]

On Thu, Sep 14, 2017 at 9:19 AM, Marcus Schäfer <m...@suse.de> wrote:

> Which firmware setup do you have in your image firmware="efi" or "uefi" ?

I have:

<type image="oem" filesystem="ext4" boot="oemboot/suse-leap42.3"
installiso="true" bootloader="grub2" kernelcmdline="splash"
firmware="efi" initrd_system="dracut">

> yast has changed the way how they support a firstboot workflow so that
> it was hard for us to provide a compatible and working system. Thus the
> current way to do use is as follows
>
> 1. Add the package to your XML description
>
> <package name="yast2-firstboot"/>

This I already had.

I made the changes and my firstboot stuff started. So some progress.

For some reason YaST was in character mode. This is in qemu. Maybe I
missed installing something so I do not get graphics here. The boot
stuff was graphic.

When setting the timezone, it complained that ntp was not installed. I
will add that.

It then failed when init_hostname was called. I am guessing that my
firstboot.xml needs updating. Until I sort this out, I don't get to a
login.

The build-time Error message is still there. But until I get the boot
stuff finished, I cannot say if it is problematic.


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> On Thu, Sep 14, 2017 at 9:19 AM, Marcus Schäfer <m...@suse.de> wrote:
>
> > Which firmware setup do you have in your image firmware="efi" or "uefi" ?
>
> I have:
>
> <type image="oem" filesystem="ext4" boot="oemboot/suse-leap42.3"
> installiso="true" bootloader="grub2" kernelcmdline="splash"
> firmware="efi" initrd_system="dracut">

ok simple efi. I have built a test oem image to see if I can reproduce
the isohybrid error but on my Leap42.2 and also in my dice(current tumbleweed)
container no such error is reported... strange

DEBUG: Found ISO header_end id at offset:
DEBUG: --> 2k blocks: 7794
DEBUG: --> 512 byte blocks(isohybrid): 31176
DEBUG: --> bytes(loop mount): 15962112
DEBUG: EXEC: [/usr/bin/mkisofs -hide header_end -hide-joliet header_end -V "KIWI Installation System" -A 0x4dd423cd -R -J -f -pad -joliet-long -sort /tmp/tmp6x8026bt -no-emul-boot -boot-load-size 4 -boot-info-table -hide boot/x86_64/boot.catalog -hide-joliet boot/x86_64/boot.catalog -b boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog -eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long -boot-load-size 30720 -o /tmp/mytest/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso /tmp/mytest/kiwi_install_media.enriccow]
DEBUG: Relocated boot catalog from sector 0x3ce5 to 0x14
DEBUG: Fixed iso catalog contents
DEBUG: EXEC: [isohybrid --offset 31176 --id 0x4dd423cd --type 0x83 --uefi /tmp/mytest/LimeJeOS-Leap-42.3.x86_64-1.42.3.install.iso]
INFO: Cleaning up InstallImageBuilder instance

> I made the changes and my firstboot stuff started. So some progress.

Yay

> For some reason YaST was in character mode. This is in qemu. Maybe I
> missed installing something so I do not get graphics here. The boot
> stuff was graphic.

Probably yes, if things are still designed in the way when I was working
in the yast team you need all of X (pattern x11) and a framebuffer server
xf86-video-fbdev

> The build-time Error message is still there. But until I get the boot
> stuff finished, I cannot say if it is problematic.

you mean the error from isohybrid ? if so I wonder how you were able
to build the image because I treat that warning messages from isohybrid
as an error and exit the build. Which version of kiwi do you currently
use ?

[Roger Oberholtzer]

On Thu, Sep 14, 2017 at 4:00 PM, Marcus Schäfer <m...@suse.de> wrote:

>> For some reason YaST was in character mode. This is in qemu. Maybe I
>> missed installing something so I do not get graphics here. The boot
>> stuff was graphic.
>
> Probably yes, if things are still designed in the way when I was working
> in the yast team you need all of X (pattern x11) and a framebuffer server
> xf86-video-fbdev

I see that I commented out the older <opensusePattern> directives. I
will now replace them with <package name="patterns-openSUSE-ZZZ"/>
calls.

>> The build-time Error message is still there. But until I get the boot
>> stuff finished, I cannot say if it is problematic.
>
> you mean the error from isohybrid ? if so I wonder how you were able
> to build the image because I treat that warning messages from isohybrid
> as an error and exit the build. Which version of kiwi do you currently
> use ?

kiwi-man-pages-9.11.2-44.1.x86_64
kiwi-tools-9.11.2-44.1.x86_64
python2-kiwi-9.11.2-44.1.x86_64


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> > you mean the error from isohybrid ? if so I wonder how you were able
> > to build the image because I treat that warning messages from isohybrid
> > as an error and exit the build. Which version of kiwi do you currently
> > use ?
>
> kiwi-man-pages-9.11.2-44.1.x86_64
> kiwi-tools-9.11.2-44.1.x86_64
> python2-kiwi-9.11.2-44.1.x86_64

ok be aware if you move to 9.11.3 any isohybrid warning will be treated
as an error and stops the build. That's because if efi is requested and
the build succeeded people will expect the iso to be able to boot via
efi which is not the case if isohybrid was not able to find the efi
loader.

In that situation you can either switch off efi (no firmware="efi")
or we find why mkisofs and isohybrid on your build host does not play
well together

[David Cassany]

Hi,

>
> In that situation you can either switch off efi (no firmware="efi")
> or we find why mkisofs and isohybrid on your build host does not play
> well together
>

I just want to add my two cents regarding the mkisofs and isohybrid
relationtship. In my host (Leap 42.2) mkisofs package do not play well
with isohybrid, but genisoimage it does. KIWI looks for mkisofs or
genisoimage in your system and uses genisoimage if no mkisofs is
present. Also note that in many systems the mkisofs is just a
genisoimage symlink provided by cdrkit-cdrtools-compat. So when I want
to use mkisofs or genisoimage in my host I just make a *temporary*
change like mv /usr/bin/mkisofs /usr/bin/mkisofs.orig, this way KIWI is
forced to use genisoimage tool, you'll see it the build log. Genisoimage
should be there since it is a requirement of KIWI. Using genisoimage my
builds work fine. Maybe we should consider the possibility of definitely
not running mkisofs in KIWI...

Regards,
David

[Roger Oberholtzer]

On my Leap 42.3 with cdrkit-cdrtools-compat-1.1.11-24.16.x86_64,
mkisofs is a link to genisoimage:


lrwxrwxrwx 1 root root 11 Sep 13 15:12 /usr/bin/mkisofs -> genisoimage

So as long as the command line options are correct when mkisofs is
really genisoimage, I guess this is in place.

For my next build (they take sooo long...) I will hide mkisofs and see
what happens.

As to having efi for the ISO install image, I really do not know
what's best. The ISO is expected to run everywhere.


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I just want to add my two cents regarding the mkisofs and isohybrid
> relationtship. In my host (Leap 42.2) mkisofs package do not play well
> with isohybrid, but genisoimage it does. KIWI looks for mkisofs or
> genisoimage in your system and uses genisoimage if no mkisofs is
> present. Also note that in many systems the mkisofs is just a
> genisoimage symlink provided by cdrkit-cdrtools-compat. So when I want
> to use mkisofs or genisoimage in my host I just make a *temporary*
> change like mv /usr/bin/mkisofs /usr/bin/mkisofs.orig, this way KIWI is
> forced to use genisoimage tool, you'll see it the build log. Genisoimage
> should be there since it is a requirement of KIWI. Using genisoimage my
> builds work fine. Maybe we should consider the possibility of definitely
> not running mkisofs in KIWI...

Thanks for the update. iirc Robert should have that situation on his
host since he deleted mkisofs and installed the compat kit which should
point to genisoimage

This is a never ending nightmare with mkisofs
vs. genisoimage. The last time I talked to the people who have more
insight I was told mkisofs produces the more standard correct results
and the genisoimage project is more or less on hold. That's why kiwi
prefer mkisofs. As it is quite easy to symlink for whatever version
people prefer I'm not much motivated to change kiwi again with the
way it selects the tool.

As I think you can't make sure genisoimage does it right all the time
a ban for mkisofs in kiwi seems not to be a problem solver to me ;)

[David Cassany]

Hi,

> and the genisoimage project is more or less on hold. That's why kiwi
> prefer mkisofs. As it is quite easy to symlink for whatever version
> people prefer I'm not much motivated to change kiwi again with the
> way it selects the tool.
>

I see and agree, just that it is quite strange to me that neither
mkisofs or cdrkit-cdrtools-compat are requirements of kiwi, however
genisoimage is, but we prefer mkisofs. Also not sure there is
cdrkit-cdrtools-compat in tumbleweed.

> As I think you can't make sure genisoimage does it right all the time
> a ban for mkisofs in kiwi seems not to be a problem solver to me ;)

Yes, I am afraid you are completely right, there is no guarantee that
moving from mkisofs to genisoimage or vice-versa will solve this issue
for most distros and maintained over the time...

Regards,
David

[Roger Oberholtzer]

The saga continues:

1. It made no difference if I hid mkisofs. I still get the error
message. But the .iso and .raw images are created. Of course, maybe
they are not 100% correct.

2. Although YaST firstboot now starts, I still get the character
version. I think all needed parts are present:

<package name="patterns-openSUSE-x11"/>
<package name="patterns-openSUSE-fonts"/>
<package name="yast2-control-center-qt"/>

3. Although one may think the following are YaSY firstboot issues, I'm
not sure. I have used my firstboot script for quite a while. It is
nothing fancy. I suspect the error in item one...

It wants to write /etc/ntp.conf. But is complains that it cannot.

I can select the keyboard. But there are more things firstboot
should do. But the system shuts down instead. It is a proper shutdown.
Not a crash.

If I boot the installed image again, I am once again in
firstboot. I once again get the complaint about /etc/ntp.conf. But the
keyboard I selected the first time is remembered. And then it shuts
down again.


This is not really so very different of an OEM image than I have built
in the past. So I cannot see what the problem could be. Here is my
definition:

<image schemaversion="6.7" name="RSL-OEM-Leap-42.3"
displayname="RambollStandardLinux">

<preferences>

<type image="oem" filesystem="ext4"
boot="oemboot/suse-leap42.3" installiso="true" bootloader="grub2"
kernelcmdline="splash" firmware="efi" initrd_system="dracut">

<oemconfig>
<oem-systemsize>49152</oem-systemsize>
<oem-boot-title>Ramboll Standard Linux
42.3</oem-boot-title>
<oem-swap>true</oem-swap>
<oem-recovery>false</oem-recovery>
<oem-device-filter>/dev/ram</oem-device-filter>
<!--
oem-partition-install>true</oem-partition-install -->
</oemconfig>
<!--
<systemdisk name="homeLV">
<volume name="home" freespace="all"
mountpoint="home"/>
</systemdisk>
-->
</type>
<version>42.3.2</version>
<packagemanager>zypper</packagemanager>
<rpm-check-signatures>false</rpm-check-signatures>
<!--
<rpm-force>true</rpm-force>
-->
<locale>en_US</locale>
<keytable>us.map.gz</keytable>
<hwclock>utc</hwclock>
<timezone>Europe/Stockholm</timezone>

<bootsplash-theme>openSUSE</bootsplash-theme>
<bootloader-theme>openSUSE</bootloader-theme>
</preferences>

Then my repositories and packages. And then:

<packages type="bootstrap">
<package name="udev"/>
<package name="filesystem"/>
<package name="glibc-locale"/>
<package name="cracklib-dict-full"/>
<package name="ca-certificates"/>
<package name="openSUSE-release"/>
</packages>

</image>

The parts that are commented out cause the build to fail.

One curious thing: swap is in /dev/sda3, and / is in /dev/sda4. I
don't know what is in /dev/sda2. /dev/sda1 is the boot stuff, I guess.

--
Roger Oberholtzer

[Roger Oberholtzer]

On Fri, Sep 15, 2017 at 8:33 AM, Roger Oberholtzer
<roger.ob...@gmail.com> wrote:

> One curious thing: swap is in /dev/sda3, and / is in /dev/sda4. I
> don't know what is in /dev/sda2. /dev/sda1 is the boot stuff, I guess.

Oops. I got that wrong. This is what is made in 'system create':

/dev/sda1 - EFI CSM (legacy BIOS)
/dev/sda2 - EFI partition (fat16)
/dev/sda3 - swap
/dev/sda4 - / (ext4)

And there are 4 partitions seen when booting. And they seem to be used
as described.



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> 1. It made no difference if I hid mkisofs. I still get the error
> message. But the .iso and .raw images are created. Of course, maybe
> they are not 100% correct.

If isohybrid could not find the efi loader it will prevent the
iso from being bootable via efi. You will see that if you try:

qemu -bios /usr/share/qemu/ovmf-x86_64.bin \
-cdrom your.install.iso -vga cirrus

This will not boot, all the rest will.

> 2. Although YaST firstboot now starts, I still get the character
> version. I think all needed parts are present:

I think if you send that feedback to the yast people they will
be able to help you. Apart from that maybe systemd-firstboot
is an alternative. Just stumbled over this article:

https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html

> The parts that are commented out cause the build to fail.

Your description looked good to me. I have build the appliance (without
your package list, as this was unknown) and also enabled the lvm setup
which builds on my system. But again I have disabled lvmetad which
causes trouble for lvm when enabled.

I could not find any users information, which would explain why
you can't login. So just added one from the template:

<users>
<user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
</users>

and could login with root:linux

[Roger Oberholtzer]

On Fri, Sep 15, 2017 at 10:24 AM, Marcus Schäfer <m...@suse.de> wrote:

Hi,

> 1. It made no difference if I hid mkisofs. I still get the error
> message. But the .iso and .raw images are created. Of course, maybe
> they are not 100% correct.

If isohybrid could not find the efi loader it will prevent the
iso from being bootable via efi. You will see that if you try:

   qemu -bios /usr/share/qemu/ovmf-x86_64.bin \
        -cdrom your.install.iso -vga cirrus

This will not boot, all the rest will.

The attached image shows how the disk is modified after the OEM image is copied to it. There are some messages that look suspect. At least to me.

I am curious where partition 4 (swap) is after partition 3 is extended to fill the disk. This is how the build said it set up the disk:


DEBUG: Initialize gpt disk
DEBUG: EXEC: [sgdisk --zap-all /dev/loop0]
INFO: --> creating EFI CSM(legacy bios) partition
DEBUG: EXEC: [sgdisk -n 1:0:+2M -c 1:p.legacy /dev/loop0]
DEBUG: EXEC: [sgdisk -t 1:EF02 /dev/loop0]
INFO: --> creating EFI partition
DEBUG: EXEC: [sgdisk -n 2:0:+20M -c 2:p.UEFI /dev/loop0]
DEBUG: EXEC: [sgdisk -t 2:EF00 /dev/loop0]
INFO: --> creating root partition
DEBUG: EXEC: [sgdisk -n 3:0:0 -c 3:p.lxroot /dev/loop0]
DEBUG: EXEC: [sgdisk -t 3:8300 /dev/loop0]
DEBUG: EXEC: [kpartx -s -a /dev/loop0]
INFO: Creating EFI(fat16) filesystem on /dev/mapper/loop0p2
DEBUG: EXEC: [mkdosfs -F16 -I -n EFI /dev/mapper/loop0p2]
INFO: Creating root(ext4) filesystem on /dev/mapper/loop0p3
DEBUG: EXEC: [mkfs.ext4 -L ROOT /dev/mapper/loop0p3]

> 2. Although YaST firstboot now starts, I still get the character
> version. I think all needed parts are present:

I think if you send that feedback to the yast people they will
be able to help you. Apart from that maybe systemd-firstboot
is an alternative. Just stumbled over this article:

    https://www.freedesktop.org/software/systemd/man/systemd-firstboot.html

I will look at this. Thanks for the pointer.
 

> The parts that are commented out cause the build to fail.

Your description looked good to me. I have build the appliance (without
your package list, as this was unknown) and also enabled the lvm setup
which builds on my system. But again I have disabled lvmetad which
causes trouble for lvm when enabled.

I have been saving this until I have the OEM working. One step at a time...
 

I could not find any users information, which would explain why
you can't login. So just added one from the template:

    <users>
        <user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
    </users>

I do have this. I just did not include it in my listing to you:


         <users profiles="root">
                <!-- user password="$1$xyz,$/IWx83kxixDFMZNdaGR5y0" home="/root" name="root" groups="root"/ -->

                <user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
        </users>

As you see, I am using your password. But after adding firstboot, I no longer get to a boot prompt...

--

Roger Oberholtzer

[Roger Oberholtzer]

On Fri, Sep 15, 2017 at 10:24 AM, Marcus Schäfer <m...@suse.de> wrote:

 

I could not find any users information, which would explain why
you can't login. So just added one from the template:

    <users>
        <user password="$1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0" home="/root" name="root" groups="root"/>
    </users>

and could login with root:linux

I found a way to get to a virtual terminal in qemu using sendkey. This is during firstboot. I get a login prompt. But root:linux does not work. It should as I have that in my spec file.

This is very odd.

 

--

Roger Oberholtzer

[Roger Oberholtzer]

The image set up in qemu looks like this:

Disk target_disk: 60 GiB, 64424509440 bytes, 125829120 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3695A76E-AAD5-4D49-92FD-B10495C7373C

Device           Start       End   Sectors  Size Type
target_disk1      2048      6143      4096    2M BIOS boot
target_disk2      6144     47103     40960   20M EFI System
target_disk3     47104 100727550 100680447   48G Microsoft basic data
target_disk4 100728832 116937135  16208304  7.7G Microsoft basic data

Interesting that the Type is "Microsoft basic data.". Maybe that is related to EFI?

I can mount the root partition from the image.

Shouldn't /etc/shadow contain the passwords? I see this for root:

root::17423::::::

Or is it that the kiwi password is used in the installer image, and not the OEM image? Usually on firstboot I set the root password. So if that was always the case, I would not have noticed. But since firstboot is failing this is not happening.

The date on /etc/shadow is from when the system was built. It has not been modified since then.

I have joined the yast+devel mailing list and posted a question. I do not think systemd-firstboot is the way to go. Seems it gets triggered if /etc is empty. I will continue with YaST firstboot.

--

Roger Oberholtzer

[David Cassany]

Hi,

>
> I have joined the yast+devel mailing list and posted a question. I do not
> think systemd-firstboot is the way to go. Seems it gets triggered if /etc
> is empty. I will continue with YaST firstboot.
>

Just in case you want to give a systemd-firstboot a try, it used to be
triggered by deleting /etc/machine-id and /var/lib/dbus/machine-id which
are created during the systemd and dbus install at build time (thus you
can remove them in config.sh). If none of these files exist
systemd-firstboot is triggered, there is no need to leave /etc/ empty,
at least it used to work this way with Leap 42.2 and I am not aware of
any change.

Regards,
David

[Roger Oberholtzer]

But what does that then run? I looked at the docs and it seemed to be
a command line where one set things. Which could be useful. But I
think YaST provides an interface to obtain values. Or?

> --
> You received this message because you are subscribed to the Google Groups "kiwi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kiwi-images...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Roger Oberholtzer

[David Cassany]

Hi,

> But what does that then run? I looked at the docs and it seemed to be
> a command line where one set things. Which could be useful. But I
> think YaST provides an interface to obtain values. Or?
>

Yes systemd-firstboot is just asking to the user some parameters at the
command line, nothing fancy at all. I haven't used YaST firstboot for a
while but I guess it provides some kind of interface even if it is
ncurses based, but well, I am not that familiar with YaST.

Regards,
David

[Roger Oberholtzer]

Status:

To get an X11 YaST GUI, one must install yast2-x11. It is not enough
to have patterns-openSUSE-x11 and yast2-control-center-qt. Of course,
one would think that yast2-control-center-qt would Require yast2-x11.
But that's not the case.

A good start for a current firstboot.xml is a current Leap 42.3
/etc/YasT2/firstboot.xml. I see that there are a few significant
differences. Why I did not think of looking there in the first place
is a mystery.

If the firstboot does not show a license, and /etc/sysconfig/firstboot
has the default LICENSE_REFUSAL_ACTION="halt", the system will be
halted. This is different than previous behavior.

After all this, I get much further. I have a few firstboot details to
sort out. The network stuff in qemu is foreign to me. I truly hope it
will all come together on Monday!

The EFI error during build continues. I think it is benign. So far.

Time to do legs at the gym, and then have a couple of beers while
doing laundry. Do I know how to have fun, or what?

--
Roger Oberholtzer

[Roger Oberholtzer]

It was ncurses based because I had not installed yast2-x11. Seems
adding yast2-control-center-qt was not enough. Shouldn't that Require
yast2-x11?

As to firstboot.xml, I am using /etc/YaST2/firstboot.xml from a
standard Leap 42.3. Of course, I am editing it to suit my needs. That
is in progress.



--
Roger Oberholtzer

[Roger Oberholtzer]

On Fri, Sep 15, 2017 at 4:17 PM, Roger Oberholtzer
<roger.ob...@gmail.com> wrote:

> As to firstboot.xml, I am using /etc/YaST2/firstboot.xml from a
> standard Leap 42.3. Of course, I am editing it to suit my needs. That
> is in progress.

Perhaps not a kiwi issue directly. But here is a pointer to a bit of
information on setting up firstboot.xml. I think it is the same for
openSUSE Leap. I don't know if there is something newer.

https://www.suse.com/documentation/sles11/book_sle_deployment/data/sec_fb_customize.html

The kiwi docs do lack information on firstboot stuff...



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> To get an X11 YaST GUI, one must install yast2-x11.

Thanks for sharing the details

> The EFI error during build continues. I think it is benign. So far.

It's annoying and still not clear to my why you still have that
issue even with genisoimage instead of mkisofs. The only trigger
for that problem was mkisofs so far... hope we'll find out what's
causing it

> Time to do legs at the gym, and then have a couple of beers while
> doing laundry. Do I know how to have fun, or what?

*lol* :-)) sounds promising

[Roger Oberholtzer]

Just to summarize, I think the kiwi build, sans /home filling the
remaining disk space via LVM, is working. All that remains is getting
dice to work so I can perhaps build there and get my partition added.

The remaining problems I have are related to openSUSE itself. I have
been finding dependencies that mysteriously are not resolved when some
packages are added. But that's not a kiwi issue.

So perhaps this thread can be considered closed.

--
Roger Oberholtzer

[David Cassany]

> The remaining problems I have are related to openSUSE itself. I have
> been finding dependencies that mysteriously are not resolved when some
> packages are added. But that's not a kiwi issue.
>
> So perhaps this thread can be considered closed.

Thanks for the update.

Regards,
David

[Marcus Schäfer]

Hi,

> Just to summarize, I think the kiwi build, sans /home filling the
> remaining disk space via LVM, is working. All that remains is getting
> dice to work so I can perhaps build there and get my partition added.

Thanks for the feedback, the dice issue is on my list, however I
have not heard back from André since he is still on vacation.

Interesting enough I have done a fresh install of Leap42.3 and
installed the dice package which worked for me. I guess some other
ruby code/gem on your system overwrites the URI namespace

[Roger Oberholtzer]

On Fri, Sep 22, 2017 at 11:42 AM, Marcus Schäfer <m...@suse.de> wrote:

> Interesting enough I have done a fresh install of Leap42.3 and
> installed the dice package which worked for me. I guess some other
> ruby code/gem on your system overwrites the URI namespace

I only have the things pulled in by YaST and dice:

libruby2_1-2_1-2.1.9-9.1.x86_64
libstorage-ruby-2.26.12-1.5.x86_64
ruby2.1-2.1.9-9.1.x86_64
ruby-2.1-6.4.x86_64
ruby2.1-rubygem-abstract_method-1.2.1-7.5.x86_64
ruby2.1-rubygem-cfa-0.6.1-1.3.x86_64
ruby2.1-rubygem-cfa_grub2-0.6.2-1.4.x86_64
ruby2.1-rubygem-cheetah-0.5.0-3.5.x86_64
ruby2.1-rubygem-dice-0.7.3-5.3.x86_64
ruby2.1-rubygem-docker-api-1.31.0-11.16.x86_64
ruby2.1-rubygem-excon-0.54.0-10.3.x86_64
ruby2.1-rubygem-fast_gettext-1_2-1.2.0-3.3.x86_64
ruby2.1-rubygem-gem2rpm-0.10.1-7.13.x86_64
ruby2.1-rubygem-ruby-augeas-0.5.0-6.18.x86_64
ruby2.1-rubygem-ruby-dbus-0.9.3-7.1.x86_64
ruby2.1-stdlib-2.1.9-9.1.x86_64
ruby-common-2.1-8.3.noarch
ruby-solv-0.6.27-1.3.x86_64
yast2-ruby-bindings-3.2.14-1.1.x86_64

I would imagine that these are the default packages.

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

got feedback from Andre, thanks much for the help with ruby :)

> > Interesting enough I have done a fresh install of Leap42.3 and
> > installed the dice package which worked for me. I guess some other
> > ruby code/gem on your system overwrites the URI namespace
>
> I only have the things pulled in by YaST and dice:

Thanks for the list, I built an Leap42.3 appliance to see if I can
reproduce this problem but for me it worked

Therefore I need your help. Could you please create a ruby test
script with the following contents:

vi uri_methods.rb

Gem.path.unshift(
"/usr/lib64/ruby/gems/2.1.0/gems/dice-0.7.10/bundle/ruby/2.1.0"
)
require 'rubygems'

require "cheetah"
require "gli"
require "pathname"
require "fileutils"
require "digest"
require "digest/bubblebabble"
require "find"
require "abstract_method"
require "rexml/document"
require "open-uri"
require "tmpdir"
require "ostruct"
require "solv"
require "json"
require "inifile"
require "time"

puts URI.methods()

Call the script with:

$ ruby uri_methods.rb

It will respond with all methods the URI namespace has after all the
dice required modules gets imported.

In your case the output should not provide a "parse" method because
that's what the error you posted said.

Now take one of the methods in the list and add the following
lines to your script by replacing XXXX with the name of the method
you chose:

puts "Now we are talking..."
puts URI.method(:XXXX).source_location

Call the script again with:

$ ruby uri_methods.rb

And now *drumroll* the output should list the source location
of the code providing the method. As this is the URI namespace
nobody else should use I usually get:

/usr/lib64/ruby/2.1.0/uri/common.rb

and I expect something different in your case, which would be
really really bad

Thanks much for your help

[Roger Oberholtzer]

I had to change the path to
"/usr/lib64/ruby/gems/2.1.0/gems/dice-0.7.3/bundle/ruby/2.1.0" as your
path does not exist.

> Call the script with:
>
> $ ruby uri_methods.rb
>
> It will respond with all methods the URI namespace has after all the
> dice required modules gets imported.
>
> In your case the output should not provide a "parse" method because
> that's what the error you posted said.

In fact, I do get a 'parse' line.

> Now take one of the methods in the list and add the following
> lines to your script by replacing XXXX with the name of the method
> you chose:
>
> puts "Now we are talking..."
> puts URI.method(:XXXX).source_location
>
> Call the script again with:
>
> $ ruby uri_methods.rb

I used parse as the method, and I got:

Now we are talking...
/usr/lib64/ruby/2.1.0/uri/common.rb
746

>
> And now *drumroll* the output should list the source location
> of the code providing the method. As this is the URI namespace
> nobody else should use I usually get:
>
> /usr/lib64/ruby/2.1.0/uri/common.rb
>
> and I expect something different in your case, which would be
> really really bad

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I had to change the path to
> "/usr/lib64/ruby/gems/2.1.0/gems/dice-0.7.3/bundle/ruby/2.1.0" as your
> path does not exist.

Why the old version ? I thought you installed as described in
http://suse.github.io/kiwi/building/build_containerized.html

Could you please use the package from here:

http://download.opensuse.org/repositories/Virtualization:/Appliances:/ContainerBuilder

That also explains the uri problem you have. in 0.7.3 dice had this bug

commit 584549bd02fe8d3561b035ddcc69070235516c20
Author: Marcus Schäfer <m...@suse.com>
Date: Thu Nov 12 11:05:46 2015 +0100

Rename uri to repo_uri

This caused a name conflict on the openuri side of ruby


Uhm, I should have asked earlier which version you used :)

[Roger Oberholtzer]

On Wed, Sep 27, 2017 at 11:00 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I had to change the path to
>> "/usr/lib64/ruby/gems/2.1.0/gems/dice-0.7.3/bundle/ruby/2.1.0" as your
>> path does not exist.
>
> Why the old version ? I thought you installed as described in
> http://suse.github.io/kiwi/building/build_containerized.html

I thought I had. I see that I was too quick when looking at the repo
Virtualization:/Appliances:/ContainerBuilder. I saw it as
Virtualization:/Appliances:/Builder, and thought I already had it as a
repo.

I have added that and updated dice.

Now it gets further.

I have a repo that is an ISO image. It is specified in config.xml as:

iso:/home/roger/source.18.00/Tools_Diskless/openSUSE-Leap-42.3-DVD-x86_64.iso

dice complains: Can't find resource type/location

This is in a different directory than where dice was told the spec
file was. Is this possible?



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I have a repo that is an ISO image. It is specified in config.xml as:
>
> iso:/home/roger/source.18.00/Tools_Diskless/openSUSE-Leap-42.3-DVD-x86_64.iso
>
> dice complains: Can't find resource type/location

correct because you build in a container and inside of the container
this local path does not exist.

I recommend to use remote protocol types because network exists in
the container. If that is too much of an effort you can also move
your iso to /tmp and reference it from there. The container shares
/tmp and /dev with the host

But I see that more as a "nasty" workaround

When building contained there should be no reference to the host
it is building on and that also covers the repos

[Roger Oberholtzer]

On Wed, Sep 27, 2017 at 2:43 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I have a repo that is an ISO image. It is specified in config.xml as:
>>
>> iso:/home/roger/source.18.00/Tools_Diskless/openSUSE-Leap-42.3-DVD-x86_64.iso
>>
>> dice complains: Can't find resource type/location
>
> correct because you build in a container and inside of the container
> this local path does not exist.
>
> I recommend to use remote protocol types because network exists in
> the container. If that is too much of an effort you can also move
> your iso to /tmp and reference it from there. The container shares
> /tmp and /dev with the host
>
> But I see that more as a "nasty" workaround

As a start to see if dice will solve my LVM issue, I have copied the
file to /tmp. I guess that it is mounted as /tmp in the container? I
get this:

Can't find resource type/location in iso:/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso

I guess the RPM cache in the container is it's own cache, not the one
kiwi had been using on the host. So I would like not to have to
download all the RPMs over the network when they are in the local ISO
image.

> When building contained there should be no reference to the host
> it is building on and that also covers the repos

Got it.

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> As a start to see if dice will solve my LVM issue, I have copied the
> file to /tmp. I guess that it is mounted as /tmp in the container? I
> get this:
>
> Can't find resource type/location in iso:/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso

That dice thing is picky on uri types :) In your XML set:

iso:///tmp/openSUSE-Leap-42.3-DVD-x86_64.iso
^^^

> I guess the RPM cache in the container is it's own cache, not the one
> kiwi had been using on the host. So I would like not to have to
> download all the RPMs over the network when they are in the local ISO
> image.

correct

[Roger Oberholtzer]

On Wed, Sep 27, 2017 at 3:47 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> As a start to see if dice will solve my LVM issue, I have copied the
>> file to /tmp. I guess that it is mounted as /tmp in the container? I
>> get this:
>>
>> Can't find resource type/location in iso:/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso
>
> That dice thing is picky on uri types :) In your XML set:
>
> iso:///tmp/openSUSE-Leap-42.3-DVD-x86_64.iso
> ^^^

OK. I got a bit further. I now get:

[32112][spec]: Mounting /tmp/openSUSE-Leap-42.3-DVD-x86_64.iso failed:
sudo: a password is required

Is there such a thing for a local file?



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> > That dice thing is picky on uri types :) In your XML set:
> >
> > iso:///tmp/openSUSE-Leap-42.3-DVD-x86_64.iso
> > ^^^
>
> OK. I got a bit further. I now get:
>
> [32112][spec]: Mounting /tmp/openSUSE-Leap-42.3-DVD-x86_64.iso failed:
> sudo: a password is required
>
> Is there such a thing for a local file?

Oh wait, can you send me the output from your dice call with the
option "--debug" included.

The mount call is performed via

sudo", "-n", "mount", location, mount_dir

but I probably forgot to add a proper sudo setup inside of the
container :(

[Marcus Schäfer]

Hi,

> but I probably forgot to add a proper sudo setup inside of the
> container :(

nope sudo in the container works, the mount happens outside of
the container. So I guess sudo on your build system when called
as the user who called dice asks for a password

It would be good to allow sudo for the user who calls dice if
a mount call is required as it is the case when you use a local
iso

Alternatively perform the mount yourself in /tmp and instead of
the iso:/// type use dir:///tmp/... in that case dice doesn't
have to call mount

[Roger Oberholtzer]

On Wed, Sep 27, 2017 at 4:32 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> > That dice thing is picky on uri types :) In your XML set:
>> >
>> > iso:///tmp/openSUSE-Leap-42.3-DVD-x86_64.iso
>> > ^^^
>>
>> OK. I got a bit further. I now get:
>>
>> [32112][spec]: Mounting /tmp/openSUSE-Leap-42.3-DVD-x86_64.iso failed:
>> sudo: a password is required
>>
>> Is there such a thing for a local file?
>
> Oh wait, can you send me the output from your dice call with the
> option "--debug" included.
>
> The mount call is performed via
>
> sudo", "-n", "mount", location, mount_dir
>
> but I probably forgot to add a proper sudo setup inside of the
> container :(

I get this in the debug:

[7774][spec]: EXEC: [sudo -n mount
/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso /tmp/d20170928-7774-edoi1o]
[7774][spec]: EXEC: [sudo -n mount
/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso /tmp/d20170928-7774-1s8urup]
[7774][spec]: EXEC: [sudo -n mount
/tmp/openSUSE-Leap-42.3-DVD-x86_64.iso /tmp/d20170928-7774-1a5595x]
[7774][spec]: Mounting /tmp/openSUSE-Leap-42.3-DVD-x86_64.iso failed:

sudo: a password is required

[7774][spec]:

If it is done outside the container, then it probably would fail as
the user running dice has no sudo permissions.

When I set up the user with sudo permissions for mount, I got much further.

I see that it is now running the docker command. Exciting.

--
Roger Oberholtzer

[Roger Oberholtzer]

On Wed, Sep 27, 2017 at 4:39 PM, Marcus Schäfer <m...@suse.de> wrote:

> Alternatively perform the mount yourself in /tmp and instead of
> the iso:/// type use dir:///tmp/... in that case dice doesn't
> have to call mount

I get much further with dice now. In fact, I see that the error it
reports is the same one that my native build reports. isohybrid gives
a warning, and kiwi considers this an error. In my native build, I got
the same warning/error. But the ISO seemed okay.

[ DEBUG ]: 07:00:46 | EXEC: [/usr/bin/genisoimage -hide header_end
-hide-joliet header_end -V "KIWI Installation System" -A 0xf863206e -R
-J -f -pad -joliet-long -sort /tmp/tmpsvcl1ubn -no-emul-boot
-boot-load-size 4 -boot-info-table -hide boot/x86_64/boot.catalog
-hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o
/tmp/kiwi_build_20170928-8239-yuicpi.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso
/tmp/kiwi_build_20170928-8239-yuicpi.dice/kiwi_install_media.8idp6_hz]
[ DEBUG ]: 07:00:58 | Relocated boot catalog from sector 0x3ce5 to 0x14
[ DEBUG ]: 07:00:58 | Fixed iso catalog contents
[ DEBUG ]: 07:00:58 | EXEC: [isohybrid --offset 31176 --id
0xf863206e --type 0x83 --uefi
/tmp/kiwi_build_20170928-8239-yuicpi.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
[ INFO ]: 07:01:12 | Cleaning up InstallImageBuilder instance
[ ERROR ]: 07:01:12 | KiwiCommandError: isohybrid: isohybrid:
Warning: more than 1024 cylinders: 1817
isohybrid: Not all BIOSes will be able to boot this device

In my native build, the created images are:

-rw-r--r-- 1 root root 1911554048 Sep 25 09:47
RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso
-rw-r--r-- 1 root root 256960 Sep 25 09:30
RSL-OEM-Leap-42.3.x86_64-42.3.2.packages
-rw-r--r-- 1 root root 9811525632 Sep 25 09:26
RSL-OEM-Leap-42.3.x86_64-42.3.2.raw
-rw-r--r-- 1 root root 3160 Sep 25 09:31
RSL-OEM-Leap-42.3.x86_64-42.3.2.verified

After dice is finished, where are the results? I see the log in the
.dice directory. But where is the build? Or is that all deleted if
there is an error? There is no /tmp/kiwi* directory.




--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I get much further with dice now. In fact, I see that the error it
> reports is the same one

Not the same one, it does not complain about not finding the efi
loader which is good. It now complains about the cylinders count
which is beyond 1024 due to the size of the image you build.

Also see http://www.syslinux.org/archives/2015-March/023306.html

However this warning:

"Not all BIOSes will be able to boot this device"

is now really something we should ignore. Problem is kiwi
treats isohybrid warnings as errors because almost all of
them are real problems

> that my native build reports. isohybrid gives
> a warning, and kiwi considers this an error. In my native build, I got
> the same warning/error. But the ISO seemed okay.

yes I'm thinking to write a patch for kiwi to ignore this
specific warning... not nice though

> [ INFO ]: 07:01:12 | Cleaning up InstallImageBuilder instance
> [ ERROR ]: 07:01:12 | KiwiCommandError: isohybrid: isohybrid:
> Warning: more than 1024 cylinders: 1817
> isohybrid: Not all BIOSes will be able to boot this device
>
> In my native build, the created images are:
>

> After dice is finished, where are the results?

Because your build ends with an error it did not produce a result.
On success it prints the information where the result is and at
any time you can call:

dice status RECIPE-PATH

the location to the result is displayed too

> .dice directory. But where is the build? Or is that all deleted if
> there is an error?

correct no results on error

[Marcus Schäfer]

Hi,

> However this warning:
>
> "Not all BIOSes will be able to boot this device"
>
> is now really something we should ignore.

See:

https://github.com/SUSE/kiwi/pull/513

Once merged a new container will appear on dockerhub and
a rebuild of your image with dice should fetch it

[Roger Oberholtzer]

On Thu, Sep 28, 2017 at 11:22 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I get much further with dice now. In fact, I see that the error it
>> reports is the same one
>
> Not the same one, it does not complain about not finding the efi
> loader which is good. It now complains about the cylinders count
> which is beyond 1024 due to the size of the image you build.

They are similar. In my native build I get:

DEBUG: EXEC: [isohybrid --offset 31176 --id 0x366e79d5 --type 0x83
--uefi /home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
ERROR: KiwiCommandError: isohybrid:
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/image/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso:
warning: unable to find efi image
isohybrid: Warning: more than 1024 cylinders: 1823

isohybrid: Not all BIOSes will be able to boot this device

In dice/docker, I get

[ DEBUG ]: 07:00:58 | EXEC: [isohybrid --offset 31176 --id
0xf863206e --type 0x83 --uefi
/tmp/kiwi_build_20170928-8239-yuicpi.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]

[ INFO ]: 07:01:12 | Cleaning up InstallImageBuilder instance
[ ERROR ]: 07:01:12 | KiwiCommandError: isohybrid: isohybrid:
Warning: more than 1024 cylinders: 1817
isohybrid: Not all BIOSes will be able to boot this device

I wonder why they the native build has the additional warning.

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I wonder why they the native build has the additional warning.

Because there for some reason still another not isohybrid compatible
mkisofs tool is installed on the host you are building.

In the container I know exactly which tools surrounds the build
which is nice.

[Marcus Schäfer]

Hi,

> Once merged a new container will appear on dockerhub and
> a rebuild of your image with dice should fetch it

new container is online now, you can give it another try.

Feedback is much appreciated

Thanks

[Roger Oberholtzer]

On Fri, Sep 29, 2017 at 10:43 AM, Marcus Schäfer <m...@suse.de> wrote:

>> Once merged a new container will appear on dockerhub and
>> a rebuild of your image with dice should fetch it
>
> new container is online now, you can give it another try.
>
> Feedback is much appreciated

I can check on Monday.

Thanks!

--
Roger Oberholtzer

[Roger Oberholtzer]

On Fri, Sep 29, 2017 at 10:43 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> Once merged a new container will appear on dockerhub and
>> a rebuild of your image with dice should fetch it
>
> new container is online now, you can give it another try.
>
> Feedback is much appreciated
>

Odd. I got a totally new error. There is no change to my kiwi config.

[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/network.service to
/usr/lib/systemd/system/wicked.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/multi-user.target.wants/wicked.service to
/usr/lib/systemd/system/wicked.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/network-online.target.wants/wicked.service to
/usr/lib/systemd/system/wicked.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/dbus-org.opensuse.Network.Nanny.service to
/usr/lib/systemd/system/wickedd-nanny.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/dbus-org.opensuse.Network.AUTO4.service to
/usr/lib/systemd/system/wickedd-auto4.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/dbus-org.opensuse.Network.DHCP4.service to
/usr/lib/systemd/system/wickedd-dhcp4.service.
[ DEBUG ]: 06:59:20 | bootstrap: Created symlink from
/etc/systemd/system/dbus-org.opensuse.Network.DHCP6.service to
/usr/lib/systemd/system/wickedd-dhcp6.service.
[ DEBUG ]: 06:59:22 | bootstrap: Output of
btrfsprogs-4.5.3-6.3.x86_64.rpm %posttrans script:
[ DEBUG ]: 06:59:22 | bootstrap: Please run mkinitrd as soon as
your system is complete.
[ DEBUG ]: 06:59:22 | bootstrap: Output of
dmraid-1.0.0.rc16-40.4.x86_64.rpm %posttrans script:
[ DEBUG ]: 06:59:22 | bootstrap: Updating /etc/sysconfig/dmraid...
[ DEBUG ]: 06:59:22 | bootstrap: Running:
gnu-unifont-bitmap-fonts-20080123-88.3-reconfigure-fonts
(gnu-unifont-bitmap-fonts,
/tmp/kiwi_build_20171002-17479-1rvj7ab.dice/kiwi_boot_root.3ybracu3/var/adm/update-scripts)
[ DEBUG ]: 06:59:22 | bootstrap: done]
[ INFO ]: Processing: [########################################] 100%
[ ERROR ]: 06:59:22 | KiwiBootStrapPhaseFailed: Bootstrap package
installation failed:
[ INFO ]: 06:59:22 | Cleaning up SystemPrepare instance

It then unmounts various things and exits.


Could there be something missing in the docker image?

While on that topic, does the docker image contain the components
needed to ass the nvidia proprietary driver. It requires compiling a
kernel module.

I really need a faster test system. It takes so long to run this on my system.



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Odd. I got a totally new error. There is no change to my kiwi config.

I guess a package install failed, probably the package repo contents
have changed ? (update repo ?)

> [ ERROR ]: 06:59:22 | KiwiBootStrapPhaseFailed: Bootstrap package
> installation failed:
> [ INFO ]: 06:59:22 | Cleaning up SystemPrepare instance

In the output there was no package install error, unfortunately zypper
just prints lots of information and the error is not always at the
end. Could you send the complete log ? or try searching for
"scriptlet failed"

> Could there be something missing in the docker image?

no don't think, nothing gets removed, just some packages added

> While on that topic, does the docker image contain the components
> needed to ass the nvidia proprietary driver. It requires compiling a
> kernel module.

don't think so, I have not added kernel-source or similar to the
container

[Roger Oberholtzer]

On Tue, Oct 3, 2017 at 7:51 AM, Roger Oberholtzer
<roger.ob...@gmail.com> wrote:

> On Mon, Oct 2, 2017 at 11:38 AM, Marcus Schäfer <m...@suse.de> wrote:
>> Hi,
>>

>>> Here is the whole log. I grep'd for 'failed' and got a few hits. So
>>> perhaps it is netter that you can see them in context.
>>
>> Here is the problem:
>>
>> [ DEBUG ]: 06:59:17 | bootstrap: (228/232) Installing: plymouth-branding-openSUSE-42.1-10.5.noarch [........done]
>> [ DEBUG ]: 06:59:17 | bootstrap: Additional rpm output:
>> [ DEBUG ]: 06:59:17 | bootstrap: warning: /tmp/kiwi_build_20171002-17479-1rvj7ab.dice/kiwi_boot_root.3ybracu3/var/cache/kiwi/packages/5c57534402c1b586153a56dfa2a4e05f/suse/noarch/plymouth-branding-openSUSE-42.1-10.5.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 3dbdc284: NOKEY
>> [ DEBUG ]: 06:59:17 | bootstrap: No kernel found in /boot or bad modules dir in /lib/modules
>> [ DEBUG ]: 06:59:17 | bootstrap: warning: %post(plymouth-branding-openSUSE-42.1-10.5.noarch) scriptlet failed, exit status 1
>>
>> This is causing zypper to exit with non zero.
>> plymouth-branding-openSUSE-42.1-10.5.noarch expects a kernel to be installed.
>> I suggest to move this package out from the bootstrap section. It seemed
>> to be installed before the kernel package was installed
>
> It's not in the bootstrap section. It is in the main image
> (type="image") description as:
>
> <package name="plymouth-branding-openSUSE" bootinclude="true"/>
>
> It has always bee this way. I have not moved it.

I see that this is how this package is specified in other config.xml
files. It does not fail when built native. This only happens in dice.


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I see that this is how this package is specified in other config.xml
> files. It does not fail when built native. This only happens in dice.

Building in a container doesn't require to have a kernel installed in
the container but the package script from
plymouth-branding-openSUSE seems to look on the system searching for
a kernel

I still see this as a packaging bug which you should be able to solve
if that package is not installed as part of the bootstrap phase
which is the case according to the log:

> [ DEBUG ]: 06:59:17 | bootstrap: warning:
%post(plymouth-branding-openSUSE-42.1-10.5.noarch) scriptlet failed, exit status
1

Have you tried moving the package to the <packages type="image">
section.

I'd like to avoid to make the dice container even bigger with an
unneeded install of a kernel package

Thanks

[Roger Oberholtzer]

On Wed, Oct 4, 2017 at 6:04 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I see that this is how this package is specified in other config.xml
>> files. It does not fail when built native. This only happens in dice.
>
> Building in a container doesn't require to have a kernel installed in
> the container but the package script from
> plymouth-branding-openSUSE seems to look on the system searching for
> a kernel
>
> I still see this as a packaging bug which you should be able to solve
> if that package is not installed as part of the bootstrap phase
> which is the case according to the log:
>
> > [ DEBUG ]: 06:59:17 | bootstrap: warning:
> %post(plymouth-branding-openSUSE-42.1-10.5.noarch) scriptlet failed, exit status
> 1
>
> Have you tried moving the package to the <packages type="image">
> section.

That is where it is. It is not in the <packages type="bootstrap"> section.

But is also includes the bootinclude="true" directive. Which is how I
always see this specified:

<package name="plymouth-branding-openSUSE" bootinclude="true"/>

All I have in the bootstrap section is:

<packages type="bootstrap">
<package name="udev"/>
<package name="filesystem"/>
<package name="glibc-locale"/>
<package name="cracklib-dict-full"/>
<package name="ca-certificates"/>
<package name="openSUSE-release"/>
</packages>

Everything else is in the <packages type="image"> section.



--
Roger Oberholtzer

[Roger Oberholtzer]

I have given up on building in Dice. Something is not correct there.
So I am once again trying to get the logical volumes correct in my
native build in on Leap 42.3 with kiwi-ng.

I have now been able to remove the volume I had on this system so that
the kiwi build does not get confused. I have also replaced
/sbin/lvmetad by doing this: cp /usr/bin/true /sbin/lvmetad

Now I do not get the error that the VG already exists. I now get:

DEBUG: custom arguments for bootloader installation
{'target_removable': None, 'efi_device': u'/dev/mapper/loop1p2',
'boot_device': u'/dev/mapper/loop1p3', 'firmware':
<kiwi.firmware.FirmWare object at 0x7f8495492650>, 'root_device':
'/dev/vgroup-rsl/LVRoot', 'system_volumes':
{'source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co/misc':
{'volume_device': '/dev/vgroup-rsl/misc', 'volume_options':
'defaults'}, 'source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co/home':
{'volume_device': '/dev/vgroup-rsl/home', 'volume_options':
'defaults'}, 'source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co/opt':
{'volume_device': '/dev/vgroup-rsl/opt', 'volume_options':
'defaults'}, 'source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co':
{'volume_device': '/dev/vgroup-rsl/LVRoot', 'volume_options':
'defaults'}}}
INFO: Installing grub2 on disk /dev/loop1
DEBUG: EXEC: [mountpoint
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d]
DEBUG: EXEC: [mount /dev/vgroup-rsl/LVRoot
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d]
DEBUG: EXEC: [mountpoint
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/boot]
DEBUG: EXEC: [mount /dev/mapper/loop1p3
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/boot]
DEBUG: EXEC: [mountpoint
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co]
DEBUG: EXEC: [mount -o defaults /dev/vgroup-rsl/LVRoot
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co]
DEBUG: EXEC: Failed with stderr: mount: mount point
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co
does not exist
, stdout: (no output on stdout)
ERROR: KiwiCommandError: mount: stderr: mount: mount point
/home/roger/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_mount_manager.NPZy5d/source/RSofT_Diskless/kiwi/RSofT-preload-openSUSE-42.3/tmp/kiwi_volumes.h8d3Co
does not exist
, stdout: (no output on stdout)
INFO: Cleaning up BootLoaderInstallGrub2 instance

My config.xml statement is now:

<systemdisk name="vgroup-rsl">
<volume name="opt" freespace="5G"
mountpoint="opt"/>
<volume name="misc" freespace="5G"
mountpoint="misc"/>
<volume name="home" freespace="all"
mountpoint="home"/>
</systemdisk>

I now need more than one partition. I wasn't sure if this was the way
to do it. I tried a <systemdisk> for each. But that seemed wrong.

My local file system is btrfs. I cannot change that. I could only
remove an existing logical volume.

I am out of ideas. This is pretty much the last kiwi thing I seem not
to get past.

--
Roger Oberholtzer

[Roger Oberholtzer]

I get the same error with one volume:

<systemdisk name="vgroup-rsl">

<volume name="home" freespace="all"
mountpoint="home"/>
</systemdisk>

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

I'll give your volume setup a try, maybe something in the dice
container is missing.

The container approach only shares the kernel with the container
the rest is an isolated environment. I can't tell if btrfs on the
host plays a role if you run lvm inside of the container. I would
say from a pure kernel perspective it could be the case but I
need to test this.

I will give it a try on a non btrfs host plus dice, if that works
I'll setup a btrfs based system and give that a try.

Give me some time to work through this, currently there is
also another new project on my plate. Thus sorry if response
times are not as fast as usual

[Marcus Schäfer]

Hi,

> > <systemdisk name="vgroup-rsl">
> > <volume name="opt" freespace="5G"
> > mountpoint="opt"/>
> > <volume name="misc" freespace="5G"
> > mountpoint="misc"/>
> > <volume name="home" freespace="all"
> > mountpoint="home"/>
> > </systemdisk>
> >
> > I now need more than one partition. I wasn't sure if this was the way
> > to do it. I tried a <systemdisk> for each. But that seemed wrong.
> >
> > My local file system is btrfs. I cannot change that. I could only
> > remove an existing logical volume.
> >
> > I am out of ideas. This is pretty much the last kiwi thing I seem not
> > to get past.
>
> I'll give your volume setup a try, maybe something in the dice
> container is missing.

I can reproduce part of the package scriptlet failed problems due
to a missing kernel in the container. Actually from a container
perspective no kernel is needed but some stupid package pre/post
scripts looks for it and fails

I'll update the container and give it another try

I was not able to reproduce any of the strange mount errors
you got and can just imagine it has something to do with
btrfs on your host... which would be really bad

Once my build works I'll upload a new dice container to dockerhub
and ask you for a next test

...stay tuned

[Marcus Schäfer]

Hi,

> Once my build works I'll upload a new dice container to dockerhub
> and ask you for a next test

ok I successfully built lvm based images with dice. The new
container is online. Would be great if you can give it another
try

Thanks

[Roger Oberholtzer]

On Mon, Oct 16, 2017 at 4:29 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> Once my build works I'll upload a new dice container to dockerhub
>> and ask you for a next test
>
> ok I successfully built lvm based images with dice. The new
> container is online. Would be great if you can give it another
> try

I think it completed. I had three types defined in the spec file.
Usually I run kiwi with the type I want. I did not do that with dice.
So I guess it took the first one, which was PXE. And all seems to be
present. If I want to get an OEM image of the same thing, I guess I
need to do the whole build again? And the results will replace those
from the previous build? How is the type of build specified to dice?
Or must I edit the config.spec file?

I'm getting excited...

--
Roger Oberholtzer

[Roger Oberholtzer]

On Mon, Oct 16, 2017 at 4:29 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> Once my build works I'll upload a new dice container to dockerhub
>> and ask you for a next test
>
> ok I successfully built lvm based images with dice. The new
> container is online. Would be great if you can give it another
> try

If I want to have all temp stuff on other than /tmp, can I do that? My
/tmp has only 8.6GB free. It is on /root. Is my only option to mount
some larger partition as /tmp?



--
Roger Oberholtzer

[David Cassany]

Hi Roger,

I think that what you need is to setup a TMPDIR env variable.

See that running:

$ export TMPDIR=$HOME
$ mktemp

creates a temporary directory in your HOME. I hope it helps

Regards,
David

[Roger Oberholtzer]

On Tue, Oct 17, 2017 at 1:55 PM, David Cassany <dcas...@suse.de> wrote:
> Hi Roger,
>
> I think that what you need is to setup a TMPDIR env variable.
>
> See that running:
>
> $ export TMPDIR=$HOME
> $ mktemp
>
> creates a temporary directory in your HOME. I hope it helps

Well, I did mount a different partition for /tmp. And after that
docker is not happy. I cannot guess what Docker kept in /tmp that it
needed. That is surely wrong. So, I now need to make Docker happy
again. I get this when running dice. I have not had a change to start
from the beginning again to see what may have gone wrong. It's next up
when I get a chance.

[4380][spec]: Job: Preparing build...
[4380][spec]: Job: Preparation failed
[4380][spec]: Preparing build environment failed with: docker: Error
response from daemon: layer does not exist.
[4380][spec]: See 'docker run --help'.
[4380][spec]:
[4380][spec]: DockerBuildSystem: Delete container...

I was soooo close...




--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> If I want to have all temp stuff on other than /tmp, can I do that? My
> /tmp has only 8.6GB free. It is on /root. Is my only option to mount
> some larger partition as /tmp?

when dice starts docker it shares /tmp and /dev from the host with
the container:

command = [
"docker", "run",
"--rm=true",
"--entrypoint=sudo",
"--privileged=true",
"--name=#{container_name}",
"-v", "#{recipe.basepath}:/vagrant",
"-v", "/tmp:/tmp",
"-v", "/dev:/dev",
Dice::DOCKER_BUILD_CONTAINER,
"bash", "-c", action
]

Thus if you need more space in /tmp you can only copy all of /tmp
to another bigger space and overmount this space to /tmp on your
host... which should be considered a temporary "fix" but not the
normal state of the system

[Roger Oberholtzer]

On Mon, Oct 16, 2017 at 4:29 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> Once my build works I'll upload a new dice container to dockerhub
>> and ask you for a next test
>
> ok I successfully built lvm based images with dice. The new
> container is online. Would be great if you can give it another
> try

I think that the LVM stuff works. I see this in the log:

[ DEBUG ]: 06:52:17 | --> kiwi_Volume_1='opt|freespace:5120|opt'
[ DEBUG ]: 06:52:17 | --> kiwi_Volume_2='misc|freespace:5120|misc'
[ DEBUG ]: 06:52:17 | --> kiwi_Volume_3='home|size:all|home'
[ DEBUG ]: 06:52:17 | --> kiwi_Volume_4='LVRoot|freespace:30|'

and a bit later:

[ DEBUG ]: 06:52:41 | EXEC: [du -s --apparent-size --block-size 1
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/opt]
[ DEBUG ]: 06:52:41 | EXEC: [bash -c find
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/opt | wc
-l]
[ DEBUG ]: 06:52:41 | EXEC: [du -s --apparent-size --block-size 1
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/home]
[ DEBUG ]: 06:52:41 | EXEC: [bash -c find
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/home | wc
-l]

[ DEBUG ]: 06:53:08 | EXEC: [vgremove --force vgroup-rsl]
[ DEBUG ]: 06:53:08 | EXEC: [pvcreate /dev/mapper/loop0p4]
[ DEBUG ]: 06:53:08 | EXEC: [vgcreate vgroup-rsl /dev/mapper/loop0p4]
[ INFO ]: 06:53:09 | Creating volumes(ext4)
[ DEBUG ]: 06:53:09 | EXEC: [mkdir -p
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/misc]
[ DEBUG ]: 06:53:09 | EXEC: [du -s --apparent-size --block-size 1
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/misc]
[ DEBUG ]: 06:53:09 | EXEC: [bash -c find
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/misc | wc
-l]
[ INFO ]: 06:53:09 | --> volume misc with 30 MB
[ DEBUG ]: 06:53:09 | EXEC: [lvcreate -L 30 -n misc vgroup-rsl]
[ DEBUG ]: 06:53:10 | EXEC: [mkfs.ext4 /dev/vgroup-rsl/misc]
[ DEBUG ]: 06:53:11 | EXEC: [du -s --apparent-size --block-size 1
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/opt]
[ DEBUG ]: 06:53:11 | EXEC: [bash -c find
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/opt | wc
-l]
[ INFO ]: 06:53:11 | --> volume opt with 401 MB
[ DEBUG ]: 06:53:11 | EXEC: [lvcreate -L 401 -n opt vgroup-rsl]
[ DEBUG ]: 06:53:12 | EXEC: [mkfs.ext4 /dev/vgroup-rsl/opt]
[ DEBUG ]: 06:53:12 | EXEC: [du -s --apparent-size --block-size 1
--exclude /tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/opt
--exclude /tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/misc
--exclude /tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root/home
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root]
[ DEBUG ]: 06:53:14 | EXEC: [bash -c find
/tmp/kiwi_build_20171018-19489-1osux4i.dice/build/image-root | wc -l]
[ INFO ]: 06:53:15 | --> volume LVRoot with 8716 MB
[ DEBUG ]: 06:53:15 | EXEC: [lvcreate -L 8716 -n LVRoot vgroup-rsl]
[ DEBUG ]: 06:53:16 | EXEC: [mkfs.ext4 -L ROOT /dev/vgroup-rsl/LVRoot]
[ INFO ]: 06:53:17 | --> fullsize volume home
[ DEBUG ]: 06:53:17 | EXEC: [lvcreate -l +100%FREE -n home vgroup-rsl]
[ DEBUG ]: 06:53:17 | EXEC: [mkfs.ext4 -L ROOT /dev/vgroup-rsl/home]
[ DEBUG ]: 06:53:18 | EXEC: [mkdir -p /tmp/kiwi_volumes.qv23_f3x]
[ DEBUG ]: 06:53:18 | EXEC: [mountpoint /tmp/kiwi_volumes.qv23_f3x]
[ DEBUG ]: 06:53:18 | EXEC: [mount -o defaults
/dev/vgroup-rsl/LVRoot /tmp/kiwi_volumes.qv23_f3x]
[ DEBUG ]: 06:53:18 | EXEC: [mkdir -p /tmp/kiwi_volumes.qv23_f3x/home]
[ DEBUG ]: 06:53:18 | EXEC: [mountpoint /tmp/kiwi_volumes.qv23_f3x/home]
[ DEBUG ]: 06:53:18 | EXEC: [mount -o defaults /dev/vgroup-rsl/home
/tmp/kiwi_volumes.qv23_f3x/home]
[ DEBUG ]: 06:53:18 | EXEC: [mkdir -p /tmp/kiwi_volumes.qv23_f3x/misc]
[ DEBUG ]: 06:53:18 | EXEC: [mountpoint /tmp/kiwi_volumes.qv23_f3x/misc]
[ DEBUG ]: 06:53:18 | EXEC: [mount -o defaults /dev/vgroup-rsl/misc
/tmp/kiwi_volumes.qv23_f3x/misc]
[ DEBUG ]: 06:53:18 | EXEC: [mkdir -p /tmp/kiwi_volumes.qv23_f3x/opt]
[ DEBUG ]: 06:53:18 | EXEC: [mountpoint /tmp/kiwi_volumes.qv23_f3x/opt]
[ DEBUG ]: 06:53:18 | EXEC: [mount -o defaults /dev/vgroup-rsl/opt
/tmp/kiwi_volumes.qv23_f3x/opt]

So at least there are no errors.

Unfortunately, the mkisofs error is not ignored, causing the build to fail:

[ DEBUG ]: 07:26:08 | "mkisofs": in paths
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" exists:
"False" mode match: not checked
[ DEBUG ]: 07:26:08 | EXEC: [/usr/bin/genisoimage -hide header_end
-hide-joliet header_end -V "KIWI Installation System" -A 0xe37e56cf -R
-J -f -pad -joliet-long -sort /tmp/tmppd5lgn1m -no-emul-boot

-boot-load-size 4 -boot-info-table -hide boot/x86_64/boot.catalog
-hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o

/tmp/kiwi_build_20171018-19489-1osux4i.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso
/tmp/kiwi_build_20171018-19489-1osux4i.dice/kiwi_install_media.fmfqplva]
[ DEBUG ]: 07:26:51 | Relocated boot catalog from sector 0x3ce5 to 0x14
[ DEBUG ]: 07:26:51 | Fixed iso catalog contents
[ DEBUG ]: 07:26:51 | EXEC: [isohybrid --offset 31176 --id
0xe37e56cf --type 0x83 --uefi
/tmp/kiwi_build_20171018-19489-1osux4i.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
[ ERROR ]: 07:26:53 | KiwiCommandError: isohybrid: isohybrid:
Warning: more than 1024 cylinders: 1761

isohybrid: Not all BIOSes will be able to boot this device

Unlike a native build where the results are still available, dice
removes all the stuff. So there is nothing to check.

--
Roger Oberholtzer

[Roger Oberholtzer]

On Wed, Oct 18, 2017 at 10:02 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> If I want to have all temp stuff on other than /tmp, can I do that? My
>> /tmp has only 8.6GB free. It is on /root. Is my only option to mount
>> some larger partition as /tmp?
>
> when dice starts docker it shares /tmp and /dev from the host with
> the container:
>
> command = [
> "docker", "run",
> "--rm=true",
> "--entrypoint=sudo",
> "--privileged=true",
> "--name=#{container_name}",
> "-v", "#{recipe.basepath}:/vagrant",
> "-v", "/tmp:/tmp",
> "-v", "/dev:/dev",
> Dice::DOCKER_BUILD_CONTAINER,
> "bash", "-c", action
> ]
>
> Thus if you need more space in /tmp you can only copy all of /tmp
> to another bigger space and overmount this space to /tmp on your
> host... which should be considered a temporary "fix" but not the
> normal state of the system

I have made a larger /tmp. dice/docker was initially confused. I
stopped/started docker and then all was ok. So I am now building with
a much larger /tmp.


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> So at least there are no errors.
>
> Unfortunately, the mkisofs error is not ignored, causing the build to fail:
>

> [ ERROR ]: 07:26:53 | KiwiCommandError: isohybrid: isohybrid:
> Warning: more than 1024 cylinders: 1761
> isohybrid: Not all BIOSes will be able to boot this device

I'll look again into this isohybrid crap and fix it

> Unlike a native build where the results are still available, dice
> removes all the stuff. So there is nothing to check.

dice keeps all the information:

dice buildlog --show /path/to/your/image/description

[Roger Oberholtzer]

On Wed, Oct 18, 2017 at 10:22 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> So at least there are no errors.
>>
>> Unfortunately, the mkisofs error is not ignored, causing the build to fail:
>>
>> [ ERROR ]: 07:26:53 | KiwiCommandError: isohybrid: isohybrid:
>> Warning: more than 1024 cylinders: 1761
>> isohybrid: Not all BIOSes will be able to boot this device
>
> I'll look again into this isohybrid crap and fix it
>
>> Unlike a native build where the results are still available, dice
>> removes all the stuff. So there is nothing to check.
>
> dice keeps all the information:
>
> dice buildlog --show /path/to/your/image/description

All the build logs are kept. I meant the ISO image one could install.
When this error happens in a native build these files still exist. I
do not see them when dice fails. The log says:


[ INFO ]: 07:26:53 | Cleaning up InstallImageBuilder instance
[ DEBUG ]: 07:26:53 | EXEC: [rm -r -f
/tmp/kiwi_build_20171018-19489-1osux4i.dice/kiwi_install_media.fmfqplva]
[ DEBUG ]: 07:26:53 | EXEC: [rm -r -f
/tmp/kiwi_build_20171018-19489-1osux4i.dice/kiwi_install_squashfs.kfgp4zjk]
[ INFO ]: 07:26:53 | Cleaning up BootImageKiwi instance
[ DEBUG ]: 07:26:53 | EXEC: [rm -r -f
/tmp/kiwi_build_20171018-19489-1osux4i.dice/kiwi_boot_root.wifi7wwn]
[ DEBUG ]: 07:26:54 | EXEC: [rm -r -f /tmp/kiwi_boot_root_copy.xawpa0gw]
[ INFO ]: 07:26:54 | Cleaning up BootImageKiwi instance
[ DEBUG ]: 07:26:54 | EXEC: [rm -r -f
/tmp/kiwi_build_20171018-19489-1osux4i.dice/kiwi_boot_root.3cljz2y4]
[ INFO ]: 07:26:54 | Cleaning up BootImageDracut instance


I just meant that I cannot test that the LVM stuff actually does as I
expect because, although that part now works in the build, I do not
have an image I can test because of the isohybrid issue. So I cannot
provide feedback about that part.

I hope you don't get tired of me!

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I'll look again into this isohybrid crap and fix it

https://github.com/SUSE/kiwi/pull/517

open for review

[Marcus Schäfer]

Hi,

> All the build logs are kept. I meant the ISO image one could install.
> When this error happens in a native build these files still exist. I
> do not see them when dice fails. The log says:

correct only on success dice bundles the image result

> I just meant that I cannot test that the LVM stuff actually does as I
> expect because, although that part now works in the build, I do not
> have an image I can test because of the isohybrid issue. So I cannot
> provide feedback about that part.

right, but in the buildlog this looked good to me too

> I hope you don't get tired of me!

nah, not at all. All your feedback makes the code better. This is
really very much appreciated thanks for still digging into this
problems, we are close to success :)

Dealing with the iso tools makes my head spinning and yes sometimes
I get tired about this /*!%censored*/ tools, but surely not about
the people still using my stuff ;)

[Marcus Schäfer]

Hi,

> > I'll look again into this isohybrid crap and fix it
>
> https://github.com/SUSE/kiwi/pull/517

merged and submitted. A new container has landed on dockerhub.
Please retry your build

[Roger Oberholtzer]

On Wed, Oct 18, 2017 at 3:42 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> > I'll look again into this isohybrid crap and fix it
>>
>> https://github.com/SUSE/kiwi/pull/517
>
> merged and submitted. A new container has landed on dockerhub.
> Please retry your build

I can do so next Tuesday. I'm out of the office until then. I'll keep
you posted.




--
Roger Oberholtzer

[Roger Oberholtzer]

On Wed, Oct 18, 2017 at 3:42 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> > I'll look again into this isohybrid crap and fix it
>>
>> https://github.com/SUSE/kiwi/pull/517
>
> merged and submitted. A new container has landed on dockerhub.
> Please retry your build

I think we are getting very close...

[ DEBUG ]: 07:46:30 | "mkisofs": in paths

"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" exists:
"False" mode match: not checked

[ DEBUG ]: 07:46:30 | EXEC: [/usr/bin/genisoimage -hide header_end
-hide-joliet header_end -V "KIWI Installation System" -A 0x3fc3be8b -R
-J -f -pad -joliet-long -sort /tmp/tmp3mc66j13 -no-emul-boot

-boot-load-size 4 -boot-info-table -hide boot/x86_64/boot.catalog
-hide-joliet boot/x86_64/boot.catalog -b
boot/x86_64/loader/isolinux.bin -c boot/x86_64/boot.catalog
-eltorito-alt-boot -b boot/x86_64/efi -no-emul-boot -joliet-long
-boot-load-size 30720 -o

/tmp/kiwi_build_20171024-20197-1lxxx0h.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso
/tmp/kiwi_build_20171024-20197-1lxxx0h.dice/kiwi_install_media.9q8rl_hx]
[ DEBUG ]: 07:47:09 | Relocated boot catalog from sector 0x3ce5 to 0x14
[ DEBUG ]: 07:47:09 | Fixed iso catalog contents
[ DEBUG ]: 07:47:09 | EXEC: [isohybrid --offset 31176 --id
0x3fc3be8b --type 0x83 --uefi
/tmp/kiwi_build_20171024-20197-1lxxx0h.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
[ INFO ]: 07:47:11 | Cleaning up InstallImageBuilder instance
[ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
by kiwi: ['']

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I think we are getting very close...
>

> [ DEBUG ]: 07:47:09 | EXEC: [isohybrid --offset 31176 --id
> 0x3fc3be8b --type 0x83 --uefi
> /tmp/kiwi_build_20171024-20197-1lxxx0h.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
> [ INFO ]: 07:47:11 | Cleaning up InstallImageBuilder instance
> [ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
> by kiwi: ['']

The list looks empty, it should not lead to an error, mabe there is
a empty line in the error output or similar which leads to a list with
an empty string which is then not empty.

Bad bug. I'll fix it

[Marcus Schäfer]

Hi,

> > I think we are getting very close...
> >
> > [ DEBUG ]: 07:47:09 | EXEC: [isohybrid --offset 31176 --id
> > 0x3fc3be8b --type 0x83 --uefi
> > /tmp/kiwi_build_20171024-20197-1lxxx0h.dice/RSL-OEM-Leap-42.3.x86_64-42.3.2.install.iso]
> > [ INFO ]: 07:47:11 | Cleaning up InstallImageBuilder instance
> > [ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
> > by kiwi: ['']
>
> The list looks empty, it should not lead to an error, mabe there is
> a empty line in the error output or similar which leads to a list with
> an empty string which is then not empty.

Here is the hopefully last fix to address this issue:

https://github.com/SUSE/kiwi/pull/526

Once merged I'll update the container.

[Marcus Schäfer]

Hi,

> I think we are getting very close...
>

> [ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
> by kiwi: ['']

I have updated the container to address this issue. Please give
dice a new build. Thanks for your patience

I'll keep my fingers crossed :)

[Roger Oberholtzer]

On Thu, Oct 26, 2017 at 10:33 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I think we are getting very close...
>>
>> [ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
>> by kiwi: ['']
>
> I have updated the container to address this issue. Please give
> dice a new build. Thanks for your patience
>
> I'll keep my fingers crossed :)

It is pulling opensuse/dice:latest. It's now just to wait for my
slowish machine to complete the build. I will keep you posted.


--
Roger Oberholtzer

[Roger Oberholtzer]

On Thu, Oct 26, 2017 at 10:33 AM, Marcus Schäfer <m...@suse.de> wrote:

> Hi,
>
>> I think we are getting very close...
>>
>> [ ERROR ]: 07:47:11 | KiwiCommandError: isohybrid issue not ignored
>> by kiwi: ['']
>
> I have updated the container to address this issue. Please give
> dice a new build. Thanks for your patience
>
> I'll keep my fingers crossed :)

The build completed without errors! I am now testing that all is
working in the image. I will report back with those findings as well.
I am testing on a real system, just to be sure.


--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> The build completed without errors!

Hui :-) how good that sounds

Thanks for testing

[Roger Oberholtzer]

On Thu, Oct 26, 2017 at 2:40 PM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> The build completed without errors!
>
> Hui :-) how good that sounds
>
> Thanks for testing

The install resulted in something different than I had expected...

linux:~ # fdisk -l /dev/sda
Disk /dev/sda: 447.1 GiB, 480103981056 bytes, 937703088 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: BFBB99CA-245F-4A84-8E7C-0CA08F63400A

Device      Start       End   Sectors  Size Type
/dev/sda1    2048      6143      4096    2M BIOS boot
/dev/sda2    6144     47103     40960   20M EFI System
/dev/sda3   47104    661503    614400  300M Linux filesystem
/dev/sda4  661504 166545855 165884352 79.1G Microsoft basic data

There is one partition, which is 79.1G. All the volumes seem to be in that partition:

Filesystem                      Size  Used Avail Use% Mounted on
devtmpfs                        7.8G     0  7.8G   0% /dev
tmpfs                           7.8G     0  7.8G   0% /dev/shm
tmpfs                           7.8G  9.1M  7.8G   1% /run
tmpfs                           7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/vgroup--rsl-LVRoot  8.4G  6.1G  1.9G  77% /
/dev/sda3                       283M   45M  220M  17% /boot
/dev/sda2                        20M  748K   20M   4% /boot/efi
/dev/mapper/vgroup--rsl-misc    4.9G  3.0M  4.7G   1% /misc
/dev/mapper/vgroup--rsl-opt     5.3G  251M  4.7G   5% /opt
/dev/mapper/vgroup--rsl-home     29G  446K   27G   1% /home
tmpfs                           1.6G  4.0K  1.6G   1% /run/user/0

My config is this:

                <type image="oem" filesystem="ext4" boot="oemboot/suse-leap42.3" installiso="true" bootloader="grub2" kernelcmdline="splash" firmware="efi" initrd_system="dracut">

                        <oemconfig>
                                <oem-systemsize>49152</oem-systemsize>
                                <oem-boot-title>Ramboll Standard Linux 42.3</oem-boot-title>
                                <oem-swap>true</oem-swap>
                                <oem-recovery>false</oem-recovery>
                                <oem-device-filter>/dev/ram</oem-device-filter>
                                <!-- oem-partition-install>true</oem-partition-install -->
                        </oemconfig>

                        <systemdisk name="vgroup-rsl">
                                <volume name="opt" freespace="5G" mountpoint="opt"/>
                                <volume name="misc" freespace="5G" mountpoint="misc"/>
                                <volume name="home" freespace="all" mountpoint="home"/>
                        </systemdisk>

                </type>

I was expecting a / of 49 GB, a swap, /opt (5G), /misc (5G) and /home (the rest of the disk, which should be 300G or so).

So / and /home are much smaller than expected. I guess all are in /dev/sda4.

The /opt and /misc things were a test. But /home on the rest of the disk is what I am after.

Perhaps I am specifying this incorrectly?

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I was expecting a / of 49 GB, a swap, /opt (5G), /misc (5G) and /home
> (the rest of the disk, which should be 300G or so).
> So / and /home are much smaller than expected. I guess all are in
> /dev/sda4.
> The /opt and /misc things were a test. But /home on the rest of the
> disk is what I am after.
> Perhaps I am specifying this incorrectly?

/var/log/boot.kiwi from this machine tells us the truth

[Roger Oberholtzer]

On Thu, Oct 26, 2017 at 3:05 PM, Marcus Schäfer <m...@suse.de> wrote:

Hi,

>    I was expecting a / of 49 GB, a swap, /opt (5G), /misc (5G) and /home
>    (the rest of the disk, which should be 300G or so).
>    So / and /home are much smaller than expected. I guess all are in
>    /dev/sda4.
>    The /opt and /misc things were a test. But /home on the rest of the
>    disk is what I am after.
>    Perhaps I am specifying this incorrectly?

/var/log/boot.kiwi from this machine tells us the truth

I have attached it. I see this at the top:

KIWI .profile contents:

kiwi_Volume_1='opt|freespace:5120|opt'

kiwi_Volume_2='misc|freespace:5120|misc'

kiwi_Volume_3='home|size:all|home'

kiwi_Volume_4='LVRoot|freespace:30|'

What I do not understand is how one gets each volume in a partition. Which is what I want. All but /home can have a fixed size. /home should fill the remaining disk after the other partitions are made.

 

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> /var/log/boot.kiwi from this machine tells us the truth

So you set a fixed root size with

+ kiwi_oemrootMB='49152'

but you have not specified any value for swap. Thus kiwi uses
twice times the ramsize which is on this system:

+ mem_size=16299048 kB
+ swapsize=31834 MB

So there is only 49152 MB - 31834 MB = 17318 MB available for
your volumes. Within that range kiwi does what you told him

+ lvextend -L +5120M /dev/vgroup-rsl/opt
+ lvextend -L +5120M /dev/vgroup-rsl/misc
+ lvextend -L +30M /dev/vgroup-rsl/LVRoot
+ lvextend -l +100%FREE /dev/vgroup-rsl/home

==> round about: 7048 MB for home

I recommend to setup a fixed value for swap as you never
know how much ram is on the target:

<oem-swapsize>value-in-MB</oem-swapsize>

[Roger Oberholtzer]

On Fri, Oct 27, 2017 at 11:27 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> /var/log/boot.kiwi from this machine tells us the truth
>
> So you set a fixed root size with
>
> + kiwi_oemrootMB='49152'
>
> but you have not specified any value for swap. Thus kiwi uses
> twice times the ramsize which is on this system:
>
> + mem_size=16299048 kB
> + swapsize=31834 MB
>
> So there is only 49152 MB - 31834 MB = 17318 MB available for
> your volumes. Within that range kiwi does what you told him
>
> + lvextend -L +5120M /dev/vgroup-rsl/opt
> + lvextend -L +5120M /dev/vgroup-rsl/misc
> + lvextend -L +30M /dev/vgroup-rsl/LVRoot
> + lvextend -l +100%FREE /dev/vgroup-rsl/home
>
> ==> round about: 7048 MB for home
>
> I recommend to setup a fixed value for swap as you never
> know how much ram is on the target:
>
> <oem-swapsize>value-in-MB</oem-swapsize>

Before I added this:

<systemdisk name="vgroup-rsl">
<volume name="opt" freespace="5G"
mountpoint="opt"/>
<volume name="misc" freespace="5G"
mountpoint="misc"/>
<volume name="home" freespace="all"
mountpoint="home"/>
</systemdisk>

/ was 48 G in one partition, and swap was 32 GB (twice ram) in another
partition. After I added this, all is now in one partition.

All I am really trying to accomplish is having /home in a separate
partition that fills whatever space is left on the disk.

Perhaps I might be better trying to get my old python script that made
the partitions, formatted and mounted them to work again. That script
was not very reliable. I suspect because of changes in how the
programs it used reports things. I guess you know all about that :)

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Before I added this:
>
> <systemdisk name="vgroup-rsl">
> <volume name="opt" freespace="5G"
> mountpoint="opt"/>
> <volume name="misc" freespace="5G"
> mountpoint="misc"/>
> <volume name="home" freespace="all"
> mountpoint="home"/>
> </systemdisk>
>
> / was 48 G in one partition, and swap was 32 GB (twice ram) in another
> partition. After I added this, all is now in one partition.

well before you added this you did not use any volume manager
kiwi supports custom volumes through lvm or by the native components
of the filesystem (e.g btrfs volume management) but I intentionally
do not support custom volume setup based on partition tables
and their horrible limitations.

So yes all of lvm is in a LVM (8e) partition and inside of that
geometry you can create lvm partitions and manage them in a nice
and safe way... what's wrong with it ? why do you want an
extra home partition outside of lvm control, that doesn't make
any sense to me

Other than that you can still have it

- move <volume name="home"/> out of the systemdisk block
- make sure you have setup <oem-systemsize> to be smaller than the
target disk. This will leave space unpartitioned
- write a one shot systemd unit script which does once at boot:

a) creates a home partition with fdisk or gpart depending on the
table type
b) create a filesystem on the partition
c) add it to fstab
d) move all of /home from disk to partition

That works but don't cry if /home runs out of space and there is no
nice way to enlarge it ;)

[Roger Oberholtzer]

On Fri, Oct 27, 2017 at 11:59 AM, Marcus Schäfer <m...@suse.de> wrote:

> So yes all of lvm is in a LVM (8e) partition and inside of that
> geometry you can create lvm partitions and manage them in a nice
> and safe way... what's wrong with it ? why do you want an
> extra home partition outside of lvm control, that doesn't make
> any sense to me

We run these systems in a vehicle on the road, collecting obscene
amounts of data at high speed. Although the power is very reliable,
things can happen. So we like to try to keep the various parts as
separate as we can. If the root partition gets messed up, perhaps we
can still access the data on /home. Having everything in one physical
volume feels like, if something goes wrong, all might be lost. That is
the main reason for this.

> Other than that you can still have it
>
> - move <volume name="home"/> out of the systemdisk block
> - make sure you have setup <oem-systemsize> to be smaller than the
> target disk. This will leave space unpartitioned
> - write a one shot systemd unit script which does once at boot:
>
> a) creates a home partition with fdisk or gpart depending on the
> table type
> b) create a filesystem on the partition
> c) add it to fstab
> d) move all of /home from disk to partition

This is what our Python script did. It was also a one-time thing that
only happened when the system was first booted after installation.

> That works but don't cry if /home runs out of space and there is no
> nice way to enlarge it ;)

That is typically not the problem. We just want whatever /home there
is to be as robust as possible.

--
Roger Oberholtzer