problem installing an RPM in a build

[Roger Oberholtzer]

I am setting up a couple software build environments. I have to say
that using kiwi/buildbox for making the tar images for this is
fantastic!

For the most part it is working. But, alas, I have a problem.

I want to add the NVIDIA CUDA NVCC compiler to the result. The bits come from

http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64

I use these on openSUSE (Tumbleweed and Leap) and they have been working great.

When I try to install in kiwi/buildbox, I get the following:


[ ERROR ]: 09:41:59 | KiwiBootStrapPhaseFailed: Bootstrap package
installation failed: Installation of
cuda-license-10-1-10.1.243-1.x86_64 failed:
Error: Subprocess failed. Error: RPM failed: warning:
/result/build/image-root/var/cache/kiwi/packages/NVIDIA
CUDA/cuda-license-10-1-10.1.243-1.x86_64.rpm: Header V4 RSA/SHA512
Signature, key ID 7fa2af80: NOKEY
/bin/sh: error while loading shared libraries: libreadline.so.7:
cannot open shared object file: No such file or directory
error: %prein(cuda-license-10-1-10.1.243-1.x86_64) scriptlet failed,
exit status 127
error: cuda-license-10-1-10.1.243-1.x86_64: install failed

I suspect it is because the cuda-license rpm wants to show me
something that I have to accept. Not sure.

Any ideas on what this might be?

The specific package that I am wanting to install is cuda-nvcc-10-2,
and I guess it pulls in the cuda-license package.

I am building for Leap 15.2 on a Tumbleweed host, using buildbox.


--
Roger Oberholtzer

[Roger Oberholtzer]

I bit more information. The error is happening when the repository is
added. It is not when adding a package from the repo. I know this
because I removed adding the package, but left the repo directive in
place, and the error occurred.

Or so it seems.

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> > [ ERROR ]: 09:41:59 | KiwiBootStrapPhaseFailed: Bootstrap package
> > installation failed: Installation of
> > cuda-license-10-1-10.1.243-1.x86_64 failed:
> > Error: Subprocess failed. Error: RPM failed: warning:
> > /result/build/image-root/var/cache/kiwi/packages/NVIDIA
> > CUDA/cuda-license-10-1-10.1.243-1.x86_64.rpm: Header V4 RSA/SHA512
> > Signature, key ID 7fa2af80: NOKEY
> > /bin/sh: error while loading shared libraries: libreadline.so.7:
> > cannot open shared object file: No such file or directory
> > error: %prein(cuda-license-10-1-10.1.243-1.x86_64) scriptlet failed,
> > exit status 127
> > error: cuda-license-10-1-10.1.243-1.x86_64: install failed
> >
> > I suspect it is because the cuda-license rpm wants to show me
> > something that I have to accept. Not sure.
> >
> > Any ideas on what this might be?
> >
> > The specific package that I am wanting to install is cuda-nvcc-10-2,
> > and I guess it pulls in the cuda-license package.
> >
> > I am building for Leap 15.2 on a Tumbleweed host, using buildbox.
>
> I bit more information. The error is happening when the repository is
> added.

That I don't believe because the above error makes it pretty
clear that the pre-install script of cuda-license calls something
that needs libreadline.so.7 which does not exist at that time
probably because of a wrong pre-req config of the package

I suggest you do the following:

Place all of the nvidia packages into:

<packages type="image"/>
<package name="nvidia..."/>
</packages>

Don't place anything in the bootstrap section that can also
be installed as part of the image(chrooted) install section

In general the bootstrap section should be as small as possible.
As the name says it's bootstrap meaning it uses the package
manager on the host to get to a minimal system from which
chroot works

Again this is not a boxbuild issue, you should see the same
problem when building from a host that does not provide
the needed libraries

Let me know if that helps

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Software Solutions Germany GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Roger Oberholtzer]

The packages to install are in the type="image" clause. Not the
bootstrap clause. But they are disabled at this point.

The thing is, it is not the package that is causing this. It is the
repository definition.

Adding this causes the error:

<repository type="rpm-md" priority="97" imageinclude="true"
alias="NVIDIA CUDA">
<source
path="http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64"/>
</repository>

When in the bootstrap phase, something is wanting to install the
cuda-license-10-1-10.1.243-1.x86_64 package. I make no reference to it
or any packages in the repo. I only add the repo. This is enough to
cause the problem.

> Don't place anything in the bootstrap section that can also
> be installed as part of the image(chrooted) install section
>
> In general the bootstrap section should be as small as possible.
> As the name says it's bootstrap meaning it uses the package
> manager on the host to get to a minimal system from which
> chroot works
>
> Again this is not a boxbuild issue, you should see the same
> problem when building from a host that does not provide
> the needed libraries

Perhaps not a buildbox issue. But it only happens in kiwi. As far as I
can tell, anyway.

Note that the NVIDIA display driver repo works fine in kiwi/buildbox.
The NVIDIA development is a different repo.

--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> The thing is, it is not the package that is causing this. It is the
> repository definition.
>
> Adding this causes the error:
>
> <repository type="rpm-md" priority="97" imageinclude="true"
> alias="NVIDIA CUDA">
> <source
> path="http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64"/>
> </repository>
>
> When in the bootstrap phase, something is wanting to install the
> cuda-license-10-1-10.1.243-1.x86_64 package. I make no reference to it
> or any packages in the repo. I only add the repo. This is enough to
> cause the problem.

The package dependency resolver will take the repo into account
and probably fetches packages from it. We should find them out.

On your host install:

zypper in python3-solv

Now call kiwi as follows:

kiwi-ng image info \
--resolve-package-list --description /path/to/your/image/description

The result lists the dependency resolution, e.g

"xz": {
"arch": "x86_64",
"installsize_bytes": 326378,
"source": "obs://openSUSE:Leap:15.2/standard",
"status": "added_by_dependency_solver",
"version": "5.2.3-lp152.5.100"
}

Check in the output which packages are fetched from the nvidia repo.
From there we can check why

> Perhaps not a buildbox issue. But it only happens in kiwi. As far as I
> can tell, anyway.

kiwi is just a user of repos at this point. But kiwi starts with an
empty root-dir whereas all other users of http://developer.download.nvidia.com
install stuff in already installed systems. The error you saw mostly
exposes packaging errors that strikes back under certain conditions

> Note that the NVIDIA display driver repo works fine in kiwi/buildbox.
> The NVIDIA development is a different repo.

yes I'm aware of it and it's not the first time that the packages in the
developer repo have room for improvement

[Roger Oberholtzer]

Hmmm.. If I only define the repo, but no packages from that repo, I do
not see anything being fetched. In this mode, the error happens when
building.

If I enable the package "cuda-nvcc-10-2", then I see this:

"cuda-license-10-2": {
"arch": "x86_64",
"installsize_bytes": 60999,
"source":
"http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64",
"status": "added_by_dependency_solver",
"version": "10.2.89-1"
},
"cuda-misc-headers-10-2": {
"arch": "x86_64",
"installsize_bytes": 15752684,
"source":
"http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64",
"status": "added_by_dependency_solver",
"version": "10.2.89-1"
},
"cuda-nvcc-10-2": {
"arch": "x86_64",
"installsize_bytes": 120500541,
"source":
"http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64",
"status": "listed_in_kiwi_description",
"version": "10.2.89-1"
},

Looks reasonable.

The first one is the one that gets complained about when only the repo
is defined.


Maybe there is some stale repo info in the build environment that is
causing this? Can I reset the repo info for these?



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Hmmm.. If I only define the repo, but no packages from that repo, I do
> not see anything being fetched.

But in this case there should also be no error, I'm confused

And in this case you also end up with the same error right ?

So what happens if you set

<packages type="image">
<package name="cuda-nvcc-10-2"/>
...other packages
</packages>

<packages type="bootstrap">
<package name="libreadline7"/>
...other bootstrap packages
</packages>

This should ensure libreadline7 exists prior install of cuda-nvcc-10-2

> Maybe there is some stale repo info in the build environment that is
> causing this? Can I reset the repo info for these?

If you build in the box this cannot happen because the box builds
in kvm snapshot mode and always starts from a clean state unless
you specify --no-snapshot

[Roger Oberholtzer]

This did not help. I get this:

[ INFO ]: 08:06:32 | Installing bootstrap packages
[ INFO ]: 08:06:32 | --> collection type: onlyRequired
[ INFO ]: 08:06:32 | --> package: ca-certificates
[ INFO ]: 08:06:32 | --> package: cracklib-dict-full
[ INFO ]: 08:06:32 | --> package: filesystem
[ INFO ]: 08:06:32 | --> package: glibc-locale
[ INFO ]: 08:06:32 | --> package: libreadline7
[ INFO ]: 08:06:32 | --> package: openSUSE-release
[ INFO ]: 08:06:32 | --> package: udev
[ INFO ]: 08:06:32 | --> package: zypper
[ INFO ]: Processing: [############### ]
38%[ 52.040804] random: crng init done
[ 52.041095] random: 7 urandom warning(s) missed due to ratelimiting
[ INFO ]: Processing: [########################################] 100%
[ ERROR ]: 08:07:08 | KiwiBootStrapPhaseFailed: Bootstrap package

installation failed: Installation of
cuda-license-10-1-10.1.243-1.x86_64 failed:
Error: Subprocess failed. Error: RPM failed: warning:
/result/build/image-root/var/cache/kiwi/packages/NVIDIA
CUDA/cuda-license-10-1-10.1.243-1.x86_64.rpm: Header V4 RSA/SHA512
Signature, key ID 7fa2af80: NOKEY
/bin/sh: error while loading shared libraries: libreadline.so.7:
cannot open shared object file: No such file or directory
error: %prein(cuda-license-10-1-10.1.243-1.x86_64) scriptlet failed,
exit status 127
error: cuda-license-10-1-10.1.243-1.x86_64: install failed

Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
Please see the above error message for a hint.

[ INFO ]: 08:07:08 | Cleaning up SystemPrepare instance




And as I said, I'm not installing any packages from here. It is just
the definition of the repository that causes this error.



--
Roger Oberholtzer

[Marcus Schäfer]

Hi,

What in god's name pulls in cuda-license from this bootstrap list ??
If you build via boxbuild there is a log file in the directory
you set as --target-dir. The log file is named

result.log

Can you attach this log file here ?

> And as I said, I'm not installing any packages from here. It is just
> the definition of the repository that causes this error.

That's very very strange

[Marcus Schäfer]

Hi Roger,

Do you have any news regarding this one ?

Thanks

[Roger Oberholtzer]

No. I have disabled trying to have this in the build.  I would like to sort it out, but to be honest, I don't know where the actual problem is.

I am doing this in a build that makes a tgz that we use to build our software. The idea is similar to boxbuild(in concept)  in that we want a controlled defined environment (actually a number of them) that we can use to build for any platform (including windows, mingw, which works perfectly via the packages on OBS). In that we want all tools that were current at the time. And that includes the nvidia cuda compiler. We really do not want to have to hunt anything down later or hope that each user installs the same version, etc.

I just don't see how adding a repo causes this. Unless I am being misled about where the kiwi build really is when this error occurs.

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> No. I have disabled trying to have this in the build.

Hmm, never give up ;)

I have modified the Leap 15.1 example from the kiwi-descriptions
repo and added:

<repository type="rpm-md" alias="nvidia">

<source path="http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64"/>
</repository>

<package name="cuda-nvcc-10-2"/>

See the attached description tarball

Next I build this using boxbuild:

kiwi-ng --profile Virtual system boxbuild --box suse -- --description /home/ms/Project/kiwi-descriptions/suse/x86_64/suse-leap-15.1 --target-dir /tmp/mytest

and it just builds. The build log shows:

DEBUG: 09:42:40 | system: Retrieving package cuda-license-10-2-10.2.89-1.x86_64 (183/188), 22.0 KiB ( 59.6 KiB unpacked)
...
DEBUG: 09:43:42 | system: (183/188) Installing: cuda-license-10-2-10.2.89-1.x86_64 [......done]

So this all works for me. Maybe the attached image description
shows differences to what you used.

I can't believe that it should end like this

[Roger Oberholtzer]

On Fri, Nov 27, 2020 at 10:49 AM Marcus Schäfer <m...@suse.de> wrote:

Hi,

>    No. I have disabled trying to have this in the build.

Hmm, never give up ;)

I have modified the Leap 15.1 example from the kiwi-descriptions
repo and added:

    <repository type="rpm-md" alias="nvidia">
        <source path="http://developer.download.nvidia.com/compute/cuda/repos/opensuse15/x86_64"/>
    </repository>

    <package name="cuda-nvcc-10-2"/>

See the attached description tarball

Next I build this using boxbuild:

    kiwi-ng --profile Virtual system boxbuild --box suse -- --description /home/ms/Project/kiwi-descriptions/suse/x86_64/suse-leap-15.1 --target-dir /tmp/mytest

and it just builds. The build log shows:

    DEBUG: 09:42:40 | system: Retrieving package cuda-license-10-2-10.2.89-1.x86_64 (183/188),  22.0 KiB ( 59.6 KiB unpacked)
    ...
    DEBUG: 09:43:42 | system: (183/188) Installing: cuda-license-10-2-10.2.89-1.x86_64 [......done]

So this all works for me. Maybe the attached image description
shows differences to what you used.

I am building a different image type than is in your file:

    <preferences>
        <type image="tbz"/>
        <version>1.0.0</version>
        <packagemanager>zypper</packagemanager>
        <locale>en_US</locale>
        <keytable>us</keytable>
        <timezone>Europe/Stockholm</timezone>
        <rpm-excludedocs>true</rpm-excludedocs>
        <rpm-check-signatures>false</rpm-check-signatures>
    </preferences>

Could it be something that happens in the tbz image?

It is the only type defined in my file.

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> I am building a different image type than is in your file:
> <preferences>
> <type image="tbz"/>
> <version>1.0.0</version>
> <packagemanager>zypper</packagemanager>
> <locale>en_US</locale>
> <keytable>us</keytable>
> <timezone>Europe/Stockholm</timezone>
> <rpm-excludedocs>true</rpm-excludedocs>
> <rpm-check-signatures>false</rpm-check-signatures>
> </preferences>
> Could it be something that happens in the tbz image?

No. The issue you see is at the time packages gets installed. In kiwi
this is the prepare step. This step is the same for all image types.
The different processing to create an image type starts after the
new root tree (the prepare step) has been done

> It is the only type defined in my file.

I also tested with your type, as expected that doesn't make any
difference.

I was more thinking on a different repo setup that you use compared
to the example I attached in my previous mail. So if you could just
add your image type spec from above to the XML description I sent
and give the boxbuild a try and see if that works it would be
imho a good step to make sure things works for you in the same
way they work for me.

Thanks

[Roger Oberholtzer]

I added the TBZ target to your file, used that, and it built ok.

    <preferences>
        <type image="tbz"/>

    </preferences>

So, I thought I would then just change the references to 15.1 to be 15.2, since that is really where I am trying to build this.

These are the changes:

    <preferences>
        <type image="tbz"/>

    </preferences>
    <repository type="rpm-md" alias="kiwi" priority="1">
        <source path="obs://Virtualization:Appliances:Builder/openSUSE_Leap_15.2"/>
    </repository>
    <repository type="rpm-md" alias="Leap_15_2" imageinclude="true">
        <source path="obs://openSUSE:Leap:15.2/standard"/>
    </repository>

Then I got an unexpected error (tried more than once):

INFO: 13:32:09 | Importing Image description to system tree
INFO: 13:32:09 | --> Importing state XML description to /result/build/image-root/image/config.xml
DEBUG: 13:32:09 | EXEC: [mkdir -p /result/build/image-root/image]
INFO: 13:32:09 | --> Importing config.sh script to /result/build/image-root/image/config.sh
DEBUG: 13:32:09 | EXEC: [cp /description/config.sh /result/build/image-root/image/config.sh]
INFO: 13:32:09 | --> Importing script helper functions
DEBUG: 13:32:09 | EXEC: [cp /usr/lib/python3.6/site-packages/kiwi/config/functions.sh /result/build/image-root/.kconfig]
INFO: 13:32:09 | Copying user defined files to image tree
DEBUG: 13:32:09 | EXEC: [rsync -r -p -t -D -H -X -A --one-file-system --links /description/root/ /result/build/image-root]
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root grep -q ^root: /etc/group]
INFO: 13:32:09 | Setting up user root
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root grep -q ^root: /etc/passwd]
INFO: 13:32:09 | --> Modifying user: root
INFO: 13:32:09 | --> Primary group for user root: root
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root usermod -p $1$wYJUgpM5$RXMMeASDc035eX.NbYWFl0 -g root root]
INFO: 13:32:09 | Setting up keytable: us
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root systemd-firstboot --help]
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root systemd-firstboot --keymap=us]
INFO: 13:32:09 | Setting up locale: en_US
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root systemd-firstboot --help]
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root systemd-firstboot --locale=en_US.UTF-8]
DEBUG: 13:32:09 | EXEC: [chroot /result/build/image-root plymouth-set-default-theme openSUSE]
DEBUG: 13:32:09 | EXEC: Failed with stderr: /usr/share/plymouth/themes/openSUSE/openSUSE.plymouth does not exist
, stdout: (no output on stdout)
ERROR: 13:32:09 | KiwiCommandError: chroot: stderr: /usr/share/plymouth/themes/openSUSE/openSUSE.plymouth does not exist
, stdout: (no output on stdout)
INFO: 13:32:09 | Cleaning up SystemPrepare instance
DEBUG: 13:32:09 | EXEC: [mountpoint -q /result/build/image-root/var/cache/kiwi]
DEBUG: 13:32:09 | EXEC: [mountpoint -q /result/build/image-root/var/cache/kiwi]
DEBUG: 13:32:09 | EXEC: [umount -l /result/build/image-root/var/cache/kiwi]

So, something different than what we are looking at.

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Then I got an unexpected error (tried more than once):

yeah this is unrelated to the problem we discuss here. SUSE has
deleted the openSUSE plymouth theme and switched to bgrt from
upstream. Just look here:

https://github.com/OSInside/kiwi-descriptions/blob/master/suse/x86_64/suse-leap-15.2/config.xml

See the settings <bootsplash-theme>bgrt</bootsplash-theme>
and package name="plymouth-theme-bgrt"/>

> /usr/share/plymouth/themes/openSUSE/openSUSE.plymouth does not exist

yep, it does not exist anymore :)

[Roger Oberholtzer]

On Fri, Nov 27, 2020 at 3:26 PM Marcus Schäfer <m...@suse.de> wrote:

Hi,

>    Then I got an unexpected error (tried more than once):

yeah this is unrelated to the problem we discuss here. SUSE has
deleted the openSUSE plymouth theme and switched to bgrt from
upstream. Just look here:

    https://github.com/OSInside/kiwi-descriptions/blob/master/suse/x86_64/suse-leap-15.2/config.xml

See the settings <bootsplash-theme>bgrt</bootsplash-theme>
and package name="plymouth-theme-bgrt"/>

>    /usr/share/plymouth/themes/openSUSE/openSUSE.plymouth does not exist

yep, it does not exist anymore :)

Okay. I have made these changes and it is building for 15.2

Next I will add my various repos to see if it is one of those that is causing the problem.

--

Roger Oberholtzer

[Marcus Schäfer]

Hi,

> Okay. I have made these changes and it is building for 15.2
> Next I will add my various repos to see if it is one of those that is
> causing the problem.

Great, thanks for your patience. We will find this issue :)

Have a good weekend