How to mimic a repo with URI plugin:/susecloud ...

[rick.shaw]

Greetings,

I wish to build a docker image for SLES15 SP2 ARM using an AWS ARM SLES15 SP2 instance as a kiwi-ng host. The example is:

ip-10-90-127-146 TD_descriptions/arm_sles15.2# zypper lr 13

Alias          : Desktop_Applications_Module_aarch64:SLE-Module-Desktop-Applications15-SP2-Pool

Name           : SLE-Module-Desktop-Applications15-SP2-Pool

URI            : plugin:/susecloud?credentials=Desktop_Applications_Module_aarch64&path=/repo/SUSE/Products/SLE-Module-Desktop-Applications/15-SP2/aarch64/product/

Enabled        : Yes

GPG Check      : (r ) Yes

Priority       : 99 (default priority)

Autorefresh    : Off

Keep Packages  : Off

Type           : rpm-md

GPG Key URI    : 

Path Prefix    : 

Parent Service : Desktop_Applications_Module_aarch64

Keywords       : pool

Repo Info Path : /etc/zypp/repos.d/Desktop_Applications_Module_aarch64:SLE-Module-Desktop-Applications15-SP2-Pool.repo

MD Cache Path  : /var/cache/zypp/raw/Desktop_Applications_Module_aarch64:SLE-Module-Desktop-Applications15-SP2-Pool

ip-10-90-127-146 TD_descriptions/arm_sles15.2# 

I want to leverage those SLES repos that are named in that instance by replicating them similar to:

  <repository priority="98" imageinclude="true" type="rpm-md" alias="Development_Tools_Module_aarch64:SLE-Module-DevTools15-SP2-Pool">

    <source path="plugin:/susecloud?credentials=Development_Tools_Module_aarch64&path=/repo/SUSE/Products/SLE-Module-Development-Tools/15-SP2/aarch64/product/"/>

  </repository>

When I try a kiwi-ng run in that context, I get:

[ INFO    ]: 01:46:46 | Reading runtime config file: '/etc/kiwi.yml'

[ INFO    ]: 01:46:46 | Loading XML description

[ INFO    ]: 01:46:46 | Support for XML markup available

[ ERROR   ]: 01:46:46 | KiwiConfigFileFormatNotSupported: Support for non-XML formatted config files requires the Python anymarkup module.

Is there some way to leverage these plugin:susecloud repos?

[Marcus Schäfer]

Hi,

> Is there some way to leverage these plugin:susecloud repos?

Your question is basically if you can use the SLE repos from
a registered SLE system to build an image with kiwi on that host
making use of that repos.

Yes you can do that but the repository definition should look
different. In my opinion the following should work:

<repository priority="98" username="NAME" password="CREDENTIALS">
<source path="SOURCE"/>
</repository>

This is the basic repo definition

Now on your system look at two places:

1. /etc/zypp/repos.d/Desktop_Applications_Module_aarch64:SLE-Module-Desktop-Applications15-SP2-Pool.repo

Take out the information that is listed in baseurl="SOURCE"
and fill in the gap of the repo definition

2. /etc/zypp/credentials.d/...

Look at this directory and search for "Desktop_Applications_Module_aarch64"
I'm not sure if it's a single file or part of a file. But you will
find credentials information for this module in this place.
Take out the credentials and fill in the gaps for
NAME and CREDENTIALS

This also binds the process to this host

Notes !:

* imageinclude is not a good idea, because the repos will only
work on this host and in your image you need a new subscription
registration process at some point

* If you build with kiwi you should put all <packages> into the
type="bootstrap" section and have no packages in the type="image"
section. The reason is that access to the repos will be handled
through the "susecloud" zypper plugin and this is not going to
work from within a chroot system. In a kiwi run the second
phsae of installing packages happens in the chroot(image-root)
and I expect problems if you do so when using this repos through
the "susecloud" plugin. If you put all packages in the bootstrap
section the install of packages will happen by the zypper call
in the first phase of the install process which is not chrooted

Overall, what you are doing is possible but comes with a price
to pay and some hurdles. There was some effort done in the "susecloud"
plugin and the way to access SLE repos to avoid abuse of them.
This makes it hard for kiwi to "just" consume them and as I wrote
it only works with a tight coupling to the already registered system.

From my opensource heart this makes things more complex that I
think they should be. But on the enterprise and the business world
this all makes sense.

The easiest way to access SLE repos for a kiwi build is:

1. Build in the open build service, yes you have access to SLES
on build.opensuse.org. If you are interested in that part I
can give you an example

2. Create a local RMT server and point your kiwi build to it
https://documentation.suse.com/sles/15-SP1/single-html/SLES-rmt/

Hope all this helps to clarify the situation.

Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Software Solutions Germany GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------

[Robert Schweikert]

Hi,

On 5/7/21 3:43 AM, Marcus Schäfer wrote:
> Hi,
>
>> Is there some way to leverage these plugin:susecloud repos?
>
> Your question is basically if you can use the SLE repos from
> a registered SLE system to build an image with kiwi on that host
> making use of that repos.
>
> Yes you can do that but the repository definition should look
> different. In my opinion the following should work:
>
> <repository priority="98" username="NAME" password="CREDENTIALS">
> <source path="SOURCE"/>
> </repository>
>
> This is the basic repo definition
>
> Now on your system look at two places:
>
> 1. /etc/zypp/repos.d/Desktop_Applications_Module_aarch64:SLE-Module-Desktop-Applications15-SP2-Pool.repo
>
> Take out the information that is listed in baseurl="SOURCE"
> and fill in the gap of the repo definition
>
> 2. /etc/zypp/credentials.d/...
>
> Look at this directory and search for "Desktop_Applications_Module_aarch64"
> I'm not sure if it's a single file or part of a file. But you will
> find credentials information for this module in this place.
> Take out the credentials and fill in the gaps for
> NAME and CREDENTIALS
>
> This also binds the process to this host

And this will only work if the instance you are using has recently
visited the update infrastructure and is in the cache, otherwise your
build will fall over.

If you build your container with all packages specified in the
<bootstrap> section, if that is possible I do not know, then kiwi will
use zypper from the host which will run the plugin that resolves the URL
and sends the proper information to the update infrastructure.

For builds with docker files we have containerbuild-regionsrv, i.e.

systemctl start containerbuild-regionsrv.service

>
> Notes !:
>
> * imageinclude is not a good idea, because the repos will only
> work on this host and in your image you need a new subscription
> registration process at some point
>
> * If you build with kiwi you should put all <packages> into the
> type="bootstrap" section and have no packages in the type="image"
> section.

Look and here is the answer to my question :D . So if you do this you
should not run into any issue because kiwi will use zypper on the host
where you building and zypper will in turn use the URL resolver provided
by cloud-regionsrv-client to resolve "plugin:susecloud"

Later,
Robert

Robert Schweikert MAY THE SOURCE BE WITH YOU
Distinguished Engineer LINUX
Technical Team Lead Public Cloud
rjsc...@suse.com
IRC: robjo

[rick.shaw]

Thanks for the explanations and suggestions; they are helpful in my understanding.

Just to see if I could get it to work, I modified the repository definition to the following.
FYI: I had to change imageinclude to false because kiwi complained about the URI otherwise. I no longer have that output, but it was something about that URI not being viable in a container.

<repository priority="98" imageinclude="false" type="rpm-md" alias="Development_Tools_Module_aarch64:SLE-Module-DevTools15-SP2-Pool" username="-redacted-" password="-redacted-">

  <source path="plugin:/susecloud?path=/repo/SUSE/Products/SLE-Module-Development-Tools/15-SP2/aarch64/product/"/>

</repository>

When I tried a run with this change, it failed with a different problem:

[ INFO    ]: 21:36:36 | Reading runtime config file: '/etc/kiwi.yml'

[ INFO    ]: 21:36:36 | Loading XML description

[ INFO    ]: 21:36:36 | Support for XML markup available

[ INFO    ]: 21:36:36 | --> loaded /tmp/kiwi.2430/config.xml

[ INFO    ]: 21:36:36 | --> Selected build type: docker

[ INFO    ]: 21:36:36 | Preparing new root system

[ INFO    ]: 21:36:36 | Setup root directory: /data/arm_sles15sp2/build/image-root

[ INFO    ]: 21:36:36 | Setting up repository plugin:/susecloud?path=/repo/SUSE/Products/SLE-Module-Development-Tools/15-SP2/aarch64/product/

[ INFO    ]: 21:36:36 | --> Type: rpm-md

[ INFO    ]: 21:36:36 | --> Priority: 98

[ ERROR   ]: 21:36:36 | KiwiUriStyleUnknown: URI schema plugin:/susecloud?path=/repo/SUSE/Products/SLE-Module-Development-Tools/15-SP2/aarch64/product/ not supported

[ INFO    ]: 21:36:36 | Cleaning up SystemPrepare instance

I had already decided to try to move away from the plugin:/susecloud URI style, but this made it final for me.

I would like to pursue the  open build service method that Marcus suggested, stating that he would provide an example if I was interested.

Marcus, could you please elaborate and provide an example of how I could get to SLE repos through OBS?

Thanks

[rick.shaw]

Marcus,

In my previous post, I asked if you could  please elaborate and provide an example of how I could get to SLE repos through OBS. Could you, please?

Thanks a lot,

-Rick

[Marcus Schäfer]

Hi Rick,

> In my previous post, I asked if you could please elaborate and provide
> an example of how I could get to SLE repos through OBS. Could you,
> please?

yep, sorry this one felt off the plate.

Please see the following example project in OBS:

https://build.opensuse.org/project/show/Virtualization:Appliances:Images:Testing_x86:sle15

Also take a look at the:

https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing_x86:sle15/meta

https://build.opensuse.org/projects/Virtualization:Appliances:Images:Testing_x86:sle15/prjconf

The contents of the prjconf and the meta data creates the
connection to the SLES repos. In the kiwi XML description you
only see a repo definition that says "obsrepositories". With
this concept you tell kiwi to use the repos as OBS provides
them.

That way you can build images for SLES.

Please note when you use such an image it will not be registered
against SLES. For this you still need a SLES subscription to
benefit from updates while the system runs

For you to build your own images you need to create an account
on obs first, just in case you don't have one

Hope that helps