Aptly distribution and component structure

[Frederic NAUDEAU]

Hi,

I'm new to aptly and I'm trying to understand how it works.

My goal will be to have multiple repos, having several components in each.

I have several group of customers, and some customers could have some specific version of a package.

group1 => customer11, customer12, customer 13

group2 => customer21, customer22

customer11 => package_1.0.0.deb

customer12 => package_1.0.0.deb

customer13 => package_2.0.0.deb

customer21 => package_2.0.0.deb, packageBis_1.0.0.deb

customer22 => package_1.0.0.deb, packageBis_1.0.0.deb

customer23 => package_2.0.0.deb, packageBis_2.0.0.deb

How can I achieve this kind of structure using Aptly ?

I've imagine having 2 distributions Group1 and Group2, and in each of them having a component per customer.

aptly -distribution="group1" –architectures=armhf,amd64 -component=customer11,customer12,customer13 repo create myRepository

aptly -distribution="group2" –architectures=armhf,amd64 -component=customer21,customer22,customer23 repo create myRepository

aptly repo add myRepository package_1.0.0.deb

aptly repo add myRepository package_2.0.0.deb

aptly repo add myRepository packageBis_1.0.0.deb

aptly repo add myRepository packageBis_2.0.0.deb

aptly snapshot create myRepository_group1 from repo myRepository

aptly publish snapshot -component=customer11,customer12,customer13 -architectures=armhf,amd64 myRepository_group1

But here I could not see how I can distribute my packages in components ?

For me they will have all the same packages inside.

So, is it possible to achieve the previous structure ?

Thanks

Frédéric

[Bernd Naumann]

Hi Frédéric,

On 20.01.21 09:40, Frederic NAUDEAU wrote:
> Hi,
>
> I'm new to aptly and I'm trying to understand how it works.
> My goal will be to have multiple repos, having several components in each.
> I have several group of customers, and some customers could have some
> specific version of a package.
>
> group1 => customer11, customer12, customer 13
> group2 => customer21, customer22
>
> customer11 => package_1.0.0.deb
> customer12 => package_1.0.0.deb
> customer13 => package_2.0.0.deb
>
> customer21 => package_2.0.0.deb, packageBis_1.0.0.deb
> customer22 => package_1.0.0.deb, packageBis_1.0.0.deb
> customer23 => package_2.0.0.deb, packageBis_2.0.0.deb
>

> *How can I achieve this kind of structure using Aptly ?*

>
> I've imagine having 2 distributions Group1 and Group2, and in each of them
> having a component per customer.
>
> aptly -distribution="group1" –architectures=armhf,amd64
> -component=customer11,customer12,customer13 repo create myRepository
> aptly -distribution="group2" –architectures=armhf,amd64
> -component=customer21,customer22,customer23 repo create myRepository
>
> aptly repo add myRepository package_1.0.0.deb
> aptly repo add myRepository package_2.0.0.deb
> aptly repo add myRepository packageBis_1.0.0.deb
> aptly repo add myRepository packageBis_2.0.0.deb
>
> aptly snapshot create myRepository_group1 from repo myRepository
> aptly publish snapshot -component=customer11,customer12,customer13
> -architectures=armhf,amd64 myRepository_group1
>

> *But here I could not see how I can distribute my packages in components ?*
> *For me they will have all the same packages inside.*
> *So, is it possible to achieve the previous structure ?*
>
> Thanks
> Frédéric
>

If I understand you: you want to have ONE repository per component, and
then you want to publish these multiple repositories/components as ONE
distribution.
See https://www.aptly.info/doc/feature/multi-component/ for reference.
(Obviously you dont need the mirror and snapshot and such, but you see
how to create multiple repos and then how to publish the dist.)

Best,
Bernd

[Frederic NAUDEAU]

Well, not really but I don't know if it's feasible.

The goal would be to have one repository per group ( or only one repository and one distribution per group ) and a component per customer.

And what I don't know, is it possible to have different packages per component and how to put a package in only one component ?

[Bernd Naumann]

On 20.01.21 09:59, Frederic NAUDEAU wrote:
> Well, not really but I don't know if it's feasible.

> The goal would be to have *one repository per group ( or only one
> repository and one distribution per group )* and a *component per customer.*

As far as I know, you can not create/use a repository with multiple
components.
https://www.aptly.info/doc/aptly/repo/create/
If you add `-component="foo"` and/or `-distribution="bar"` then this is
used only as a default when you use publish.

A publish can have / be composed of multiple components.
But a publish with multiple components is the result/merge of multiple
repos...
https://www.aptly.info/doc/aptly/publish/repo/

Again, please have a look at
https://www.aptly.info/doc/feature/multi-component/ too.

> And what I don't know, is it possible to have different packages per
> component and how to put a package in only one component ?

You can add a package only to one repo at a time. Sure, you can repeat
`aptly repo add <repo> <pkg>` for multiple repos.
And, in any case you can add as much different versions of a package to
one/each repo as you like.


Bernd

[Frederic NAUDEAU]

Ok but what is the purpose of components ? How to use them ?

[Bernd Naumann]

On 20.01.21 11:06, Frederic NAUDEAU wrote:
> Ok but what is the purpose of components ? How to use them ?
>

> https://www.debian.org/doc/debian-policy/ch-archive

Components are used to split an archive in to "areas". For Debian its
used to separate 'contrib' and 'non-free' from 'main'. Ubuntu is similar
by making i.e. a difference between 'universe' and 'multiverse'.

But a distribution can have a totally happy life with just one component.

I've also encountered archives (on the web) which abuses components for
versioning, but I think this is just broken and complicates stuff.

I recommend to use a distribution either coupled to the "distribution"
(i.e. Debian Buster, Ubuntu Xenial, etc...) or if you for instance have
statically compiled packages without external dependencies, you could
use a distribution to separate your major versions and use these for
various OSs.

In your case, where you want to provide different versions of one or
more packages for different customers, I would evaluate one archive per
customer, and if you need really more then one component. You could also
setup more then one archive per customer. In the end you need just a few
shell commands to glue everything together.

[Frederic NAUDEAU]

Many thanks for your answers.

Adding a package can only be repository wide ?

I mean if I have:

repository: test_repository, that contains 2 distributions: distrib1 and distrib2, and each distribution contains 2 components: stable and test:

test_repository

    distrib1

        stable

        test

    distrib2

        stable

        test

If I do an add:

aptly repo add test_repository mypackage_1.0.0.deb

aptly repo add test_repository mypackage_2.0.0.deb

How can I do to only propose mypackage v2.0.0 in the test component, as I don't want it yet in my stable component ?

test_repository

    distrib1

        stable

            mypackage_1.0.0.deb

        test

            mypackage_1.0.0.deb

            mypackage_2.0.0.deb

Indeed, on the client machine I will add in the sources.list a:

deb http://debs.myaptlyserver.com distrib1 stable

or

deb http://debs.myaptlyserver.com distrib1 test

Like this, on a test machine I can only use testing packages and on the field machines I will use the stable packages.

Sorry, but it's really something I don't understand on how to spread my packages in Aptly.

[Bernd Naumann]

Hi Frederic,

I'm not sure if I understand you correct regarding your repository
setup, let me try to rephrase it...

0) your use-case would be an archive of self build packages
1) first you create a (aptly) `repo`, optional already with a
"distribution" and "component"
2) this `repo` can be published (if your aptly repo is configured with
distribution and component, then the publish can take this info from the
repo, otherwise you need to set it for the publish)
3) that's all.

An aptly repo has only one distribution and one component. To mimic i.e.
Debian Buster, you would need three (3) aptly repos, each with
distribution=buster, but each one would have only one of "main",
"contrib", or "non-free" used as component.

If you now have for example a package which is build one time for ubuntu
xenial and one time for debian buster, so booth packages are different,
you would have to create TWO `repo` and each `repo` would be published
on its own. (The default component "main" could be used for booth.)

Then there is this one special case, where the package is identical for
all distributions (read as buster, xenial and so on...) then you COULD
just setup one `repo` and only `publish`. But most package builds have
different dependencies, so you should/have to use one `repo` for each of
your target distributions.

MAYBE there a good use-case to use components for "stable", "testing",
and "unstable", but I would recommend to do it like debian does it:
Each one is a different distribution. So in aptly terms, you would have
three repos and three publishes. (Every time I'm stumpled upon archives
in the wild which used components for their internal versioning some
issues where involved, so personally dislike it.)


I try to pickup your example....

aptly repo create -distribution="stable" -component="main" my_corp_stable

aptly repo create -distribution="testing" -component="main" my_corp_testing

aptly repo create -distribution="unstable" -component="main"
my_corp_unstable

and then you would add i.e. a new version of a package to the unstable
repo, and then you publish one or all three

aptly pubish repo my_corp_stable
aptly pubish repo my_corp_testing
aptly pubish repo my_corp_unstable

Later, If your confirmed everything is cool, and you like to add this
version to stable, you could either just add the deb-file to the stable
repo or you can use aptly repo copy to copy one or more packages from
one aptly repo to another. Then again you need to update the publish.
(For many aptly operations you can use dpkg-filter-querys! Make use of
them!)

The components used by Debian and Ubuntu are mainly to separate software
in terms of free-ness or quality. So there is no technical reason, but
users of Debian for instance are free to have a system with only free
software, so these users would not include contrib and/or non-free, just
main.


TL;DR; Do not use components for versioning. Use different distributions
instead.

I hope I could help a bit.
Feel free to ask again if thinks are still unclear.

Bernd

On 20.01.21 14:38, Frederic NAUDEAU wrote:
> Many thanks for your answers.
> Adding a package can only be repository wide ?
> I mean if I have:

> repository: *test_repository*, that contains 2 distributions: *distrib1*
> and *distrib2*, and each distribution contains 2 components: *stable* and
> *test:*

>
> test_repository
> distrib1
> stable
> test
> distrib2
> stable
> test
>
> If I do an add:
>

> *aptly repo add test_repository mypackage_1.0.0.deb*
> *aptly repo add test_repository mypackage_2.0.0.deb*
>
> *How can I do to only propose mypackage v2.0.0 in the test component, as I
> don't want it yet in my stable component ?*
>
> test_repository
> distrib1
> stable
> *mypackage_1.0.0.deb*
> test
> *mypackage_1.0.0.deb*
> *mypackage_2.0.0.deb*

>
> Indeed, on the client machine I will add in the sources.list a:
>
> deb http://debs.myaptlyserver.com distrib1 stable
>
> or
>
> deb http://debs.myaptlyserver.com distrib1 test
>

> Like this, on a *test machine* I can only use *testing packages* and on the *field
> machines* I will use the *stable packages*.

[Frederic NAUDEAU]

Many thanks for your time.

I think it's quite clear, I will build my Aptly repos following your recommendations.

If I will have other questions I will post a new questions.

Frédéric