aptly and multi-component for local repositories

[Sumit-MailingLists]

I'm currently building out a repository for a project I'm working on called BigSense. I've created my entire apt/yum server in asnible and use aptly for the debian packages. But as I'm publishing build packages, I've had some conflicting packages and realized I had everything setup incorrectly. The multi-component documentation is a little confusing. I'd like to help update some of it, but I feel I need some help to understand the best way to accomplish what I'm trying to do.

So I have two aptly configuration files that look like the following:

{
  "rootDir" : "/home/repoman",
  "ppaDistributorID" : "Ubuntu",
  "ppaCodename" : "trusty",
  "architectures" : ["amd64","i386","all"]
}

One says "trusty" and the other says "wheezy" for ppaCodename. Immediately I realize my first mistake. They can't share rootDirs. The only difference between the two packages are upstart vs system-v init scripts (and I've through about renaming all the codenames to "upstart" and "systemv" and adding "systemd" for Jessie respectively, but that's another issue).

I also have three components: stable, testing and nightly. My ansible script uses a loop of two lists to run the following set of commands:

aptly repo create -comment "bigsense repository" -component=nightly -config=trusty.aptly trusty-nightly
aptly repo create -comment "bigsense repository" -component=testing -config=trusty.aptly trusty-testing
aptly repo create -comment "bigsense repository" -component=stable -config=trusty.aptly trusty-stable

aptly repo create -comment "bigsense repository" -component=nightly -config=wheezy.aptly wheezy-nightly
aptly repo create -comment "bigsense repository" -component=testing -config=wheezy.aptly wheezy-testing
aptly repo create -comment "bigsense repository" -component=stable -config=wheezy.aptly wheezy-stable

Then I have ansible do an initial publish of my repo, looping to create the following commands:

aptly -passphrase-file=<pgp-password-file> --distribution=trusty --component=nightly,testing,stable --config=trusty.aptly publish repo trusty-nightly trusty-testing, trusty-stable

aptly -passphrase-file=<pgp-password-file> --distribution=wheezy --component=nightly,testing,stable --config=wheezy.aptly publish repo wheezy-nightly wheezy-testing, wheezy-stable

I have a script that gets called from my Jenkins-CI instance that figured out if the package is nightly, testing or stable (numbers like 0.3 are stable, numbers with letters are unstable like 0.3alpha and version with a git hash are nightly like 0.3-alpha-12-fe23123) and it adds them like so:

aptly -config=trusty.conf repo add trusty-nightly debs/trusty/ltsense_0.1alpha-23-g591b292_all.deb
aptly -config=whezy.conf repo add whezy-nightly debs/whezy/ltsense_0.1alpha-23-g591b292_all.deb

and of course that's where the problem is. Two distros sharing the same root dir, so the same db files. Since the packages are different, I constantly get things like so:

ERROR: unable to publish: unable to process packages: error linking file to /home/repoman/public/pool/nightly/b/bigsense/bigsense_0.2.3alpha-16-gf33342e_all.deb: file already exists and is different

So I'm assuming I can do everything the way I've done so far, but I need different rootDirs in the configuration files so they have totally different db and pool directories. Then I get confused. I realize I then need to somehow make a snapshot of each repo and then publish all those snapshots together so I get a single tree with trusty and wheezy, both with all three components.

What are the commands I need to run to make those snapshots happen and publish everything in one directory I can serve up on an nginx instance?

Thanks
Sumit

[Andrey Smirnov]

Hi Sumit!

First of all, I haven't got your point of using two configuration files with only difference in `ppaCodename`. This option is used only when your create mirror with PPA url, but your example doesn't use that. And it would be much easier to specify full HTTP URLs rather than having to deal with two configuration files :)

Second, if I got your example correctly, you build two packages which have the same (arch, name, version) (and, filename), but have different contents. I would call such packages as 'conflicting'. aptly can manage such packages correctly in one rootDir (database), there's no problem. There are two restrictions:

 1. One repo, snapshot or published repository can't contain two conflicting packages (it's fine to have them in different repos, snapshots, ...).

 2. You can't publish two repositories with conflicting packages under the same root (that's limitation of Debian repository structure). That's what you're trying to do, I believe. The reason is that both conflicting package files would end up under the same name in package pool (/home/repoman/public/pool/nightly/b/bigsense/bigsense_0.2.3alpha-16-gf33342e_all.deb in your case).

My suggestions:

1. Don't use two configuration files.

2. Either make packages non-conflicting (by adding something to version of package name, to make them different, like ltsense_0.1alpha-23-g591b292~upstart and ltsense_0.1alpha-23-g591b292~systemv). 

   .. Or publish them under different prefixes. So you would have two subdirectories /home/repoman/public/systemv/ and  /home/repoman/public/upstart/ (or trusty and wheezy). Prefix is last parameter in `aptly publish repo` command, it defaults to empty (publish under root).

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.