Errors while using aptly to mirror an aptly repository

389 views
Skip to first unread message

Carlos González

unread,
May 17, 2017, 1:00:47 PM5/17/17
to aptly-discuss
Hi Everyone,

We are experimenting with a hierarchy of multiple aptly servers and we noticed that we are getting the following error while trying to update the multiverse mirror on the slave:

root@tst-ubuntu-repo02:~# aptly mirror update trusty-multiverse-mirror
Downloading http://<master server>/ubuntu/latest/dists/trusty/InRelease...
gpgv: Signature made Wed 17 May 2017 12:07:08 PM GMT using RSA key ID 286E910B
gpgv: Good signature from "<signature>"
Downloading & parsing package files...
Downloading http://<master server>/ubuntu/latest/dists/trusty/multiverse/binary-i386/Packages.bz2...
ERROR: unable to update: no candidates for http://<master server>/ubuntu/latest/dists/trusty/multiverse/debian-installer/binary-i386/Packages found
root@tst-ubuntu-repo02:~#

I noticed "debian-installer" directory is not being created on multiverse, why?

Inline image 1

This is our configuration:

On the "master aptly server" are using multi-component repository publishing so we created 4 mirrors:
trusty-main-mirror
trusty-multiverse-mirror
trusty-restricted-mirror
trusty-universe-mirror

And we also took the following 4 snapshots:
trusty-main-snapshot-2017-05-17
trusty-multiverse-snapshot-2017-05-17
trusty-restricted-snapshot-2017-05-17
trusty-universe-snapshot-2017-05-17

And we publish them using the "latest" prefix:
# aptly publish show trusty latest
Prefix: latest
Distribution: trusty
Architectures: amd64 i386
Sources:
  restricted: trusty-restricted-snapshot-2017-05-17 [snapshot]
  multiverse: trusty-multiverse-snapshot-2017-05-17 [snapshot]
  main: trusty-main-snapshot-2017-05-17 [snapshot]
  universe: trusty-universe-snapshot-2017-05-17 [snapshot]
root@tst-ubuntu-repo01:~#

Im attaching the output of aptly mirror/snapshot show from both servers.

Regards,
Carlos
mirror_snapshot_details_repo01.txt
mirror_details_repo02.txt

Andrey Smirnov

unread,
May 22, 2017, 4:25:51 PM5/22/17
to Carlos González, aptly-discuss
Hey Carlos, 

Do you have `-with-udebs` option enabled while creating your mirror from upstream?

By default aptly doesn't mirror .udeb packages.

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Carlos González

unread,
May 22, 2017, 6:47:56 PM5/22/17
to Andrey Smirnov, aptly-discuss
Hi Andrey,

That is correct, for example on the mirror_snapshot_details_repo01.txt you can see that it shows "Download .udebs: yes":

Begin command output: ----------------------------
Name: trusty-main-mirror
Archive Root URL: http://mirrors.mit.edu/ubuntu/
Distribution: trusty
Components: main
Architectures: i386, amd64
Download Sources: no
Download .udebs: yes
Last update: 2017-05-17 12:00:06 UTC
Number of packages: 14287

Information from release file:
Architectures: amd64 arm64 armhf i386 powerpc ppc64el
Codename: trusty
Components: main restricted universe multiverse
Date: Thu, 08 May 2014 14:19:09 UTC
Description:  Ubuntu Trusty 14.04

Label: Ubuntu
Origin: Ubuntu
Suite: trusty
Version: 14.04
....

Regards,
Carlos

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee

Andrey Smirnov

unread,
May 22, 2017, 7:16:22 PM5/22/17
to Carlos González, aptly-discuss
I'm not sure what might the problem here. You might want to check whether `.udeb` packages are really part of snapshot. Publish should automatically create `debian-installer` hierarchy once it hits at least one `.udeb` package.

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee

Carlos González

unread,
May 23, 2017, 12:24:52 PM5/23/17
to Andrey Smirnov, aptly-discuss
Hi Andrey,

Yeah, it does looks like a problem. All the repo's mirror have udeb:

root@tst-ubuntu-repo01:/servers/ubuntu# aptly mirror list | egrep 'trusty-(main|multiverse|restricted|universe)'
 * [trusty-main-mirror]: http://mirrors.mit.edu/ubuntu/ trusty [udeb]
 * [trusty-multiverse-mirror]: http://mirrors.mit.edu/ubuntu/ trusty [udeb]
 * [trusty-restricted-mirror]: http://mirrors.mit.edu/ubuntu/ trusty [udeb]
 * [trusty-universe-mirror]: http://mirrors.mit.edu/ubuntu/ trusty [udeb]
root@tst-ubuntu-repo01:/servers/ubuntu#

I can see many udeb pacakges:

root@tst-ubuntu-repo01:/servers/ubuntu# find public/latest/pool/ -name '*.udeb' | head
public/latest/pool/main/libb/libbsd/libbsd0-udeb_0.6.0-2ubuntu1_amd64.udeb
public/latest/pool/main/libb/libbsd/libbsd0-udeb_0.8.2-1_amd64.udeb
public/latest/pool/main/libb/libbsd/libbsd0-udeb_0.6.0-2ubuntu1_i386.udeb
public/latest/pool/main/libb/libbsd/libbsd0-udeb_0.8.2-1_i386.udeb
public/latest/pool/main/libd/libdrm/libdrm2-udeb_2.4.52-1_i386.udeb
public/latest/pool/main/libd/libdrm/libdrm2-udeb_2.4.67-1_amd64.udeb
public/latest/pool/main/libd/libdrm/libdrm2-udeb_2.4.67-1_i386.udeb
public/latest/pool/main/libd/libdrm/libdrm2-udeb_2.4.52-1_amd64.udeb
public/latest/pool/main/libd/libdebian-installer/libdebian-installer4-udeb_0.102ubuntu1_amd64.udeb
public/latest/pool/main/libd/libdebian-installer/libdebian-installer4-udeb_0.102ubuntu1_i386.udeb
root@tst-ubuntu-repo01:/servers/ubuntu#

root@tst-ubuntu-repo01:/servers/ubuntu# find public/latest/pool/ -name '*.udeb' | wc -l
1199
root@tst-ubuntu-repo01:/servers/ubuntu#

But for trusty I noticed debian-installer was only created for the main and universe components:

root@tst-ubuntu-repo01:/servers/ubuntu# find public/latest/ -name 'debian-installer' | grep trusty
public/latest/dists/trusty/main/debian-installer
public/latest/dists/trusty/universe/debian-installer
root@tst-ubuntu-repo01:/servers/ubuntu#

Regards,
Carlos

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

Andrey Smirnov

unread,
May 23, 2017, 2:34:30 PM5/23/17
to Carlos González, aptly-discuss
Carlos, so what's the issue there? Probably there are no udebs in other components? I'm just trying to understand the problem.

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

Carlos González

unread,
May 24, 2017, 12:32:31 AM5/24/17
to Andrey Smirnov, aptly-discuss
Hi Andrey,


Probably there are no udebs in other components?

It looks like that is the case, downloading http://mirrors.mit.edu/ubuntu/dists/trusty/multiverse/debian-installer/binary-amd64/Packages.bz2 contains an empty file.

So what's the issue there?
I would expect consistency if aptly is able to pull from a public mirror I would expect to be able to use the same expect to use the same steps to mirror the aptly publishment.

Regards,
Carlos

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee

Andrey Smirnov

unread,
May 24, 2017, 11:07:19 AM5/24/17
to Carlos González, aptly-discuss
I'm still trying to understand the root cause, but if I got it correctly, aptly creates debian-installer indexes while publishing only for components which have udebs, while mirroring aptly requires every component to have debian-installer index. Is my understanding correct?

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee

Carlos González

unread,
May 24, 2017, 11:10:53 AM5/24/17
to Andrey Smirnov, aptly-discuss
Yeah, that looks like the root cause.

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

Andrey Smirnov

unread,
May 25, 2017, 4:34:07 PM5/25/17
to Carlos González, aptly-discuss
Would you mind submitting issue at GitHub? Should be pretty easy to fix on mirroring side.

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

Carlos González

unread,
May 25, 2017, 7:21:06 PM5/25/17
to Andrey Smirnov, aptly-discuss

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee



--
K-lee

Andrey Smirnov

unread,
May 26, 2017, 12:44:16 PM5/26/17
to Carlos González, aptly-discuss
Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee

--
You received this message because you are subscribed to the Google Groups "aptly-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aptly-discus...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
K-lee



--
K-lee



--
K-lee
Reply all
Reply to author
Forward
0 new messages