Slow Mirror

[Denny B]

Hello everyone,

since two days we aren't able to synchronize our local mirror with apt.puppetlabs.com.
The maximum speed to download everything is about 50-100kb/s which is really really slow for our whole datacenter.
Are there any problems with your mirror? Did you maybe blocked us?

Currently it takes ages to get a synchron mirror.


Thanks in advance.

Best regards,
Denny



For proper debug:

Output rsync:
[..]
dists/lucid-20130301133454158727269/.refs/dependencies/rubygem-mustache_0.99.4-1puppetlabs1.dsc
         915 100%    1.22kB/s    0:00:00 (xfer#25, to-check=1508/1866)
dists/lucid-20130301133454158727269/.refs/dependencies/rubygem-mustache_0.99.4-1puppetlabs1_all.deb
       79302 100%   59.80kB/s    0:00:01 (xfer#26, to-check=1507/1866)
dists/lucid-20130301133454158727269/.refs/dependencies/rubygem-mustache_0.99.4-1puppetlabs2.diff.gz
        1710 100%   11.52kB/s    0:00:00 (xfer#27, to-check=1506/1866)
dists/lucid-20130301133454158727269/.refs/dependencies/rubygem-mustache_0.99.4-1puppetlabs2.dsc
         915 100%    6.16kB/s    0:00:00 (xfer#28, to-check=1505/1866)
dists/lucid-20130301133454158727269/.refs/dependencies/rubygem-mustache_0.99.4.orig.tar.gz
       37714 100%   82.58kB/s    0:00:00 (xfer#29, to-check=1504/1866)
dists/lucid-20130301133454158727269/dependencies/binary-all/Packages
        7686 100%   15.07kB/s    0:00:00 (xfer#30, to-check=1499/1866)
dists/lucid-20130301133454158727269/dependencies/binary-amd64/Packages
       11370 100%   18.98kB/s    0:00:00 (xfer#31, to-check=1498/1866)
dists/lucid-20130301133454158727269/dependencies/binary-i386/Packages
       11371 100%   18.14kB/s    0:00:00 (xfer#32, to-check=1497/1866)
dists/lucid-20130301133454158727269/dependencies/source/Sources
       13142 100%   17.48kB/s    0:00:00 (xfer#33, to-check=1496/1866)
[..]

We connect to following IP: 198.58.114.168
Our IP: 217.79.215.X

[Daniel Dreier]

On Wed, Jun 10, 2015 at 3:07 AM, Denny B <bortfel...@gmail.com> wrote:

Hello everyone,

since two days we aren't able to synchronize our local mirror with apt.puppetlabs.com.
The maximum speed to download everything is about 50-100kb/s which is really really slow for our whole datacenter.
Are there any problems with your mirror? Did you maybe blocked us?

Currently it takes ages to get a synchron mirror.

Denny -

Thanks for the heads-up. I'm on the operations team responsible for apt.puppetlabs.com, and I've created a ticket internally to track the issue. We'll try and reproduce the issue and reach out to you directly as needed, at the very least to confirm a fix.

--

Daniel Dreier

Technical Operations Engineer

GPG: BA4379FD

[Daniel Parks]

On Wednesday, June 10, 2015 at 4:30:33 AM UTC-7, Denny B wrote:

since two days we aren't able to synchronize our local mirror with apt.puppetlabs.com.
The maximum speed to download everything is about 50-100kb/s which is really really slow for our whole datacenter.

Hi Denny,

I'm the other Daniel on the ops team at Puppet. :)

I can rsync from my house, so it is probably a network problem. Traceroute looks okay. Could you try downloading a large file over HTTP and see how fast it is?

For example, I get:

$ curl -o /dev/null http://yum.puppetlabs.com/fedora/f21/products/SRPMS/razor-torquebox-3.1.1.5-1.fc21.src.rpm

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100 61.8M  100 61.8M    0     0  5976k      0  0:00:10  0:00:10 --:--:-- 6320k

(Sorry, I'm posting this from Google Groups and I can't tell if this is going to turn out as HTML or just plain text.)

Thanks,

Daniel

[Denny B]

Hello Daniel and Daniel,

thank you very much for your response.


The output you wanted is here:
root@s110077:~# curl -o /dev/null http://yum.puppetlabs.com/fedora/f21/products/SRPMS/razor-torquebox-3.1.1.5-1.fc21.src.rpm

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

 15 61.8M   15  9.7M    0     0   167k      0  0:06:18  0:00:59  0:05:19 83367


We can't download faster than 1mb/s. When I use another Server from other network everything works fine.
Can you check if there is a speedlimit active for 217.79.215.0/24 ?
To other mirrors (such as debian) we can download as fast as possible (~30-50mb/s)

[Daniel Parks]

On Thu, Jun 11, 2015 at 3:25 AM, Denny B <bortfel...@gmail.com> wrote:

We can't download faster than 1mb/s. When I use another Server from other network everything works fine.
Can you check if there is a speedlimit active for 217.79.215.0/24 ?
To other mirrors (such as debian) we can download as fast as possible (~30-50mb/s)

 

We don’t do any kind of bandwidth limiting for that box, and our hosting provider reports that there shouldn’t be any problems. Looks like there’s a problem between our networks. Could you open a ticket with your hosting provider to see what they can do? Feel free to give them my email (d...@puppetlabs.com) and I’ll do whatever I can to help.

Here’s a traceroute, which may be useful to them:

traceroute to 217.79.215.252 (217.79.215.252), 30 hops max, 60 byte packets

 1  router2-dal.linode.com (67.18.7.162)  89.761 ms  89.752 ms  89.739 ms

 2  ae2.car02.dllstx2.networklayer.com (67.18.7.93)  88.928 ms  112.495 ms  112.474 ms

 3  po102.dsr01.dllstx2.networklayer.com (70.87.254.81)  89.601 ms  89.579 ms  90.085 ms

 4  po21.dsr01.dllstx3.networklayer.com (70.87.255.65)  90.022 ms  89.993 ms  89.959 ms

 5  ae17.bbr01.eq01.dal03.networklayer.com (173.192.18.226)  89.320 ms  89.290 ms ae17.bbr02.eq01.dal03.networklayer.com (173.192.18.230)  89.202 ms

 6  ae7.bbr02.eq01.dal03.networklayer.com (173.192.18.209)  89.192 ms ae1.bbr01.tl01.atl01.networklayer.com (173.192.18.135)  74.128 ms  77.054 ms

 7  ae0.bbr01.eq01.wdc02.networklayer.com (173.192.18.152)  91.707 ms ae1.bbr01.tl01.atl01.networklayer.com (173.192.18.135)  77.037 ms  73.739 ms

 8  ae7.bbr02.eq01.wdc02.networklayer.com (173.192.18.195)  83.151 ms  84.234 ms ae0.bbr01.eq01.wdc02.networklayer.com (173.192.18.152)  85.741 ms

 9  ae7.bbr02.eq01.wdc02.networklayer.com (173.192.18.195)  83.301 ms ae0.bbr01.eq01.ams02.networklayer.com (50.97.18.215)  164.310 ms ae7.bbr02.eq01.wdc02.networklayer.com (173.192.18.195)  84.078 ms

10  ae0.bbr01.eq01.ams02.networklayer.com (50.97.18.215)  161.824 ms  161.688 ms  163.396 ms

11  ae7.bbr01.xn01.fra01.networklayer.com (50.97.18.218)  168.007 ms ae0.bbr02.xn01.fra01.networklayer.com (50.97.18.217)  168.363 ms  206.595 ms

12  ae7.bbr01.xn01.fra01.networklayer.com (50.97.18.218)  208.187 ms  207.047 ms  206.973 ms

13  decix-2.eu.equinix.net (80.81.193.234)  185.045 ms  184.852 ms ge-2-2-22-ed2.ixsolutions.net (217.79.208.25)  193.202 ms

14  ge-2-2-22-ed2.ixsolutions.net (217.79.208.25)  193.112 ms  193.140 ms  193.050 ms

15  * * *

Thanks,

Daniel

[Xav Paice]

On 12/06/15 12:05, Daniel Parks wrote:

On Thu, Jun 11, 2015 at 3:25 AM, Denny B <bortfel...@gmail.com> wrote:

We can't download faster than 1mb/s. When I use another Server from other network everything works fine.
Can you check if there is a speedlimit active for 217.79.215.0/24 ?
To other mirrors (such as debian) we can download as fast as possible (~30-50mb/s)

 

We don’t do any kind of bandwidth limiting for that box, and our hosting provider reports that there shouldn’t be any problems. Looks like there’s a problem between our networks. Could you open a ticket with your hosting provider to see what they can do? Feel free to give them my email (d...@puppetlabs.com) and I’ll do whatever I can to help.

When we go to http://apt.puppetlabs.com/pool/trusty/main/p/puppet/ there's no files at all - although the server responds pretty quickly - is this a known thing?

[Daniel Parks]

On Tue, Jun 16, 2015 at 3:53 PM, Xav Paice <xavp...@gmail.com> wrote:

When we go to http://apt.puppetlabs.com/pool/trusty/main/p/puppet/ there's no files at all - although the server responds pretty quickly - is this a known thing?

I guess serving no files might help with load issues. :)

It shows files for me… I believe it’s updated via rsync, and so the files shouldn’t just disappear, but maybe I’m wrong. I’ll have to look into it, but it’s not the first thing on my list unless other people are running into this.

Thanks for the heads up, though!

Daniel

 

[Daniel Parks]

On Tuesday, June 16, 2015, Xav Paice <xavp...@gmail.com> wrote:

When we go to http://apt.puppetlabs.com/pool/trusty/main/p/puppet/ there's no files at all - although the server responds pretty quickly - is this a known thing?

Turns out we messed up an exclude when rsyncing the other day, and lost those while moving from one server to another. Oops.

Sorry about that!

And thanks for letting me know.

Daniel

[Denny Bortfeldt]

Hello Daniel,

just for your interest: Seems the slow download is produced by our own network (bug in router/switch). So we'll investigate this issue on our own and will respond to you if we found the "bug" and puppetlab is still slow for us.

Please pardon the inconvenience.

Best regards,

Denny

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/5dvAj1ijQAE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAPNukCk8iqShSF_GpAFGuxiXnpm3o_Gfqrd-bdppJFQuHpvuoA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.