What changed on yum.puppetlabs.com and apt.puppetlabs.com?

[Sean]

Hi,

Has something significant changed on repo sites?  For years, I have been using lftp to selectively mirror parts of these repos to use on Internet isolated networks.  lftp is a great tool and a little easier and more efficient than wget.  Note, we are not able to use rsync, it is not permitted as an outbound protocol.

So, what's happening when I use lftp today?  Every directory on the repo looks like a file, not a directory, see below:

$ lftp http://apt.puppetlabs.com
cd ok, cwd=/                                          
lftp apt.puppetlabs.com:/> ls
-rw-r--r--  --  index_by_name_reverse.html
-rw-r--r--  --  index_by_lastModified.html
-rw-r--r--  --  index_by_size.html
-rw-r--r--  --  index_by_name.html
-rw-r--r--  --  DEB-GPG-KEY-puppet
-rw-r--r--  --  DEB-GPG-KEY-puppetlabs
-rw-r--r--  --  DEB-GPG-KEY-reductive
-rw-r--r--  --  README.txt
-rw-r--r--  --  dists
-rw-r--r--  --  keyring.gpg
-rw-r--r--  --  pool
-rw-r--r--  --  pubkey.gpg
...<snip>...

What should happen?

$ lftp http://mirror.centos.org
cd ok, cwd=/                                          
lftp mirror.centos.org:/> ls
drwxr-xr-x  --  /          
-rw-r--r--  --  HEADER.images/modern-CentOS-logo.png
-rw-r--r--  --  icons/blank.gif
-rw-r--r--  --  icons/unknown.gif
-rw-r--r--           11  2018-03-01 16:12  TIME
-rw-r--r--  --  icons/folder.gif
drwxr-xr-x            -  2017-09-06 06:57  altarch
drwxr-xr-x            -  2009-09-09 05:18  centos-2
drwxr-xr-x            -  2007-10-22 17:16  centos-3
drwxr-xr-x            -  2011-03-02 14:00  centos-4
drwxr-xr-x            -  2014-09-29 08:59  centos-5
drwxr-xr-x            -  2017-04-05 11:36  centos-6
drwxr-xr-x            -  2017-09-13 20:25  centos-7
drwxr-xr-x            -  2017-09-13 13:56  centos
-rw-r--r--          132  2018-03-01 16:10  dir_sizes_altarch
-rw-r--r--          924  2018-03-01 16:10  dir_sizes_centos
drwxr-xr-x            -  2014-07-07 12:49  mirrorscripts
-rw-r--r--  --  icons/text.gif
...<snip>...

As a result, my entire mirror of both apt/yum.puppetlabs.com turned into a bunch of static files, deleting a heap of data I'm going to have figure out how to download again using some other means.  I attempted to reconfigure my mirror script to use wget instead of lftp, but it only downloaded html versions of the directory indexes, no RPMs, no repodata.

Any ideas or anything as to what's up?

Thanks for the assistance!

[Molly Waggett]

Hi Sean,

We have opened CPR-519 and are looking into your issue.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7355d1ea-9a01-4338-ad53-7fa9fc8cc80d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Molly Waggett

she/her/hers

Release Engineer @ Puppet, Inc.

[Sean]

Thank you Ms. Waggett, I have made comments in the ticket so that the data from my post here is now there in full.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7355d1ea-9a01-4338-ad53-7fa9fc8cc80d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.