Bug#863014: [debian.mirrors.ovh.net]: site trace, sync frequency, syncscript

[Peter Palfrader]

Package: mirrors
User: mir...@packages.debian.org
Usertags: mirror-problem

Hi,

I was checking some things in the Debian mirror universe and noticed
a problem with your mirror

o trace file:
I notice there is no tracefile matching your site name
debian.mirrors.ovh.net in
http://debian.mirrors.ovh.net/debian/project/trace/ -- Are you using our ftpsync script to
mirror Debian? It should produce the trace files we require
http://XXX/debian/project/ftpsync/ftpsync-current.tar.gz

o We require mirrors sync regularly - usually four times a day.
It seems your mirror syncs way less frequent.

Also see
https://mirror-master.debian.org/status/mirror-info/debian.mirrors.ovh.net.html

Cheers,
--
| .''`. ** Debian **
Peter Palfrader | : :' : The universal
https://www.palfrader.org/ | `. `' Operating System
| `- https://www.debian.org/

[Felix Krohn]

Hello Peter,

Yes, the sync frequency isn't adapted yet after migrating the Debian
mirrors to a new infrastructure. It's work in rogress, will be completed
on Monday. I'll add the trace file at the same time.
On this new setup we don't use ftpsync anymore, as mirror integrity is
guaranteed using snapshots - the new content is pushed to the publicly
accessible repo only after the sync has been completely successful.

Best regards,
Felix

* Peter Palfrader <wea...@debian.org> [2017-05-20 09:34]:

--
"This code was not supposed to go in to production yet" (© developerexcuses.com)

[Peter Palfrader]

On Sat, 20 May 2017, Felix Krohn wrote:

> Yes, the sync frequency isn't adapted yet after migrating the Debian
> mirrors to a new infrastructure. It's work in rogress, will be completed
> on Monday. I'll add the trace file at the same time.

Sounds good.

> On this new setup we don't use ftpsync anymore, as mirror integrity is
> guaranteed using snapshots - the new content is pushed to the publicly
> accessible repo only after the sync has been completely successful.

Does not sound so good. ftpsync does more than just semi-atomically
update the mirror. It also ensures things go right when your upstream
is currently syncing for instance. By all means combine ftpsync with
snapshots if you want, but please do use ftpsync for the actual mirror
run.

[Felix Krohn]

Hello Peter,

* Peter Palfrader <wea...@debian.org> [2017-05-20 13:10]:

> > Yes, the sync frequency isn't adapted yet after migrating the Debian
> > mirrors to a new infrastructure. It's work in rogress, will be completed
> > on Monday. I'll add the trace file at the same time.
> Sounds good.

cronjobs are adapted, with */6 frequency.

> By all means combine ftpsync with snapshots if you want, but please do
> use ftpsync for the actual mirror run.

That as well, I adapted our workflow to use ftpsync instead of direct
rsync calls.

https://mirror-master.debian.org/status/mirror-info/debian.mirrors.ovh.net.html
seems to already have picked up this up.

Our backend storage isn't currently reachable by rsync, I'll try to reach
out to the mirror downstream of us to see what we can do.

With this migration we also discontinued ftp services, but it seems as
this hasn't been linked by d.o/mirror/list for more than a year now, so
it shouldn't be a problem I believe.
I monitored ftp logs over the last week and could mostly only see
accesses to files that aren't mirrored anymore (squeeze and older)

Meanwhile, I noticed that our debian-cd repository
(http://debian.mirrors.ovh.net/debian-cd/) isn't listed at
https://www.debian.org/CD/http-ftp/, and I'm not sure it ever was. Should I
open a separate ticket to include it?

And one more question, our new setup permits us to provide the same
content at different geolocations, mainly France and Canada for now.
Would you prefer keeping those under different DNS names? Or is it
possible to list a mirror twice under the same name, but different
locations? (for example replacing debian.bhs.mirrors.ovh.net)

Thanks and best regards,
Felix Krohn
--
"I did a quick fix last time but it broke when we rebooted" (© developerexcuses.com)