[RFC] Considerations around a mirror container registry

[Stephen Augustus]

Hey everyone!

There have been a few conversations on various issues about mirroring images from Docker Hub, Quay, <insert-other-popular-registries-here>, to make the project a little bit more resilient to external outages or policy changes.

As this is something we're considering, I'd like to try and consolidate the various conversations in one place: https://github.com/kubernetes/sig-release/issues/1369

Some things I'd love to get feedback on from you all (in the issue):

• Do you think we need a mirror? Why?
• Which external images do you depend on? Where do they live?

The conversation is pretty greenfield at the moment, so any and all feedback is welcome!

-- Stephen

[Stephen Augustus]

Awesome feedback, Mikaël!

Would you mind commenting on the issue so we can capture it there as well?

-- Stephen

On Wed, Dec 2, 2020 at 6:14 PM Mikaël Cluseau <mikael....@gmail.com> wrote:

Hi,

funny enough, I recently had to solve the problem of reducing the bandwidth usage of the pulls all around clusters, every time an image moves or a daemonset is deployed. This also allows for faster restart delays since the blobs are locally cached. Both those issues were important for my customers.

That bandwidth usage is mainly the blobs so I wrote a pull-through cache recording those blobs and serving them directly. Supports parallel pulls of the same blob, and peer-checking to avoid hitting the upstream while allowing high availability.

https://github.com/mcluseau/docker-registries-mirror

--
You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAOqU-DQzqzUEPUQxLgaQt2%3DYnEdJBTKr2jEcaknVU4c_Z%3Do6kw%40mail.gmail.com.

[Mikaël Cluseau]

Hi,

funny enough, I recently had to solve the problem of reducing the bandwidth usage of the pulls all around clusters, every time an image moves or a daemonset is deployed. This also allows for faster restart delays since the blobs are locally cached. Both those issues were important for my customers.

That bandwidth usage is mainly the blobs so I wrote a pull-through cache recording those blobs and serving them directly. Supports parallel pulls of the same blob, and peer-checking to avoid hitting the upstream while allowing high availability.

https://github.com/mcluseau/docker-registries-mirror

Le jeu. 3 déc. 2020 à 00:00, Stephen Augustus <steph...@agst.us> a écrit :

--

[Stephen Augustus]

Much appreciated!

On Wed, Dec 2, 2020, 18:33 Mikaël Cluseau <mikael....@gmail.com> wrote:

I clearly had a different context in mind when I linked my usecase to your message, but sure, I did ;)

[Mikaël Cluseau]

I clearly had a different context in mind when I linked my usecase to your message, but sure, I did ;)