Docker rate limiting

[Stephen Benjamin]

Hi all,

We're starting to see our image builds being hit with Docker Hub's
rate limiting[1]. For example, this build[2] failed with:

Could not pull base image: API error (500): toomanyrequests: You
have reached your pull rate limit. You may increase the limit by
authenticating and upgrading:
https://www.docker.com/increase-rate-limit

We currently use CentOS 8 as our base image, retrieved from Docker
hub. There are not any official images from CentOS published to Quay.
It looks like the recommended approach at this point is to push our
base image into the metal3-io org on Quay.

I figured I'd start this thread to see if anyone else had any
suggestions or ideas? I'm not keen for us to be in the business of
maintaining our CentOS images, but I'm not sure we have any other
option at this point.


Thanks

- Stephen


[1] https://www.docker.com/increase-rate-limits
[2] https://quay.io/repository/metal3-io/ironic/build/d8ef7805-a98c-427b-808b-fb453f8437a1

[Michael Hrivnak]

Docker amended their plan to be more friendly for OSS projects: https://www.docker.com/blog/expanded-support-for-open-source-software-projects/

TL;DR the centos project can apply for an exemption as an OSS project, which would enable anyone to pull their images without rate limits. I don't know if anyone at the centos project is looking at that, but it would be a reasonable next step to check in with them.

--
You received this message because you are subscribed to the Google Groups "Metal3 Development List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metal3-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/metal3-dev/CACmh_16%3D-cbWFC7nbo%2BCJ0QBAykhRsZEaOBScP04UE5s3VBs8A%40mail.gmail.com.

--

Michael Hrivnak

Senior Principal Software Engineer, RHCE 

Red Hat

[Stephen Benjamin]

On Tue, Nov 17, 2020 at 3:30 PM Michael Hrivnak <mhri...@redhat.com> wrote:
>
> Docker amended their plan to be more friendly for OSS projects: https://www.docker.com/blog/expanded-support-for-open-source-software-projects/
>
> TL;DR the centos project can apply for an exemption as an OSS project, which would enable anyone to pull their images without rate limits. I don't know if anyone at the centos project is looking at that, but it would be a reasonable next step to check in with them.

Thanks! Zane also found out that CentOS maintains a registry:
https://wiki.centos.org/ContainerPipeline, whose index page 502's, but
registry.centos.org/centos/centos:8 seems to work.

I reached out to centos-devel to ask:
https://lists.centos.org/pipermail/centos-devel/2020-November/056183.html

[Michael Hrivnak]

Great. FWIW I asked about it on freenode last month in #centos-devel, and here's what I got:

[2020-10-12 13:25:06] <mhrivnak> Where is the best place from which to pull centos container images? I know docker hub has been a popular place to access them, but their rate limiting is making that less usable. Is registry.centos.org the best source for centos images?
[2020-10-12 13:25:46] <mhrivnak> I did read this: https://wiki.centos.org/ContainerPipeline
[2020-10-12 13:28:50] <Arrfab> mhrivnak: registry.centos.org isn't probably maintained so not up2date
[2020-10-12 13:29:48] <rsc> Arrfab: is there a specific reason that images are not pushed to quay.io, too?
[2020-10-12 13:37:45] <Arrfab> rsc: no access ? I think there is a ticket about that on pagure.io/centos-infra .. iirc bstinson said he has