Docker enforcing pull rate limit for a singularity build?

[Tashrif B.]

Hello there,

I am trying to understand how the Docker pull rate limit is enforced. I am doing the following:

singularity build my_img.sif Singularity

where the content of Singularity is as follows:

Bootstrap:docker
From:centos:7.4.1708

Although I haven't done a single pull in the last six hours, I am getting the following:

[root@rc-predict singularity]# singularity build my_img.sif Singularity
INFO:    Starting build...
FATAL:   While performing build: conveyor failed to get: Error reading manifest 7.4.1708 in docker.io/library/centos: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

To make sure this limit is not coming from centos account, I verified that I can do the following independently:

docker pull centos:7.4.1708

Did anyone else experience this issue/know of a fix?

Best,
Tashrif

[v]

Hey Tashrif,

When you build a singularity recipe with a docker base, this means that the singularity client has to use the API to retrieve the manifest and the layers. As far as I know, this counts toward the pull. As for how it's enforced, if you've logged in then you are providing a token that links to your account. If you haven't logged in, it might be based on ip address or similar (so you could try from a different machine). It does say that:

Anonymous and Free Docker Hub users are limited to 100 and 200 container image pull requests per six hours

So it could just be that their "counter" isn't reset yet? The best bet would be to try tomorrow, or earlier from a different machine. Aside from caching images / layers so I don't need to pull again, I don't think there are any magic tricks. You might find Docker buildkid useful, for other reasons too https://brianchristner.io/what-is-docker-buildkit/.

Best,

Vanessa

--
You received this message because you are subscribed to the Google Groups "singularity" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/singularity/CAHgtwXRMZBaRkqV%3D1FJFXxytss-niwAodmxQ5HFDjR1aSNEfSQ%40mail.gmail.com.