HTTP request blocking by CAs for CRL, CPS, AIA caIssuers

[Dexter Castor Döpping]

Hi all,

Recently I wrote a script to get resources hosted by CAs. For some CAs
the requests were blocked by a WAF.

In one case (http://www.microsoft.com/pkiops/*) I think the blocking
happens because of the user-agent header. Probably because it's a
generic one (python-requests/2.32.5). Changing the UA header immediately
stops my requests from getting blocked. [I reported this and they're
looking into changing their configuration to not block this UA.]

In another case (http://ca-repository.desc.gov.ae/*) I don't know why
I'm being blocked. Perhaps because of the user-agent header, or maybe
because I exceeded a rate limit. Waiting some time or changing request
details hasn't fixed this. I suspect I've been IP blocked, but no idea
if it's temporary or indefinitely.

Section 4.9.7 of the BRs says the following: CRLs MUST be available via
a publicly-accessible HTTP URL (i.e., "published").

I'm wondering if blocking requests breaks this "publicly-accessible"
requirement. Perhaps it depends on why requests are blocked? Rate limits
make sense to me, but geo blocking does not. Blocking user agents may be
a grey area.

The baseline requirements also says that certain certificates must
contain a CRL URL. But is there a requirement for the CRLs to be
"publicly-accessible" through the URLs embedded in the certificate?

RFC 5280 doesn't really clarify things. It says that the CRL/AIA
caIssuers URIs embedded in certs must "point to" certain file types, but
that language seems vague.

The strictest interpretation is that the server should always return 200
OK along with the proper file.

Another interpretation is that status code 200 tells the client they can
get the resource "pointed to" by the URI, so in that case it must return
a proper file. Other data can be returned with different status codes,
like deny message with status code 403. (Perhaps status code 404 can not
be used because this tells the client that the URI doesn't "point to"
anything).

In the loosest interpretation any status code can effectively be used
for anything. For example, HTML error messages with status code 200.
(That's what Microsoft's server does when blocking requests.)

I think it'd be good if there are guidelines for what reasons CAs
can/can't block requests, whether they can (temporarily) IP block you,
and what the server should respond with if you're blocked (e.g., what
status codes, should it give a block reason message?, contact method?).

What do other people think? What do the standards currently say about
CRL/AIA caIssuers availability? And should they be changed/clarified? I
saw that DigiCert plans to start that discussion regarding caIssuers AIA
availability on the CABF, which is a good development:
https://bugzilla.mozilla.org/show_bug.cgi?id=2009491#c2

Kind regards,
Dexter

[Peter Gutmann]

Dexter Castor Döpping <dexter.c...@gmail.com> writes:

>In another case (http://ca-repository.desc.gov.ae/*) I don't know why I'm
>being blocked. Perhaps because of the user-agent header, or maybe because I
>exceeded a rate limit.

That's an interesting point, at what stage do you decide you're being DoS'd
and rate-limit? I was pleasantly surprised recently when I was debugging a
resource-exhaustion issue on an embedded device that I could run a CA-cert
fetch in a loop without being rate-limited (thank you to whoever set up
cacerts.digicert.com for not rate-limiting :-). However since it could turn
into a DoS (I only did around 200 fetches with a wait between each one so
barely a blip), presumably there would need to be some text in any guidelines
about when it was permissible to rate-limit.

Peter.

[Hanno Böck]

Hi,

On Mon, 19 Jan 2026 22:55:47 +0100
Dexter Castor Döpping <dexter.c...@gmail.com> wrote:

> Recently I wrote a script to get resources hosted by CAs. For some
> CAs the requests were blocked by a WAF.

I've been hit by this before, and I'd very much appreciate if we could
have some basic sanity rules.

It's understandable that some form of abuse prevention takes place
(e.g., ratelimits), but I don't see how blocking certain user agents is
acceptable. It should be possible to validate certificates based on the
information in the certificate, and it should not be upon the CA to
decide which software is allowed to do this.

While at it, I also wonder if there's an expectation to send CRLs and
Issuer certs with correct MIME types. (The correct MIME types are
application/pkix-crl for CRLs and application/pkix-cert for issuer
certificates. There are a couple of obsolete or inofficial CRL mime
types in use, e.g. application/x-x509-crl or application/x-pkcs7-crl.)

--
Hanno Böck - Independent security researcher
https://itsec.hboeck.de/
https://badkeys.info/

[Adriano Santoni]

There clearly is such an expectation, since RFC5280 states that those content-types "SHOULD" be specified when serving CRLs and CA certificates, respectively.

-- Adriano