Use of Certificate/Public Key Pinning

[Nuno Ponte]

Dear m.d.s.p.,

I would like to bring into discussion the use of certificate/public key pinning and the impacts on the 5-days period for certificate revocation according to BR §4.9.1.1.

Recently, we (Multicert) had to rollout a general certificate replacement due to the serial number entropy issue. Some of the most troubled cases to replace the certificates were customers doing certificate pinning on mobile apps. Changing the certificate in these cases required configuration changes in the code base, rebuild app, QA testing, submission to App stores, call for expedited review of each App store, wait for review to be completed and only then the new app version is made available for installation by end users (which is turn are required to update the app the soonest).

Meeting the 5-days deadline with this sort of process is “challenging”, at best.

A first approach is to move from certificate pinning to public key pinning (PKP). This prevents the need to update the app in many of the certificate replacement operations, where the public key is reused and the certificate can be replaced transparently to the app (generically, an “User Agent” doing PKP).

However, in the event of a serious security incident that requires re-key (such as key compromise), the certificate must be revoked in less than 24 hours (for the benefit of everyone – subscriber, relying parties, issuing CA, etc). It’s virtually impossible to release a new app version within this timeframe. And this, I think, make a very strong point against the use of PKI.

On the other side, PKP is a simple yet powerful and effective technique to protect against MITM and other attacks. It seems to be widely used in apps with advanced threat models (mobile banking, sensitive personal information, etc) and there are many frameworks available (including native support in Android via Network Security Configuration [1]).

There are several possible mitigation actions, such as pinning more than one public key to have more than one certificate to quickly rollover in case of a revocation. Even then, it is very likely that all the redundant key pairs were generated and maintained by the same systems and procedures, and thus all of them will become effectively compromised.

Ultimately, it may become common practice that 1) PKP frameworks are set to bypass revocation checks or 2) PKP is done with private certificates (homemade, self-signed, managed ad-hoc with no CRL/OCSP services). Does any of this leads to a safer Internet?

I don’t expect this thread to end up into an absolute conclusion advocating for or against, but opening it to discussion and contributions may help to document possible strategies, mitigations, alternatives, pros & cons, and hopefully provide guidance for an educated decision.

Best regards,

Nuno Ponte
Multicert SA

[1] https://developer.android.com/training/articles/security-config

[Paul Wouters]

On Mon, 12 Aug 2019, Nuno Ponte via dev-security-policy wrote:

> Recently, we (Multicert) had to rollout a general certificate replacement due to the serial number entropy issue. Some of the most troubled cases to replace the certificates were customers doing certificate pinning on mobile apps. Changing the certificate in these cases required configuration changes in the code base, rebuild app, QA testing, submission to App stores, call for expedited review of each App store, wait for review to be completed and only then the new app version is made available for installation by end users (which is turn are required to update the app the soonest).
>
> Meeting the 5-days deadline with this sort of process is “challenging”, at best.

The OS and/or App should look at Certificate Transparency, instead of
hacks that hardcode the certificate serial number.

Paul

[Tom Ritter]

PKP is a footgun. Deploying it without being prepared for the
situations you've described is ill-advised. There's a few options
available for organizations who want to pin, in increasing order of
sophistication:


Enforce Certificate Transparency. You're not locked into any CA or
key, only that the certificate has been published publicly.

Pin to a CA or a couple of CAs - this reduces the
operational/availability risk while increasing the security risk.
(Although still a reduction from the entire set of CAs of course.)

Pin to leaf *keys*, as you suggest, and ensure that they cannot all be
compromised at once through the use of offline storage and careful key
mangement. Use the keys to get certificates when needed. As you note,
if you can't manage these keys securely and separately, you need to go
to something less sophisticated, like pinning to CAs.

Pin to a locally managed trust anchor, and operate a root CA oneself,
managing it as one would a public CA (offline root, possibly offline
intermediates, etc)


-tom

On Tue, 13 Aug 2019 at 15:12, Nuno Ponte via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:
>
> Dear m.d.s.p.,
>
> I would like to bring into discussion the use of certificate/public key pinning and the impacts on the 5-days period for certificate revocation according to BR §4.9.1.1.
>

> Recently, we (Multicert) had to rollout a general certificate replacement due to the serial number entropy issue. Some of the most troubled cases to replace the certificates were customers doing certificate pinning on mobile apps. Changing the certificate in these cases required configuration changes in the code base, rebuild app, QA testing, submission to App stores, call for expedited review of each App store, wait for review to be completed and only then the new app version is made available for installation by end users (which is turn are required to update the app the soonest).
>
> Meeting the 5-days deadline with this sort of process is “challenging”, at best.
>

> A first approach is to move from certificate pinning to public key pinning (PKP). This prevents the need to update the app in many of the certificate replacement operations, where the public key is reused and the certificate can be replaced transparently to the app (generically, an “User Agent” doing PKP).
>
> However, in the event of a serious security incident that requires re-key (such as key compromise), the certificate must be revoked in less than 24 hours (for the benefit of everyone – subscriber, relying parties, issuing CA, etc). It’s virtually impossible to release a new app version within this timeframe. And this, I think, make a very strong point against the use of PKI.
>
> On the other side, PKP is a simple yet powerful and effective technique to protect against MITM and other attacks. It seems to be widely used in apps with advanced threat models (mobile banking, sensitive personal information, etc) and there are many frameworks available (including native support in Android via Network Security Configuration [1]).
>
> There are several possible mitigation actions, such as pinning more than one public key to have more than one certificate to quickly rollover in case of a revocation. Even then, it is very likely that all the redundant key pairs were generated and maintained by the same systems and procedures, and thus all of them will become effectively compromised.
>
> Ultimately, it may become common practice that 1) PKP frameworks are set to bypass revocation checks or 2) PKP is done with private certificates (homemade, self-signed, managed ad-hoc with no CRL/OCSP services). Does any of this leads to a safer Internet?
>
> I don’t expect this thread to end up into an absolute conclusion advocating for or against, but opening it to discussion and contributions may help to document possible strategies, mitigations, alternatives, pros & cons, and hopefully provide guidance for an educated decision.
>
> Best regards,
>
> Nuno Ponte
> Multicert SA
>
> [1] https://developer.android.com/training/articles/security-config
>
>
>
>
>

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy

[Matthew Hardeman]

I feel that there's a great deal of consultancy and assistance that CAs and
PKI professionals could bring to their more sophisticated customers with
scenarios such as these where public key pinning an a field-deployed
application may present problems for certificates being revoked.

A best practices document explaining to the application developers and
server-side teams that:

1. An app which calls a server-side API under your control should _always_
do so on a TLS endpoint at a different hostname & SNI label than any
browser-facing websites.
2. Following step 1's guidance means that you can control the lifecycle of
the certificate for the services accessed by your own application(s)
separate from WebPKI facing certificates meant to facilitate a TLS
authenticated session to a modern browser.
3. It also means that the endpoints serving the application CAN but don't
have to be from a publicly trusted PKI. For compatibility reasons, they
generally should be, if there are any external consumers of the API, but
ultimately if their own application wishes to PIN, they should pre-create
several certificates with distinct keys and write their app to override the
platform trust decisioning and pin on the set of keys that their API
endpoint certificates will have, ignoring revocation and requiring that the
presented leaf certificate be a signature over one of the set of pinned
public keys.

This is essentially free in virtually all deployment models today.

Oversubscribing TLS endpoints (for our purposes let's say a DNS based
hostname and TLS SNI label define a TLS endpoint) for different target
audiences, especially when those audiences are modern browsers in
combination with anything else, is one of the most significant causes of
compatibility issues and legacy cruft which have historically hindered the
agility of the WebPKI.

On Tue, Aug 13, 2019 at 10:12 AM Nuno Ponte via dev-security-policy <

[Ryan Sleevi]

On Tue, Aug 13, 2019 at 11:12 AM Nuno Ponte via dev-security-policy <

Nuno,

Thanks for raising this. I appreciate that a CA is attempting to provide
guidance to their Subscribers, since, as you note, the CA is still beholden
to the Baseline Requirements, and the Subscriber still beholden to their
Subscriber Agreement.

As one of the co-authors of the HTTP Public Key Pinning RFC, RFC 7469, I
would readily encourage your Subscribers not to pin at all. However, if
they are going to pin to anything, then it:
1) Should be to the public key, not certificate
2) Should only be to the root
3) Should require an agreement with the root that the Root (i.e. Multicert)
will disclose all possible certificate paths and root certificates that are
unexpired and may exist.
- This includes cross-signed certificates

Note that mobile devices, the scenario you discuss for pinning, already
largely do not implement revocation checks and do not offer a way for
applications to control it, so I don't think that should be a concern.

Of course, many of these techniques predate improvements to the ecosystem,
such as CAA and Certificate Transparency. Many of the concerns that
motivated HPKP, which subsequently evolved into the aforementioned OS APIs,
were predicated on an assumption of a CA that has lax validation methods.
Yet techniques such as CAA exist to give site operators meaningful control
- and to ensure that any CA that violates such checks, and ignores CAA, is
unambiguously at risk of being distrusted by the entire platform.

If there's concern about the DNS registrar being hacked, the Subscriber has
the option of choosing a better registrar before hand, without resorting to
PKP, and/or transitioning to registries with better security practices
overall. In general, this means avoiding ccTLDs, which are known lax due to
their lack of any formal IANA or ICANN agreements with respect to
operations. Certificate Transparency offers a compliment to this, ensuring
that even if the Registrar or Registry is compromised, misissued
certificates can be promptly detected for subsequent investigation and
remediation.

However, as to your question about private PKIs potentially being more
secure - it is indeed possible this may be the right answer! Organizations
which desire complete lifecycle control over their certificates may benefit
from the use of such private PKIs - while also being wholly responsible for
the operational security of such, which naturally will be something their
regulatory oversight bodies will take great interest in.

Beyond that, what Matt and Tom said is absolutely correct. Making use of
multiple (distinct) TLS endpoints, even on the same IP, is absolutely
crucial for reducing these risks.

As a CA, the best thing you can do is to make sure your Subscribers know
that they are taking on operational risk when they pin, and the CA is not
empowered to assist in resolving that.

[Tim Hollebeek]

So, pinning is an extremely complicated topic that I've always wanted to write a blog post about, but have never had the time to do it. It happens fairly regularly that we have to assist a company that has painted themselves into a corner with a poorly thought out pinning scheme.

In my experience, it's mostly bad implementations of pinning that cause most of the problems, not pinning itself. I've seen people pin leaf certificate serial numbers, for example. That's guaranteed to end badly.

If you do feel the need to pin, please make sure you've thought through all the possible certificate replacement use cases and make sure you have designed, implemented, AND TESTED them to make sure they work.

All too often, a customer complains things stop working when they replace their certificate ... but no one at the company is aware what exactly is being pinned or why.

Use pinning with care.

-Tim