Concerns about very-short-lived certificates

[Hanno Böck]

Hi,

I hope this is a good place to share these thoughts, but given it's the
"all things CA" discussion place, it seems to fit.

In a recent post, Let's Encrypt mentions plans for very-short-lived
certificates with a lifetime of 6 days:
https://letsencrypt.org/2024/12/11/eoy-letter-2024/
There have been other discussions going in similar directions, e.g.,
a proposed 45 days cap on certs.

I have some concerns with this direction. I think there were good
reasons to shorten certificate lifetimes in the past. Not having
multi-year certificates is a good thing, as we no longer have
multi-year transition periods to flush out things from the ecosystem.
It's probably also good to shorten cert lifetimes to values that
strongly encourage automation.

I get why people want even shorter certs. People have been trying to
fix revocation, and I think everyone eventually got to the conclusion
that fixing revocation is hard. Replacing revocation with very short
certs is a natural idea.
But I think there's a point where the benefits of even shorter
lifetimes should be balanced against some downsides.

One of the biggest improvements in the WebPKI/CA ecosystem in recent
years was the introduction of Certificate Transparency. And one of the
things CT has brought us was that it allows many people to monitor the
WebPKI ecosystem for irregularities.

But that, of course, relies on people actually doing that. And
monitoring CT becomes more challenging with increasing volumes of
certificates.

The volume of CT-logged certs is already enormous, and it's already
challenging to deal with this. I'm, e.g., scaninng CT-logged certs for
compromised keys.

Many rely not on actually reading CT logs, but utilize crt.sh. It is
great that this service exists, but I'd like to point out the following:
* There's currently no other service like crt.sh. It's essentially a
single point of failure of a lot of "WebPKI security stuff" people do.
* crt.sh is experiencing challenges with availability and cert
backlogs, likely because dealing with this volume is challenging.

In essence, many of us (me included) rely on crt.sh doing the hard work
to deal with CT logs.

Imagining something like a more than 10x increase in cert volumes (which
would be the consequence of 6-day certs as the norm) probably means
many people will just stop utilizing CT to find security issues in the
WebPKI ecosystem. Add to that the fact that, depending on how fast the
Quantum Cryptopocalypse will materialize itself, we may also have to
increase the size per certificate quite substantially.
I think this should be considered when discussing very-short-lived
certs.

--
Hanno Böck - Independent security researcher
https://itsec.hboeck.de/
https://badkeys.info/

[Jeffrey Walton]

I share your concern over short-lived certificates, but for a
different reason: key continuity. Key continuity has proven to be a
much better security property than gratuitous key rotations based on
reading of tea leaves by tasseomancers.

I think the devil will be in the details. If the automated process
reuses the same key in the default use case, then the policy change
should not damage key continuity schemes. In this case, I'm guessing
Let's Encrypt is analyzing a CSR to detect a compromised key since
Josh Aas's post specifically names it as the problem being solved. If
the process performs gratuitous key rotations in the default use case
during a CSR submission, then it is probably doing more harm than
good.

And here, the "default use case" means whatever happens when `certbot
--apache` or `certbot --nginx` is run to setup the automation.

Jeff

[Bas Westerbaan]

* There's currently no other service like crt.sh. It's essentially a
  single point of failure of a lot of "WebPKI security stuff" people do.

Not to detract from your point, but I'm happy to say there is a new kid in town: https://www.merklemap.com/

Imagining something like a more than 10x increase in cert volumes (which
would be the consequence of 6-day certs as the norm) probably means
many people will just stop utilizing CT to find security issues in the
WebPKI ecosystem. Add to that the fact that, depending on how fast the
Quantum Cryptopocalypse will materialize itself, we may also have to
increase the size per certificate quite substantially.
I think this should be considered when discussing very-short-lived
certs.

These are valid concerns. Let me add another: with static CT, running CT logs is easier, which will hopefully lead to more logs in the ecosystem. That will also lead to more logs that have to be followed.

Sticking to long-lived certs doesn't solve our PQ problems with CT, and I don't think it's necessary to make CT PQ ready. Something will have to give though.

In MTC [1] for instance we have 14 day certs, but make that manageable by reducing the size of each cert, and deduplicating certificates across logs by moving the logs to the CA. 

 

Best,

 Bas

[1] https://davidben.github.io/merkle-tree-certs/draft-davidben-tls-merkle-tree-certs.html

 

--
Hanno Böck - Independent security researcher
https://itsec.hboeck.de/
https://badkeys.info/

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/20241212133018.05ec7cae%40computer.

[Matt Palmer]

On Thu, Dec 12, 2024 at 02:09:55PM +0100, 'Bas Westerbaan' via dev-secur...@mozilla.org wrote:
> > * There's currently no other service like crt.sh. It's essentially a
> > single point of failure of a lot of "WebPKI security stuff" people do.
>
> Not to detract from your point, but I'm happy to say there is a new kid in
> town: https://www.merklemap.com/

As far as I can tell, merklemap.com is not currently a suitable
alternative to crt.sh. I sincerely hope it does in the future, though.

- Matt

[Matt Palmer]

On Thu, Dec 12, 2024 at 09:56:11AM -0500, Jeffrey Walton wrote:
> I share your concern over short-lived certificates, but for a
> different reason: key continuity. Key continuity has proven to be a
> much better security property than gratuitous key rotations based on
> reading of tea leaves by tasseomancers.

Do you have any citations you can share? My experience is precisely the
opposite, and I have the 2M compromised keys, and ~thousands of
consequently compromised TLS certificates, to back that experience.

>From a brief web search, I'm not finding very much on the topic of key
continuity. The most relevant-looking result is
https://datatracker.ietf.org/doc/draft-gutmann-keycont/, which is an I-D
that expired in 2009, and does not appear to have been pursued since.

- Matt

[Peter Gutmann]

Matt Palmer <mpa...@hezmatt.org> writes:
>On Thu, Dec 12, 2024 at 09:56:11AM -0500, Jeffrey Walton wrote:
>> I share your concern over short-lived certificates, but for a
>> different reason: key continuity. Key continuity has proven to be a
>> much better security property than gratuitous key rotations based on
>> reading of tea leaves by tasseomancers.
>
>Do you have any citations you can share?

Google "SSH", that's been running for about the same time as TLS using key
continuity. TLS has a booming global cybercrime industry built around the
failure of certificates to deal with spoofing, SSH has very little in the way
of spoofing (granted they're very different protocols serving different
purposes).

>From a brief web search, I'm not finding very much on the topic of key
>continuity. The most relevant-looking result is
>https://datatracker.ietf.org/doc/draft-gutmann-keycont/, which is an I-D that
>expired in 2009, and does not appear to have been pursued since.

Bit of an odd choice to take an ancient expired RFC draft given the large
amount of research publications around this, commenting as the author of said
draft I had a go at it but given that there's an established billion-dollar
industry built around selling bit strings I figured working on an RFC that
pointed out alternatives was going to be a non-starter.

Peter.

[Matt Palmer]

On Fri, Dec 13, 2024 at 08:15:43AM +0000, Peter Gutmann wrote:
> Matt Palmer <mpa...@hezmatt.org> writes:
> >On Thu, Dec 12, 2024 at 09:56:11AM -0500, Jeffrey Walton wrote:
> >> I share your concern over short-lived certificates, but for a
> >> different reason: key continuity. Key continuity has proven to be a
> >> much better security property than gratuitous key rotations based on
> >> reading of tea leaves by tasseomancers.
> >
> >Do you have any citations you can share?
>
> Google "SSH", that's been running for about the same time as TLS using key
> continuity. TLS has a booming global cybercrime industry built around the
> failure of certificates to deal with spoofing, SSH has very little in the way
> of spoofing (granted they're very different protocols serving different
> purposes).

Yes, they're very, _very_ different protocols, serving very, _very_
different purposes. I do a lot of SSHing, but its to a fairly stable,
rarely-changing set of names. The set of machines I make HTTPS
connections to is far broader and ever-changing. TOFU for HTTPS would
be... really something.

> >From a brief web search, I'm not finding very much on the topic of key
> >continuity. The most relevant-looking result is
> >https://datatracker.ietf.org/doc/draft-gutmann-keycont/, which is an I-D that
> >expired in 2009, and does not appear to have been pursued since.
>
> Bit of an odd choice to take an ancient expired RFC draft given the large
> amount of research publications around this,

The research publications areen't coming up on DDG, but your draft was
-- that's why I made the "odd choice" of mentioning it. Would you be
able to share links to some more relevant reading material?

- Matt

[Peter Gutmann]

Matt Palmer <mpa...@hezmatt.org> writes:

>The research publications areen't coming up on DDG, but your draft was --
>that's why I made the "odd choice" of mentioning it. Would you be able to
>share links to some more relevant reading material?

I don't have references lying around but there's been quite a few publications
on trust models that look at this and related things like PGP's web of trust,
as well as evaluations of whether they work or not [0]. Not trying to be
abstruse here, I just don't have a record of things I would have read 15 years
ago, or perhaps merely glanced at 15 years ago.

Peter.

[0] Conflict-of-interest disclaimer, I'm the author of one of those, "Do Users
Verify SSH Keys?",
https://static.usenix.org/publications/login/2011-08/pdfs/Gutmann.pdf.

[Peter Gutmann]

Peter Gutmann <pgu...@cs.auckland.ac.nz> writes:

>Not trying to be abstruse here, I just don't have a record of things I would
>have read 15 years ago, or perhaps merely glanced at 15 years ago.

Just remembered one of them, "Perspectives: Improving SSH-style Host
Authentication with Multi-Path Probing" from 2008, on SSH-style key continuity
for TLS servers (it also has a multipath notary architecture to allow for
history-based assertions like "Offered key conflicts with cached key, but has
been consistently seen for Z days", but its main feature is the use of key
continuity). Use to be available as a browser extension, not sure if it
still is today.

Peter.

[Pierre Barre]

Hi Matt,

I was forwarded this thread, and as its creator, I wanted to reach out directly.

Would you mind sharing which specific features you're looking for to make it a suitable alternative for your needs?

Best regards,

Pierre Barre

[Rob Stradling]

> Many rely not on actually reading CT logs, but utilize crt.sh. It is

> great that this service exists, but I'd like to point out the following:

> * There's currently no other service like crt.sh. It's essentially a

>   single point of failure of a lot of "WebPKI security stuff" people do.

> * crt.sh is experiencing challenges with availability and cert

>   backlogs, likely because dealing with this volume is challenging.

Hi Hanno.  I just wanted to (belatedly) acknowledge this observation.

crt.sh has indeed been struggling with a growing log entry ingestion backlog over the past couple of months, but we've been able to put measures in place to reverse this trend.  More details at https://groups.google.com/g/crtsh/c/sgYa8RBaY1k/m/P5UmbXeZCQAJ.

> Imagining something like a more than 10x increase in cert volumes (which

> would be the consequence of 6-day certs as the norm) probably means

> many people will just stop utilizing CT to find security issues in the

> WebPKI ecosystem. Add to that the fact that, depending on how fast the

> Quantum Cryptopocalypse will materialize itself, we may also have to

> increase the size per certificate quite substantially.

> I think this should be considered when discussing very-short-lived

> certs.

I suspect that a 10x increase in cert volumes would pose scalability challenges for most CT logs and CT monitors!

---

From: dev-secur...@mozilla.org <dev-secur...@mozilla.org> on behalf of Hanno Böck <ha...@hboeck.de>
Sent: 12 December 2024 12:30
To: dev-secur...@mozilla.org <dev-secur...@mozilla.org>
Subject: Concerns about very-short-lived certificates

 

This Message Is From an External Sender

This message came from outside your organization.

Report Suspicious

 

Hi,

I hope this is a good place to share these thoughts, but given it's the
"all things CA" discussion place, it seems to fit.

In a recent post, Let's Encrypt mentions plans for very-short-lived
certificates with a lifetime of 6 days:

https://urldefense.com/v3/__https://letsencrypt.org/2024/12/11/eoy-letter-2024/__;!!J5K_pWsD!xw2f4HgBHZ3b9dxTyMQTPtk-lRzKKWqZgZV5yXAUSgeH0kwpc1r_BLGMoNvFMeQTGzvfObPy70rZ$

https://urldefense.com/v3/__https://itsec.hboeck.de/__;!!J5K_pWsD!xw2f4HgBHZ3b9dxTyMQTPtk-lRzKKWqZgZV5yXAUSgeH0kwpc1r_BLGMoNvFMeQTGzvfORbMCRJi$
https://urldefense.com/v3/__https://badkeys.info/__;!!J5K_pWsD!xw2f4HgBHZ3b9dxTyMQTPtk-lRzKKWqZgZV5yXAUSgeH0kwpc1r_BLGMoNvFMeQTGzvfOY6O9iuI$

-- 
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org

.
To view this discussion visit https://urldefense.com/v3/__https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/20241212133018.05ec7cae*40computer__;JQ!!J5K_pWsD!xw2f4HgBHZ3b9dxTyMQTPtk-lRzKKWqZgZV5yXAUSgeH0kwpc1r_BLGMoNvFMeQTGzvfObpq7AaB$.

[Matt Palmer]

On Sun, Dec 15, 2024 at 11:09:38PM -0800, Pierre Barre wrote:
> Hi Matt,
>
> I was forwarded this thread, and as its creator, I wanted to reach out
> directly.
>
> Would you mind sharing which specific features you're looking for to make
> it a suitable alternative for your needs?

Here's a few things off the top of my head:

* It requires JS to do anything useful.

* The search box only takes subdomains, not other identifiers I'm
commonly interested in (like SPKI fingerprints).

* You can't see any details about certificates without an account.

* Only a subset of search results are apparently available without a
paid account.

Do your own thing as you like, of course, but merklemap.com is a very,
very different beast to crt.sh.

- Matt

[Pierre Barre]

Hi Matt,

> > * It requires JS to do anything useful.

These days, most browsers support JavaScript. You might want to give one a try! ;-)

Joking aside, I get where you're coming from, but I don't think this is a hill worth dying on. That battle is mostly lost. Even if I personally preferred the way the "old web" used to work, the industry has largely moved on. Sticking to the old approach often means reinventing the wheel at every step, and I have more productive things to do.

I also think having a thin JS client as a website allows to design APIs in a sensible way as they are used first hand.

> * The search box only takes subdomains, not other identifiers I'm
> commonly interested in (like SPKI fingerprints).

You can currently do this by modifying the URL directly, for example:

https://www.merklemap.com/certificates/ba7924eedf9c95809bc4f46dce070b946560054e1ad7494e210627bfc57b358a

Not the best UX, I'll admit.

> * You can't see any details about certificates without an account.
>
> * Only a subset of search results are apparently available without a
> paid account.

Merklemap.com isn't sponsored, and the operating costs are substantial. The dataset is massive— even well-funded organizations struggle with the scale of this kind of data, as seen with https://letsencrypt.org/2024/03/14/introducing-sunlight/ even though CT log operators only store a subset of the data, and only provide a very limited set of APIs.

Merklemap, on the other hand, is exhaustive. It contains the full history of certificate transparency since its inception. The main database has just under 100 billion rows, spans dozens of terabytes on NVMe storage, and the search index is massive. The system is able to ingest around 100,000 certificates per second (which is useful to handle backlog). That kind of scale isn’t free to operate.

I strongly believe that for the health of the ecosystem, services like this need to be self-sustaining. A funded model ensures they can remain available and continue improving.

Best,
Pierre

> --
> You received this message because you are subscribed to the Google
> Groups "dev-secur...@mozilla.org" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dev-security-po...@mozilla.org.
> To view this discussion visit

> https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/99f0c9fc-76af-4f13-97ce-dba27b376b37%40mtasv.net.

[Phillip Hallam-Baker]

I can't see the point of 45 day certs. Either you do a year or you do 7 days. Pointless to do anything in the middle. I prefer to have systems fail quickly than after the consultant who installed them has disappeared.

The problem we have with TLS security is that the WebPKI was designed to make it as safe using a credit card online as in a bricks and mortar store. That is all it was ever built for. It was not built to enable encryption because the NSA and FBI was making that impossible. Those who were there remember 40 bit encryption.

The WebPKI is now being used for a large number of things and there are many it really isn't very good at. Google funded LetsEncrypt as a means of enabling ubiquitous encryption and preventing ISPs from intercepting Web pages to replace Google ads with ads from the ISP. And they did it in a way that broke the controls intended to protect people doing shopping online.

So the net result is that now we have a WebPKI that works fairly well for any service or device that is reliably connected to the public Internet. ACME doesn't really work well at the moment for IoT devices belonging to consumers because consumers don't have the technical skill, the DNS access etc. required to deploy for a home device. But that is fixable and something I hope to demonstrate a fix for at the Bangkok IETF next month.

I have a prototype that allows a user with the DNS handle @phill.hallambaker.com to configure devices and services in their domain. So I buy a NAS, I scan a QR code on it, give it the name 'nas.hallambaker.com' and the thing now talks to my network, is reachable as https://nas.hallambaker.com/ and I can log in with my DNS Handle using my '@nywhere' account.

That solves almost every use case except the 'what if Internet is out for more than 7 days and the cert life is limited to 6' and 'what if I lose the DNS name I am using for my handles'.

My solution to that is to actually configure the device with two sets of certificates, one under the public WebPKI and nas.hallambaker.com and the other under a private CA hierarchy as nas.hallambaker.alt. And that is under my own personal root and with certs that can have long lifespans because I actually have a functioning revocation system on the local private CA.

So I can of course install my private root into my browsers. But that is a bit of a pain for ordinary users and I don't like the idea of them installing private roots, a bad habit to start.

So imagine we have a CA like LetsEncrypt called LetsAuthenticate.com and it does two things

1) Reserves names in .alt on a first come first served basis.

2) Issues cross certificates to the party that reserved the name.

That solves the problem of creating a set of names and credentials for the home to use when it is off-grid and for writing internal scripts that have to survive a public domain name change.

[Jeffrey Walton]

On Sun, Feb 16, 2025 at 12:20 PM Pierre Barre <pierre...@barre.sh> wrote:
>
> > > * It requires JS to do anything useful.
>
> These days, most browsers support JavaScript. You might want to give one a try! ;-)
>
> Joking aside, I get where you're coming from, but I don't think this is a hill worth dying on. That battle is mostly lost. Even if I personally preferred the way the "old web" used to work, the industry has largely moved on. Sticking to the old approach often means reinventing the wheel at every step, and I have more productive things to do.
>
> I also think having a thin JS client as a website allows to design APIs in a sensible way as they are used first hand.

One counterpoint to JavaScript-enabled sites: I don't want JavaScript
running on login pages. JavaScript can egress credentials through the
DOM. Consider, there's a httponly flag to protect cookies from
JavaScript, but nothing to protect the precursor credentials from
JavaScript. Maybe the web needs a HTML tag for secure credential entry
that only the browser handles (and sends to the server).

As a datapoint, checkout all the unconstrained JavaScript that runs on
Alight's login pages. Alight is HR-as-a-Service and provides health
and retirement recordkeeping and benefits. Searching the web finds a
company called RTX that uses them; see
<http://www.yourtotalrewards.com/rtx>. There are trackers, traditional
analytics and AI analytics scraping the page. The promiscusity
instills no confidence in me.

Alight has serious data security problems. They are being investigated
for an undisclosed/unreported data breach; see MARTIN J.WALSH,
Secretary of Labor vs ALIGHT SOLUTIONS LLC (No. 21-3290 (7th Cir.
2022)). The company tortures users with {username, password, PIN,
3-each security questions}-tuples. And they do all the dumb things
with passwords, like composition and complexity requirements; and
gratuitous rotations. The company _does not_ support YubiKeys.

> > * The search box only takes subdomains, not other identifiers I'm
> > commonly interested in (like SPKI fingerprints).
>
> You can currently do this by modifying the URL directly, for example:
>
> https://www.merklemap.com/certificates/ba7924eedf9c95809bc4f46dce070b946560054e1ad7494e210627bfc57b358a
>
> Not the best UX, I'll admit.
>
> > * You can't see any details about certificates without an account.
> >
> > * Only a subset of search results are apparently available without a
> > paid account.
>
> Merklemap.com isn't sponsored, and the operating costs are substantial. The dataset is massive— even well-funded organizations struggle with the scale of this kind of data, as seen with https://letsencrypt.org/2024/03/14/introducing-sunlight/ even though CT log operators only store a subset of the data, and only provide a very limited set of APIs.
>
> Merklemap, on the other hand, is exhaustive. It contains the full history of certificate transparency since its inception. The main database has just under 100 billion rows, spans dozens of terabytes on NVMe storage, and the search index is massive. The system is able to ingest around 100,000 certificates per second (which is useful to handle backlog). That kind of scale isn’t free to operate.
>
> I strongly believe that for the health of the ecosystem, services like this need to be self-sustaining. A funded model ensures they can remain available and continue improving.

Jeff

[Matt Palmer]

On Sun, Feb 16, 2025 at 06:19:42PM +0100, Pierre Barre wrote:
> > > * It requires JS to do anything useful.
>
> These days, most browsers support JavaScript. You might want to give one a try! ;-)

Yeah, that's the kind of snide remark that gets you negative credibility
points. There are significant security and privacy downsides to
unfettered JavaScript execution.

Also, you asked for my time to provide you with assistance with your
commercial endeavour in identifying gaps between your service and crt.sh,
and one of the primary differences is that crt.sh does not require JS to
work.

> > * The search box only takes subdomains, not other identifiers I'm
> > commonly interested in (like SPKI fingerprints).
>
> You can currently do this by modifying the URL directly, for example:
>
> https://www.merklemap.com/certificates/ba7924eedf9c95809bc4f46dce070b946560054e1ad7494e210627bfc57b358a
>
> Not the best UX, I'll admit.

It's a terrible UX. Also, hitting that URL redirects to a sign-in page,
which highlights another *huge* difference to crt.sh: your service
cannot be used as a means to reference certificates in any public
discussion or incident report.

- Matt

[Pierre Barre]

Subject: Professionalism and Constructive Discussion

Matt,

Your response crosses the line from technical discussion into personal attacks. Calling my remark “snide” and implying it reduces my credibility is unprofessional. If you disagree, address the technical point rather than resorting to condescension.

Additionally, your claim about JavaScript security issues in Merklemap looks like trolling. There is no JS-specific security concern related to Merklemap, and, ironically, the very forum we’re using requires JavaScript to function.

I’m also not sure what credentials you hold to assess the credibility of others, but resorting to such remarks is just childish. Your tone and approach in this conversation come across as bullying rather than constructive dialogue. Since it doesn’t seem like this discussion will become more productive, I’ll leave it here.

Pierre.