Intent to Ship: Deprecate TLS SHA-1 server signatures

[David Adrian]

Contact emails

dad...@google.com

Explainer

None

Specification

https://www.rfc-editor.org/rfc/rfc9155.html

Summary

Chrome is removing support for signature algorithms using SHA-1 for server signatures during the TLS handshake. This does not affect SHA-1 support in server certificates, which was already removed, or in client certificates, which continues to be supported. SHA-1 can be temporarily re-enabled via the temporary InsecureHashesInTLSHandshakesEnabled enterprise policy. This policy will be removed in Chrome 123.

Blink component

Internals>Network>SSL

Search tags

tls, ssl, sha1

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

At most 0.02% of page loads use the SHA1 fallback. However, we cannot disambiguate between a flaky first connection, and actually requiring SHA1. We expect the actual amount is lower.

Gecko: Positive (https://github.com/mozilla/standards-positions/issues/812)

WebKit: Positive (https://github.com/WebKit/standards-positions/issues/196)

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None

Debuggability

n/a, this happens pre-devtools

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

No

Flag name on chrome://flags

use-sha1-server-handshakes

Finch feature name

DisableSHA1ServerSignature

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=658905

Launch bug

https://launch.corp.google.com/launch/4233200

Estimated milestones

Shipping on desktop	117
OriginTrial desktop last	116
OriginTrial desktop first	115
DevTrial on desktop	115

Shipping on Android	117
OriginTrial Android last	116
OriginTrial Android first	115
DevTrial on Android	115

OriginTrial webView last	116
OriginTrial webView first	115

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/4832850040324096

Links to previous Intent discussions

Intent to Experiment: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGkh42JZz%3De_TRVwumqgTj-A7543BR7JLBUR_GzVN_oOWhKVvg%40mail.gmail.com

This intent message was generated by Chrome Platform Status.

[Jeffrey Yasskin]

This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship". I'll let an API owner say if there's a reason to re-send it; probably there isn't.

On Mon, Sep 18, 2023 at 3:47 PM 'David Adrian' via blink-dev <blin...@chromium.org> wrote:

Contact emails

dad...@google.com

Explainer

None

The RFC's introduction at https://www.rfc-editor.org/rfc/rfc9155.html#name-introduction is a pretty good explainer for why we should remove SHA-1 signatures.

 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGkh42LiSGgfN1trVXfrmCW0Upk9r9GK4XYZQm5Y8RSzphn_DA%40mail.gmail.com.

[David Adrian]

> This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship".

You're absolutely right that it should be, unfortunately that's not the subject Chrome Status generated. I'll file an issue.

> The RFC's introduction at https://www.rfc-editor.org/rfc/rfc9155.html#name-introduction is a pretty good explainer for why we should remove SHA-1 signatures.

Agreed. Noting in general, there is a large process mismatch between TLS launches and the Blink launch process, as discussed in https://groups.google.com/a/chromium.org/g/blink-dev/c/CmlXjQeNWDI/m/r-AUe0OqAQAJ. That's why this Intent looks a little different.

As for the launch itself, I'll note it's been at 10% on Finch for a couple weeks and everything looks gray, so we should be safe to ramp up to 100%. The only thing of note was a correlation with an unrelated crash in Blink, since the deprecation rollout was fairly large. It only showed at 10%, not 1%.

[Jeffrey Yasskin]

On Mon, Sep 18, 2023 at 4:11 PM David Adrian <dad...@google.com> wrote:

> This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship".

You're absolutely right that it should be, unfortunately that's not the subject Chrome Status generated. I'll file an issue.

Oops, yes, you did everything right here. There's already https://github.com/GoogleChrome/chromium-dashboard/issues/2749 about changing this subject line, and now https://github.com/GoogleChrome/chromium-dashboard/issues/3346 to align the Chrome Status UI with the launching-features page.

[Yoav Weiss]

On Tue, Sep 19, 2023 at 1:35 AM 'Jeffrey Yasskin' via blink-dev <blin...@chromium.org> wrote:

On Mon, Sep 18, 2023 at 4:11 PM David Adrian <dad...@google.com> wrote:

> This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship".

You're absolutely right that it should be, unfortunately that's not the subject Chrome Status generated. I'll file an issue.

Oops, yes, you did everything right here. There's already https://github.com/GoogleChrome/chromium-dashboard/issues/2749 about changing this subject line, and now https://github.com/GoogleChrome/chromium-dashboard/issues/3346 to align the Chrome Status UI with the launching-features page.

> The RFC's introduction at https://www.rfc-editor.org/rfc/rfc9155.html#name-introduction is a pretty good explainer for why we should remove SHA-1 signatures.

Agreed. Noting in general, there is a large process mismatch between TLS launches and the Blink launch process, as discussed in https://groups.google.com/a/chromium.org/g/blink-dev/c/CmlXjQeNWDI/m/r-AUe0OqAQAJ. That's why this Intent looks a little different.

I wouldn't categorize it as a large process mismatch. But that's an orthogonal discussion.

 

As for the launch itself, I'll note it's been at 10% on Finch for a couple weeks and everything looks gray, so we should be safe to ramp up to 100%. The only thing of note was a correlation with an unrelated crash in Blink, since the deprecation rollout was fairly large. It only showed at 10%, not 1%.

On Mon, Sep 18, 2023 at 3:53 PM Jeffrey Yasskin <jyas...@google.com> wrote:

This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship". I'll let an API owner say if there's a reason to re-send it; probably there isn't.

On Mon, Sep 18, 2023 at 3:47 PM 'David Adrian' via blink-dev <blin...@chromium.org> wrote:

Contact emails

dad...@google.com

Explainer

None

The RFC's introduction at https://www.rfc-editor.org/rfc/rfc9155.html#name-introduction is a pretty good explainer for why we should remove SHA-1 signatures.

 

Specification

https://www.rfc-editor.org/rfc/rfc9155.html

Summary

Chrome is removing support for signature algorithms using SHA-1 for server signatures during the TLS handshake. This does not affect SHA-1 support in server certificates, which was already removed, or in client certificates, which continues to be supported. SHA-1 can be temporarily re-enabled via the temporary InsecureHashesInTLSHandshakesEnabled enterprise policy. This policy will be removed in Chrome 123.

Blink component

Internals>Network>SSL

Search tags

tls, ssl, sha1

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

At most 0.02% of page loads use the SHA1 fallback. However, we cannot disambiguate between a flaky first connection, and actually requiring SHA1. We expect the actual amount is lower.

Are we thinking that 0.02% is a loose upper bound? Is that correct?

Any way to sample a few sites to validate that assumption?

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANh-dXnM7SzAOh2y6hcuezDpo-yCW%3DtNg0%3D1ErEMCFN%3DSSpsQQ%40mail.gmail.com.

[Yoav Weiss]

On Tue, Sep 19, 2023 at 7:45 AM Yoav Weiss <yoav...@chromium.org> wrote:

On Tue, Sep 19, 2023 at 1:35 AM 'Jeffrey Yasskin' via blink-dev <blin...@chromium.org> wrote:

On Mon, Sep 18, 2023 at 4:11 PM David Adrian <dad...@google.com> wrote:

> This should probably be an "Intent to Deprecate and Remove" rather than an "Intent to Ship".

You're absolutely right that it should be, unfortunately that's not the subject Chrome Status generated. I'll file an issue.

Oops, yes, you did everything right here. There's already https://github.com/GoogleChrome/chromium-dashboard/issues/2749 about changing this subject line, and now https://github.com/GoogleChrome/chromium-dashboard/issues/3346 to align the Chrome Status UI with the launching-features page.

> The RFC's introduction at https://www.rfc-editor.org/rfc/rfc9155.html#name-introduction is a pretty good explainer for why we should remove SHA-1 signatures.

Agreed. Noting in general, there is a large process mismatch between TLS launches and the Blink launch process, as discussed in https://groups.google.com/a/chromium.org/g/blink-dev/c/CmlXjQeNWDI/m/r-AUe0OqAQAJ. That's why this Intent looks a little different.

I wouldn't categorize it as a large process mismatch. But that's an orthogonal discussion.

 

As for the launch itself, I'll note it's been at 10% on Finch for a couple weeks and everything looks gray, so we should be safe to ramp up to 100%. The only thing of note was a correlation with an unrelated crash in Blink, since the deprecation rollout was fairly large. It only showed at 10%, not 1%.

 How would we know of breakage in those 10%? Would that look like users filing issues? Something else?

Also, are those 0.02% driven by origins? Specific user platforms? Something else?

[David Adrian]

>  How would we know of breakage in those 10%? Would that look like users filing issues? Something else?

Usually we see it reflected in DAU and users filing issues. This is what happened when we had issues during root store deployment. We also have Net.SSL_Connection_Error, which does show a slight decrease in OK (0.009%) and some shifting of errors around. However, this doesn't seem to have manifested in any other changes.

> Also, are those 0.02% driven by origins? Specific user platforms? Something else?

Specific server platforms, +David Benjamin can provide more detail.

[David Benjamin]

As with anything in TLS, the protocol is client-offer / server-select, which means we pick up all the implications of that. We can measure what the server selects, but that doesn't actually tell us what will happen if we change the ClientHello:

- Perhaps the server preferentially chooses SHA-1, but will pick SHA-2 if not offered. (I.e. SHA-1-preferring instead of SHA-1-requiring)

- Perhaps the server picks SHA-2 regardless but, for some odd reason, breaks when SHA-1 is not offered. This could be a buggy, non-compliant fingerprinting script, or some weird behavior.

Usually, these effects are small enough that we just measure the server selection and don't worry about it. Here, we could not do that. First, we knew from previously sampling sites that some servers (older IIS) fall in the first category. These servers would not break, but would seem to break if we measure the server selection. We also knew, again from sampling and due to the multiple places where this ClientHello field is used, the second category also shows up (in many cases, also older IIS). These servers would break but would seem not to if we measure the server selection.

Thus, we used a different measurement strategy. We send a SHA-1-less ClientHello and then, on error, retry with SHA-1 restored. This does not work for security (an attacker can always force us onto the second round), but it helps counteract the above effects for a more accurate measurement... up to a point. Up to a point because, in exchange for clearing those effects, we pick a different effect: transient network errors will also trigger the retry. And thus our numbers inherently have noise at the scale of transient network errors.

See also https://groups.google.com/a/chromium.org/g/blink-dev/c/RShdgyaDoX4/m/ly5-WQEBBgAJ, where we previously discussed this same client-offer / server-select property.

This is a difference from web platform things, where we can simply observe in Chrome what APIs the site uses and moves on.

 

Any way to sample a few sites to validate that assumption?

I'm not sure what you mean. Networks occasionally flake, so any kind of fallback measurements will capture some flakiness. Sampling sites won't tell us anything about that.

 

Also, are those 0.02% driven by origins? Specific user platforms? Something else?

This is 0.02% of TLS connections (sorry, page loads was a typo). This is another of those differences between networking and web platform features. The denominator is unavoidably different because of where in the stack this logic applies. Where it is actually an incompatibility with a site, it will be based on the origin. This is almost always due to a bug in the server software because every TLS 1.2 implementations supports SHA-2. (E.g. very very old OpenSSLs, with several known, unrelated security vulnerabilities have a bug where, with some server software, mess this up.)

Where it is a transient network error above it is, well, a transient network error and presumably driven by network conditions.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfVPG6np80msDXGDyzqOMA6E-7mtqFQpDSw8w5m3X%3DEKOg%40mail.gmail.com.

[Yoav Weiss]

For web platform features, where we have both usecounters in the HTTPArchive and potentially UKM, we can gather potentially breaking URLs/origins, grab a few dozens random names from the list and deep dive into them to see if they show breakage or not.

I'm not sure that we have anything similar for the netstack, where we could e.g. report the failures in ways that enable us to later follow up and run a similar manual check. (e.g. by reporting the failing DNS name and/or IP)

 

 

Also, are those 0.02% driven by origins? Specific user platforms? Something else?

This is 0.02% of TLS connections (sorry, page loads was a typo).

OK, that seems better, as presumably the denominator is smaller then it would be for page loads (which is what we typically use). Do you have a sense of the difference between denominators? (so, how many page loads typically happen per TLS connection?)

[David Adrian]

There are approximately 30x TLS connections relative to pageloads, due to subresources. Once you have a TLS connection open for a main frame, I'm not sure how many page loads happen across it. Probably a lot, but it's still dominated by subresources.

In practice, the 0.02% bound appears to have shaken out to sub 0.01% (0.009%), determined by looking at differences in the "OK" result for Net.SSL_Connection_Error.

[Yoav Weiss]

LGTM1 to remove support, given that 0.009% of TLS connections divided by ~30 is roughly at our typical threshold. Also given the dominance of subresource TLS connections, it's likely that breakage would hit one of those subresources, making it less likely to cause functional damage (compared to e.g. the navigation request TLS connection).  

[Philipp Hancke]

Should/can this also be applied to WebRTC's DTLS connections?

(we don't have numbers but that can be fixed)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfXorTe7SSPt8iZt5ZjVi0Qh8qd%2BPdEGMOzSQj45e0Z1VQ%40mail.gmail.com.

[David Benjamin]

Ah yeah, we should apply this to WebRTC too. Based on the kinds of issues we've seen, hopefully that one will be easy (I wouldn't expect WebRTC to be impacted by the kinds of server bugs we've seen), but certainly we'll need numbers.

Measurement is a little complicated for HTTPS (we had to do this fallback dance to avoid a ton of false positives on old Schannel behavior, which is why we had to pick up transient network errors as a different source of false positives). But since WebRTC talks to different kinds of peers, I suspect you all can just histogram the server-selected algorithm with SSL_get_peer_signature_algorithm, and just assume that's an accurate enough predictor. (TLS is client-offer / server-select, so we never learn the server's full capabilities, only what it happened to pick in response to our ClientHello. This makes measurements inherently more complex than measuring JavaScript API usage, but in simple cases just measuring the server selection gives good enough numbers.)

[David Benjamin]

To clarify, I meant that we should apply this to WebRTC in a separate launch. This one will just be HTTPS. We don't have numbers or a flag for WebRTC right now, and we usually end up doing WebRTC separately anyway, for better or worse. :-)

[Daniel Bratell]

LGTM2 (for the original request, just to be clear)

I assume it will have the normal flag that will allow it to be turned off if our interpretation is incorrect, since it is so hard to measure with a high accuracy.

/Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF8qwaB5P6Ys%3DxMyB%3D7HvLJEa0Axp5uCcfbPQqurN2ADdasYgQ%40mail.gmail.com.

[Chris Harrelson]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/29ca51ba-5890-4767-82d1-0e4e21669374%40gmail.com.

[David Benjamin]

Yup, it's finch-gated. (Finch feature name in the original email.)

[Ivan Radisson]

Hi

The recent change in Chrome has broken the connection to many websites... ☹️

https://community.letsencrypt.org/search?q=ERR_SSL_PROTOCOL_ERROR%20order%3Alatest

https://talk.plesk.com/search/2147545/?q=ERR_SSL_PROTOCOL_ERROR&t=post&o=relevance

https://community.spiceworks.com/search?filter_by=d30&query=err_ssl_protocol_error

[Ivan Radisson]

Some reports can be found here as well:

https://bugs.chromium.org/p/chromium/issues/list?q=ERR_SSL_PROTOCOL_ERROR&can=2

On Wednesday, September 27, 2023 at 6:04:54 PM UTC+1 David Benjamin wrote:

[David Benjamin]

Hi Ivan,

Yeah, we've had a couple reports come in from the bugtracker. Most seem to have been caused by an extremely outdated OpenSSL (several even still vulnerable to Heartbleed!) and were resolved by the servers updating their OpenSSL. I've put together a troubleshooting doc here with the server bugs we're aware of:

https://chromium.googlesource.com/chromium/src/+/main/docs/security/tls-sha1-server-signatures.md

I'll see about replying to the ones from outside our bugtracker. Thanks for pointing them out!