Intent to Deprecate and Remove: Trust in existing Symantec-issued Certificates

136,911 views
Skip to first unread message

Ryan Sleevi

unread,
Mar 23, 2017, 12:03:20 PM3/23/17
to blink-dev, awha...@chromium.org

Note: Historically, the Google Chrome team has not used the Blink Process for Certificate Authority-related security issues, of which there have been a number over the years. However, we are interested in exploring using this process for such changes, as it provides a greater degree of transparency and public participation. Based on the level of participation and feedback we receive, we may consider using this for the future. However, as CA-related security incidents may require immediate response to protect users, this should not be seen as a guarantee that this process can be used in future incident responses.


Primary eng (and PM) emails:

rsl...@chromium.org awha...@chromium.org


Summary

Since January 19, the Google Chrome team has been investigating a series of failures by Symantec Corporation to properly validate certificates. Over the course of this investigation, the explanations provided by Symantec have revealed a continually increasing scope of misissuance with each set of questions from members of the Google Chrome team; an initial set of reportedly 127 certificates has expanded to include at least 30,000 certificates, issued over a period spanning several years. This is also coupled with a series of failures following the previous set of misissued certificates from Symantec, causing us to no longer have confidence in the certificate issuance policies and practices of Symantec over the past several years. To restore confidence and security of our users, we propose the following steps:

  • A reduction in the accepted validity period of newly issued Symantec-issued certificates to nine months or less, in order to minimize any impact to Google Chrome users from any further misissuances that may arise.

  • An incremental distrust, spanning a series of Google Chrome releases, of all currently-trusted Symantec-issued certificates, requiring they be revalidated and replaced.

  • Removal of recognition of the Extended Validation status of Symantec issued certificates, until such a time as the community can be assured in the policies and practices of Symantec, but no sooner than one year.


Motivation

As captured in Chrome’s Root Certificate Policy, root certificate authorities are expected to perform a number of critical functions commensurate with the trust granted to them. This includes properly ensuring that domain control validation is performed for server certificates, to audit logs frequently for evidence of unauthorized issuance, and to protect their infrastructure in order to minimize the ability for the issuance of fraudulent certs.


On the basis of the details publicly provided by Symantec, we do not believe that they have properly upheld these principles, and as such, have created significant risk for Google Chrome users. Symantec allowed at least four parties access to their infrastructure in a way to cause certificate issuance, did not sufficiently oversee these capabilities as required and expected, and when presented with evidence of these organizations’ failure to abide to the appropriate standard of care, failed to disclose such information in a timely manner or to identify the significance of the issues reported to them.


These issues, and the corresponding failure of appropriate oversight, spanned a period of several years, and were trivially identifiable from the information publicly available or that Symantec shared.


The full disclosure of these issues has taken more than a month. Symantec has failed to provide timely updates to the community regarding these issues. Despite having knowledge of these issues, Symantec has repeatedly failed to proactively disclose them.  Further, even after issues have become public, Symantec failed to provide the information that the community required to  assess the significance of these issues until they had been specifically questioned. The proposed remediation steps offered by Symantec have involved relying on known-problematic information or using practices insufficient to provide the level of assurance required under the Baseline Requirements and expected by the Chrome Root CA Policy.


In January 2015, Symantec-issued certificates represented more than 30% of the valid certificates by volume. While changes in the CA ecosystem have seen that share decrease over the past two years, there is still a significant compatibility risk for an immediate and complete distrust. Further, due to overall TLS ecosystem concerns, we understand that it may take non-trivial effort for some site operators to find suitable solutions, as the need to support older devices may necessitate the use of particular CAs, meaning that distrust of new certificates also has significant compatibility risk.


To balance the compatibility risks versus the security risks, we propose a gradual distrust of all existing Symantec-issued certificates, requiring that they be replaced over time with new, fully revalidated certificates, compliant with the current Baseline Requirements. This will be accomplished by gradually decreasing the ‘maximum age’ of Symantec-issued certificates over a series of releases, distrusting certificates whose validity period (the difference of notBefore to notAfter) exceeds the specified maximum.


The proposed schedule is as follows:

Chrome 59 (Dev, Beta, Stable): 33 months validity (1023 days)

Chrome 60 (Dev, Beta, Stable): 27 months validity (837 days)

Chrome 61 (Dev, Beta, Stable): 21 months validity (651 days)

Chrome 62 (Dev, Beta, Stable): 15 months validity (465 days)

Chrome 63 (Dev, Beta): 9 months validity (279 days)

Chrome 63 (Stable): 15 months validity (465 days)

Chrome 64 (Dev, Beta, Stable): 9 months validity (279 days)


The proposed schedule attempts to avoid making changes in Chrome 63 Stable, as that would be released during the winter holiday production freeze many organizations undergo. This is solely to reduce disruption for site operators and users, and attempts to resume with Chrome 64 following the holiday season. Further, the practical impact of the changes in Chrome 59 and 60 are relatively minimal, due to many of the certificates issued during that period of time being issued using SHA-1, which is no longer supported for certificates in Chrome.


In addition, we propose to require that all newly-issued certificates must have validity periods of no greater than 9 months (279 days) in order to be trusted in Google Chrome, effective Chrome 61. This ensures that the risk of any further misissuance is, at most, limited to nine months, and more importantly, that if any further action is warranted or necessary, that the entire ecosystem can migrate within that time period, thus minimizing the risk of further compatibility issues.


By combining these two steps, we can ensure that the level of assurance in Symantec-issued certificates is able to match what is expected by Google Chrome and the ecosystem, and that the risks posed both from past and possible future misissuance is minimized as much as possible.


Given the nature of these issues, and the multiple failures of Symantec to ensure that the level of assurance provided by their certificates meets the requirements of the Baseline Requirements or Extended Validation Guidelines, we no longer have the confidence necessary in order to grant Symantec-issued certificates the “Extended Validation” status. As documented with both the current and past misissuance, Symantec failed to ensure that the organizational attributes, displayed within the address bar for such certificates, meet the level of quality and validation required for such display. Therefore, we propose to remove such indicators, effective immediately, until Symantec is able to demonstrate the level of sustained compliance necessary to grant such trust, which will be a period no less than a year. After such time has passed, we will consider requests from Symantec to re-evaluate this position, in collaboration with the broader Chromium community.


Compatibility and Interoperability Risk

As with any reduction in trust in a Certificate Authority, this poses a non-trivial degree of compatibility risk. This is because site operators desire to have their certificates recognized in all client browsers, and if one or more browsers fail to trust a given CA, this is prevented from happening.


On the other hand, all site operators expect that certificates will only be issued for their domains upon their request, and the failure to have that assurance significantly undermines the security of HTTPS for both site operators and users.


This compatibility risk is especially high for Symantec-issued certificates, due to their acquisition of some of the first CAs, such as Thawte, Verisign, and Equifax, which are some of the most widely supported CAs. Distrusting such CAs creates further difficulty for providing secure connections to both old and new devices alike, due to the need to ensure the CA a site operator uses is recognized across these devices.


Further, the immediate distrust of a CA, as has been necessary in the past, can significantly impact both site operators and users. Site operators are forced to acquire certificates from other CAs, without having the opportunity and time to research which CAs best meet their needs, and users encounter a substantial number of errors until those site operators act, conditioning them to ignore security warnings. In the event that only a single browser distrusts such a CA, the error is often seen as the browser’s fault, despite it being a failure of the CA to provide the necessary level of assurance, and the site operator to respond in a timely fashion.


Assessing the compatibility risk with both Edge and Safari is difficult, because neither Microsoft nor Apple communicate publicly about their changes in trust prior to enacting them.


While Mozilla conducts their discussions regarding Certificate Authorities in public, and were the first to be alerted of these latest issues, they have not yet begun discussion of the next steps to how best to protect their users. Our hope is that this proposal may be seen as one that appropriately balances the security and compatibility risks with the needs of site operators, browsers, and users, and we welcome all feedback.


Alternative implementation suggestion for web developers

This proposal allows for web developers to continue to use Symantec issued certificates, but will see their validity period reduced. This ensure that web developers are aware of the risk and potential of future distrust of Symantec-issued certificates, should additional misissuance events occur, while also allowing them the flexibility to continue using such certificates should it be necessary.


Usage information from UseCounter:

For a variety of non-technical reasons, we do not currently instrument the usage of CAs. Further, few public metrics exist for intersecting usage information with the validity period, since only certificates valid greater than nine months will be affected outside of their normal replacement cycle. From Mozilla Firefox’s Telemetry, we know that Symantec issued certificates are responsible for 42% of certificate validations. However, this number is not strictly an indicator for impact, as this number is biased towards counting certificates for heavily-trafficked sites, and whose issuance is fully automated and/or whose validity periods will be unaffected, thus significantly overstating impact. By phasing such changes in over a series of releases, we aim to minimize the impact any given release poses, while still continually making progress towards restoring the necessary level of security to ensure Symantec issued certificates are as trustworthy as certificates from other CAs.

Vincent Lynch

unread,
Mar 23, 2017, 12:35:16 PM3/23/17
to blink-dev
"Therefore, we propose to remove [EV] indicators, effective immediately,"


Two questions about this:

1. Can you clarify what "effective immediately" means? Will this occur with the next stable release of Chrome? Or will an update be pushed to the current stable version?

2. Can you confirm that the removal of EV indicators will affect *all* Symantec-issued EV certificates, regardless of their compliance with Chrome's reduced validity schedule?

Ryan

unread,
Mar 23, 2017, 12:48:32 PM3/23/17
to blink-dev


On Thursday, March 23, 2017 at 12:35:16 PM UTC-4, Vincent Lynch wrote:
"Therefore, we propose to remove [EV] indicators, effective immediately,"


Two questions about this:

1. Can you clarify what "effective immediately" means? Will this occur with the next stable release of Chrome? Or will an update be pushed to the current stable version?

As this is a security related incident, the proposal is to land this in Trunk and update Dev and Beta to reflect this, treating this as "Medium" severity per https://www.chromium.org/developers/severity-guidelines by treating this as an address bar spoofing issue.
 
2. Can you confirm that the removal of EV indicators will affect *all* Symantec-issued EV certificates, regardless of their compliance with Chrome's reduced validity schedule?

Correct. Extended Validation certificates are recognition of the processes and controls related to validating the information in the certificates is accurate and correct. The issues highlighted have undermined that confidence. The validity issue is separate and mitigates future issues that may arise or be discovered.

kevin...@ankuraconsulting.com

unread,
Mar 23, 2017, 1:03:16 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
Can you please clarify if any related CAs are affected, such as GeoTrust and Thawte, are affected?


On Thursday, March 23, 2017 at 12:03:20 PM UTC-4, Ryan Sleevi wrote:

Ryan Sleevi

unread,
Mar 23, 2017, 1:06:41 PM3/23/17
to kevin...@ankuraconsulting.com, blink-dev, awha...@chromium.org, Ryan Sleevi
On Thu, Mar 23, 2017 at 1:03 PM, <kevin...@ankuraconsulting.com> wrote:
Can you please clarify if any related CAs are affected, such as GeoTrust and Thawte, are affected?

All Symantec issued certificates. GeoTrust and Thawte are CAs operated by Symantec, simply afforded different branding.

While this list may need to be updated for some recently created roots, https://chromium.googlesource.com/chromium/src/+/master/net/data/ssl/symantec/README.md may accurately capture the state of impact

pzb...@gmail.com

unread,
Mar 23, 2017, 1:30:36 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
On Thursday, March 23, 2017 at 9:03:20 AM UTC-7, Ryan Sleevi wrote:

To balance the compatibility risks versus the security risks, we propose a gradual distrust of all existing Symantec-issued certificates, requiring that they be replaced over time with new, fully revalidated certificates, compliant with the current Baseline Requirements. This will be accomplished by gradually decreasing the ‘maximum age’ of Symantec-issued certificates over a series of releases, distrusting certificates whose validity period (the difference of notBefore to notAfter) exceeds the specified maximum.


The proposed schedule is as follows:

Chrome 59 (Dev, Beta, Stable): 33 months validity (1023 days)

Chrome 60 (Dev, Beta, Stable): 27 months validity (837 days)

Chrome 61 (Dev, Beta, Stable): 21 months validity (651 days)

Chrome 62 (Dev, Beta, Stable): 15 months validity (465 days)

Chrome 63 (Dev, Beta): 9 months validity (279 days)

Chrome 63 (Stable): 15 months validity (465 days)

Chrome 64 (Dev, Beta, Stable): 9 months validity (279 days)


The proposed schedule attempts to avoid making changes in Chrome 63 Stable, as that would be released during the winter holiday production freeze many organizations undergo. This is solely to reduce disruption for site operators and users, and attempts to resume with Chrome 64 following the holiday season. Further, the practical impact of the changes in Chrome 59 and 60 are relatively minimal, due to many of the certificates issued during that period of time being issued using SHA-1, which is no longer supported for certificates in Chrome


I'm trying to convert this to concrete terms.

If a sever authentication certificate has a notBefore date of 2017-03-24T00:00:00Z and a notAfter date of 2018-03-23T23:59:59Z (approximately 365 days), and this intent is implemented as described, is it accurate that this certificate will be accepted by Chrome Beta until Chrome 63 and by Chrome Stable 64?
 

In addition, we propose to require that all newly-issued certificates must have validity periods of no greater than 9 months (279 days) in order to be trusted in Google Chrome, effective Chrome 61.


How is "newly-issued" defined?
 

This ensures that the risk of any further misissuance is, at most, limited to nine months, and more importantly, that if any further action is warranted or necessary, that the entire ecosystem can migrate within that time period, thus minimizing the risk of further compatibility issues.


By combining these two steps, we can ensure that the level of assurance in Symantec-issued certificates is able to match what is expected by Google Chrome and the ecosystem, and that the risks posed both from past and possible future misissuance is minimized as much as possible.


 Thanks,
Peter

Andrew Ayer

unread,
Mar 23, 2017, 2:02:14 PM3/23/17
to blink-dev
Hi Ryan,

What restrictions, if any, will be placed on the reuse of validation
information?

Regards,
Andrew

ben.m...@gmail.com

unread,
Mar 23, 2017, 2:07:03 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
(just to be super clear I'm writing this in a personal capacity and not representing the thoughts of my employer)

As a developer I feel like this plan might be excessively confusing. Given this list of dates, how would I calculate when I have to do something? Do I even remember when my cert was issued? Also, this process seems to aim at a gradual breakage, but from a developer's perspective this is not gradual -- I own a single site, and during one of the chrome releases my site will break the day that release goes to stable.

In addition the lack of a single deadline would seem to make it hard for a campaign to get site owners to replace these certs to be effective. If I were in the shoes of Symantec, I'd start emailing my customers offering them a free renewal of their certificate. If I were another CA I'd start offering Symantec customers a free replacement for their old certificate. If customers are affected over the course of a year it's hard for people to make these kinds of campaigns high profile.

Did you guys consider something where there would be an escalating level of distrust (eg, in one version you get a warning rather than a green lock, the next release a red not secure, then a intersticial)? This would allow attentive operators who might not hear about this change otherwise to figure out that there is an issue before their site breaks completely. It also gives all parties involved a small number of times where they can promote this change.

If other browser vendors might implement a similar policy it could make sense to tie those events to dates rather than releases so that this type of promotion could be more effective.



On Thursday, March 23, 2017 at 9:03:20 AM UTC-7, Ryan Sleevi wrote:

kane...@gmail.com

unread,
Mar 23, 2017, 2:28:26 PM3/23/17
to blink-dev
> Did you guys consider something where there would be an escalating level of distrust (eg, in one version you get a warning rather than a green lock, the next release a red not secure, then a interstitial)? This would allow attentive operators who might not hear about this change otherwise to figure out that there is an issue before their site breaks completely. It also gives all parties involved a small number of times where they can promote this change.

As a developer, this sounds reasonable to me.

Wording & iconography such as this might be reusable for future incidents:

#######
(i) Warning | https://example.com

Your connection to this site is not fully secure

The site presented old identity information which needs to be revalidated. Learn more
#######

- Learn more would link to a help center article saying you need to get your certificate reissued
- Not overly alarmist - use of Info icon, "Warning" instead of "Not Secure"
- Text does not imply operator is at fault, but they do need to take action
- Notice could be upgraded to red in a later release, then to interstitial
- Needs new translation work - "Warning" is not an already used term in the address bar


Kane York

Chris Palmer

unread,
Mar 23, 2017, 2:33:27 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org, ben.m...@gmail.com
On Thursday, March 23, 2017 at 11:07:03 AM UTC-7, ben.m...@gmail.com wrote:

As a developer I feel like this plan might be excessively confusing. Given this list of dates, how would I calculate when I have to do something?

This is a good and important question. A big part of the solution is automating re-issuance. Automation is a good value-add that various CAs and resellers are offering now, because lots of people need it for lots of reasons. I won't endorse any particular vendor, but there are several vendors you can choose from that are quite affordable and who provide automated re-issuance.

Combined with the gradual move to certificates with shorter lifespans anyway (as a way of coping with problems like this, and with the difficulty of certificate revocation generally), automation is a necessity going forward.

PhistucK

unread,
Mar 23, 2017, 2:36:28 PM3/23/17
to Ben Maurer, blink-dev, awha...@chromium.org, Ryan Sleevi
Perhaps this just needs a similar treatment to the one SHA-1 got/mixed content get (simply showing a website as secure, but not emphasizing that it is insecure). No interstitial on one hand, but also not shown as secure on the other hand.
I think this is the best outcome, as Symantec has (repeatedly) demonstrated that it cannot be trusted and not showing a website as secure indicates that it cannot be completely trusted (which is the case).

Also, some warning in the console might help (with a cutoff date, obviously), in anticipation for an interstitial or some other drastic action.


​Also, note that the iconography and taxonomy was simplified. There are three states, if I remember correctly (four, if you count Extended Validation​, but ignore that for now) - Secure (great HTTPS), Not Secure (for form filling HTTP websites, data URLs and so on) and none (HTTP without forms and HTTPS with problems). A warning signal is probably a step back and does not mean a lot to user.


PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

lloth...@gmail.com

unread,
Mar 23, 2017, 7:30:44 PM3/23/17
to blink-dev
Could you put the dates those versions hit stable in the original post?

evan...@gmail.com

unread,
Mar 23, 2017, 9:12:35 PM3/23/17
to blink-dev, lloth...@gmail.com
See: https://www.chromium.org/developers/calendar

2017

 Release   Estimated Week of Stable
56 Jan 31st, 2017
57 Mar 14th, 2017
58 Apr 25th, 2017
59 Jun 6th, 2017
60 Aug 1st, 2017
61 Sep 12th, 2017
62 Oct 24th, 2017
63 Dec 12th, 2017

tpg...@gmail.com

unread,
Mar 23, 2017, 9:52:14 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
I would like more details on the issues. "...allowed least four parties access to their infrastructure in a way to cause certificate issuance..." is pretty vague. Did these 4 parties then issue 30,000 certificates? What's wrong with these certificates exactly?

The issues previously known are 1) "test" certificates issued without domain owner's consent, though Symantec claimed that they were only used internally; 2) certificates issued for unregistered domains.

Google Internet Authority G2 CA that issues many of Google's TLS certificates is signed by GeoTrust Global CA, which presumably is owned by Symantec now. So do Google's own certificates fall into the "gradually untrustworthy" category?

jus...@gerhardt.link

unread,
Mar 23, 2017, 10:53:59 PM3/23/17
to blink-dev
Google semi-recently purchased a trusted root ca certificate.

matt.so...@gmail.com

unread,
Mar 23, 2017, 11:11:00 PM3/23/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
When will we know if these proposals are accepted and will become reality?

Rik Cabanier

unread,
Mar 23, 2017, 11:19:30 PM3/23/17
to matt.so...@gmail.com, blink-dev, awha...@chromium.org, rsl...@chromium.org
On Thu, Mar 23, 2017 at 8:10 PM, <matt.so...@gmail.com> wrote:
When will we know if these proposals are accepted and will become reality?

Usually, once 3 blink owners send a lgtm (=looks good to me) to this email thread, the intent is accepted and the engineer's changes will be accepted into the chromium code base.
 

--

Ryan Sleevi

unread,
Mar 23, 2017, 11:52:00 PM3/23/17
to Peter Bowen, blink-dev, awha...@chromium.org, Ryan Sleevi
On Thu, Mar 23, 2017 at 1:30 PM, <pzb...@gmail.com> wrote:
I'm trying to convert this to concrete terms.

If a sever authentication certificate has a notBefore date of 2017-03-24T00:00:00Z and a notAfter date of 2018-03-23T23:59:59Z (approximately 365 days), and this intent is implemented as described, is it accurate that this certificate will be accepted by Chrome Beta until Chrome 63 and by Chrome Stable 64?

I believe you meant "accepted by Chrome Beta (until Chrome 63) and by Chrome Stable (until Chrome 64)", correct? If so, that is correct.

Alternatively stated, using the end-goal as the measuring stick, if Chrome 64 were to come out on January 30, 2018 (more or less 6 weeks after December 12, modulo holidays), then the earliest accepted certificate would be one whose notBefore was on or after April 26, 2017 (279 days before January 30, 2018). If Chrome 64 were used on January 31, 2018, then the earliest accepted certificate would be one whose notBefore was issued on or after April 27, 2017. Both of these would only work if their validity periods were less than or equal to 279 days.
 

In addition, we propose to require that all newly-issued certificates must have validity periods of no greater than 9 months (279 days) in order to be trusted in Google Chrome, effective Chrome 61.


How is "newly-issued" defined?

Chrome 60 is estimated to be branched the week of May 25, 2017, as detailed at https://www.chromium.org/developers/calendar . On or after that branch point, in which 'trunk' becomes the intended and eventual Chrome 61 development tree, a change would be landed to require all certificates issued on or after that date have validity periods less than or equal to nine months. For example, a certificate issued on May 26, 2017 would need to expire on or before March 1, 2018. This would work through the normal development cycle of Canary -> Dev -> Beta and be released to Chrome Stable during the week of September 12, 2017. Note these represent estimated dates, as mentioned on the page.

The proposed definition for this is to rely upon the notBefore as an appropriate signal. This does create risk, for example, if a certificate issued on May 26, 2017 has its "notBefore" set to a date prior to the actual issuance date. An alternative definition that may mitigate that risk, but provide site operators less flexibility, is to use the earliest accepted Signed Certificate Timestamp presented with the certificate (either within the certificate, via a stapled OCSP response, or with the TLS extension), and rely on the Trusted Certificate Transparency Logs to provide an accurate timestamp. This provides less flexibility for other Chromium embedders (in that it relies on Certificate Transparency, for which may not be an appropriate dependency for these embedders), and less flexibility for site operators (in that it measures "first disclosed", not necessarily issuance date), but can provide a stronger security guarantee by providing a counter-signature as to the earliest possible time the certificate was 'publicly' known.



As a proposal, I'm especially looking for feedback as to any potential interoperability and compatibility risks, and welcome suggestions on how best to minimize. If the Blink OWNERS would find it helpful and useful, and an appropriate use of WICG, this could be iterated further in a WICG Repository to provide a more formal explainer and pseudo-code to reduce that risk. As I mentioned, this is a particularly new and unique approach for something that has historically been announced post-facto of implementation, and feedback is welcome on how best to minimize risks.

Ryan Sleevi

unread,
Mar 23, 2017, 11:58:24 PM3/23/17
to Andrew Ayer, blink-dev


On Thu, Mar 23, 2017 at 2:01 PM, Andrew Ayer <ag...@andrewayer.name> wrote:
What restrictions, if any, will be placed on the reuse of validation
information?

For better or worse, the reuse of validation information is not web-visible, and as a consequence, is effectively not a quantifiable part of the Web Platform, beyond the audit engagement of a Certificate Authority .

However, this also highlights a gap in incompletely addressing the security concerns. For example, if the information reused was obtained by not following the proper and expected procedures - such as by one of the Delegated Third Parties who were improperly responsible for validating the information that appeared within these 30,000 certificates - then the issuance of a new certificate would fail to meaningfully ensure the information has been appropriately validated.

A way to mitigate these concerns - and ensure the proper level of security - would be to require that any information that appears within certificates issued after a particular date ONLY rely on information which Symantec has independently validated, consistent with the Baseline Requirements.

Ryan Sleevi

unread,
Mar 24, 2017, 12:09:57 AM3/24/17
to Ben Maurer, blink-dev, awha...@chromium.org, Ryan Sleevi
On Thu, Mar 23, 2017 at 2:07 PM, <ben.m...@gmail.com> wrote:
(just to be super clear I'm writing this in a personal capacity and not representing the thoughts of my employer)

As a developer I feel like this plan might be excessively confusing. Given this list of dates, how would I calculate when I have to do something? Do I even remember when my cert was issued? Also, this process seems to aim at a gradual breakage, but from a developer's perspective this is not gradual -- I own a single site, and during one of the chrome releases my site will break the day that release goes to stable.

This is the unfortunate reality of the CA ecosystem, in which the trust afforded to any given site is binary, but the trust afforded to a CA is expressed in gradations, dependent upon how the certificate was validated and when it was validated.

The simple answer would be that, if adopted by Blink OWNERS, then to minimize all parties should strive to replace their certificates with versions whose validity period is less than or equal to 279 days (9 months). By replacing the existing certificates with such a certificate, there should be no negative effect. The remaining dates relate to a ratcheting enforcement that attempts to minimize the breakage from any given release.

Alternatively, should it be that measuring by validity period represents too great an ecosystem concern, an alternative approach would be to express this requirement as "The certificate was issued on or after this date" - such as Chrome 59 rejecting certificates issued prior to 1023 days than the current date. This approach would replace certificates in batches based on issuance. Accompanied with a requirement that certificates issued on or after the proposed Chrome 61 'open' date (the day on or after the Chrome 60 "branch" date of May 25, 2017) have validity periods less than or equal to 9 months, this would also see the eventual harmonization of nine month validity, but affect a different population of certificates.

Would such an approach be easier to understand and communicate?
 
Did you guys consider something where there would be an escalating level of distrust (eg, in one version you get a warning rather than a green lock, the next release a red not secure, then a intersticial)? This would allow attentive operators who might not hear about this change otherwise to figure out that there is an issue before their site breaks completely. It also gives all parties involved a small number of times where they can promote this change.

Yes, this was considered. This is similar to the approach used with SHA-1 deprecation, but in this case, may cause undesired security risk, in that such certificates continue to be used, and thus private information (such as cookies) are still transmitted, and privileged features (such as geolocation) may still be enabled. This proposal was an attempt to balance those security concerns with the interoperability and compatibility risk.

Ryan Sleevi

unread,
Mar 24, 2017, 12:23:36 AM3/24/17
to tpg...@gmail.com, blink-dev, awha...@chromium.org, Ryan Sleevi
On Thu, Mar 23, 2017 at 9:52 PM, <tpg...@gmail.com> wrote:
I would like more details on the issues. "...allowed least four parties access to their infrastructure in a way to cause certificate issuance..." is pretty vague. Did these 4 parties then issue 30,000 certificates? What's wrong with these certificates exactly?

The issues previously known are 1) "test" certificates issued without domain owner's consent, though Symantec claimed that they were only used internally; 2) certificates issued for unregistered domains.

Apologies for not including the fullness of details in the original report, and this may hopefully address any confusion as to "why".

The initial problem report was reported publicly, through Mozilla's dev.security.policy mailing list, at https://groups.google.com/d/msg/mozilla.dev.security.policy/fyJ3EK2YOP8/yvjS5leYCAAJ

In the course of understanding these issues, representatives of Mozilla and Google both addressed follow-up questions to Symantec, as did broader members of the community and peers of the Mozilla Root CA Certificate module.

Symantec's replies are (generally) available at https://bugzilla.mozilla.org/show_bug.cgi?id=1334377 and share further details.

These entities are CrossCert (Korea Electronic Certificate Authority), Certisign Certificatadora Digital, Certsuperior S. de R. L. de C.V., and Certisur S.A.. Each of these entities were authorized by Symantec to perform validation services for information within the certificate, including organizational information and domain names. This process is permitted by the Baseline Requirements, but requires both that the CA accept liability for any issues that emerge through such a relationship, and that the CA ensure these entities are appropriately audited to the equivalent criteria for the validation roles that they perform, so that all certificates issued meet a consistent level of quality.

As demonstrated through the information provided, these four entities did not follow the appropriate practices or did not possess the appropriate and necessary audits from the appropriate parties. Symantec has acknowledged they were actively aware of this for at least one party, failed to disclose this to root programs, and did not sever the relationship with this party.

In effect, each of these parties were able to effect issuance by validating information improperly. At least 30,000 certificates were issued by these parties, with no independent way to assess the compliance of these parties to the expected standards. Further, these certificates cannot be technically identified or distinguished from certificates where Symantec performed the validation role. As a consequence, the insufficient demonstration of compliance, along with the inability to distinguish such certificates, combined with the incomplete identification of the scope of the issues, create a degree of uncertainty related to the entire corpus of certificates, for which the only meaningful way to restore that confidence is to propose a gradual distrusting of the existing certificates, so that all new certificates are fully validated according to the appropriate standards.

The supporting information related to further claims can be founded within the above linked thread and the formal responses.
 

Google Internet Authority G2 CA that issues many of Google's TLS certificates is signed by GeoTrust Global CA, which presumably is owned by Symantec now. So do Google's own certificates fall into the "gradually untrustworthy" category?

As discovered during the follow-up of a previous incident - https://security.googleblog.com/2015/10/sustaining-digital-certificate-security.html - Symantec has cross-signed two organizations who operate wholly independent infrastructure that is audited as such - Apple and Google. This is documented at https://chromium.googlesource.com/chromium/src/+/master/net/data/ssl/symantec/README.md

As such, these certificates are not "Symantec-issued", for any reasonable definition, and are effectively operated by independent third parties, and thus would be proposed to be excluded from these requirements.

This evaluation of trust is not based upon an intrinsic property, but on the public disclosure and ability to independently confirm and assess the policies and practices related to issuance by these two parties.

yuhong...@hotmail.com

unread,
Mar 24, 2017, 12:29:56 AM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org, tpg...@gmail.com
Interestingly, many are 90 days valid, but not all of them (for example, Nest). 

hot73...@gmail.com

unread,
Mar 24, 2017, 12:37:05 AM3/24/17
to blink-dev
Can you elaborate on how the gradual distrust will work? If a Symantec issued cert is valid for three years, will it be immediately treated as invalid if the validity period in Chrome 59 is 33 months?

Or will the 33 month validity period only allow the certificate to be valid for 33 months (and gradually decreasing) from the date of issue effectively ignoring the expiration date as issued?

jussi....@valu.fi

unread,
Mar 24, 2017, 1:42:15 AM3/24/17
to blink-dev

Two questions about this:

1. Can you clarify what "effective immediately" means? Will this occur with the next stable release of Chrome? Or will an update be pushed to the current stable version?

As this is a security related incident, the proposal is to land this in Trunk and update Dev and Beta to reflect this, treating this as "Medium" severity per https://www.chromium.org/developers/severity-guidelines by treating this as an address bar spoofing issue.
 
2. Can you confirm that the removal of EV indicators will affect *all* Symantec-issued EV certificates, regardless of their compliance with Chrome's reduced validity schedule?

Correct. Extended Validation certificates are recognition of the processes and controls related to validating the information in the certificates is accurate and correct. The issues highlighted have undermined that confidence. The validity issue is separate and mitigates future issues that may arise or be discovered.

So, does this mean that e.g. paypal.com will lose its EV status? Doesn't this introduce whole new possibilities for fake/spoof Paypal websites as the original site doesn't have EV status anymore?

userw...@gmail.com

unread,
Mar 24, 2017, 2:29:31 AM3/24/17
to blink-dev, ag...@andrewayer.name, rsl...@chromium.org

So out of curiosity:

Assumptions:
1. Say I am Symantec or any SubCA/RA/DTP/subdivision/... operating under one of their roots.
2. I am not exempt from the measures (Apple/Google subCAs) or one of the naughty third parties (CrossCert/Certisign/Certsuperior/Certisur).
3. No additional violations of the rules by Symantec are discovered (yeah, bear with me here...)

Goal: Have trusted certs in Chrome despite the sanctions.

Possible way to achieve this:

1. non-EV:
a. Existing: To make sure my certificates remain trusted in Chrome, I provide my customers a way to reissue/convert their existing multi-year certs to some with max validity period of 9 months (and I allow them to reissue for as long as the original cert/contract would have lasted). I can do that because the existing validation info lasts for up to 39 months. That cert swap is all that is needed for customer sites to remain trusted and nobody has to worry about transition periods/Chrome updates breaking things. All correct?
b. New: I promptly start to issue new certs with <= 9 months validity. I could, however, still offer multi-year contracts because validation info for my new certs also lasts for 39 (or 27 in 2018) months just like it does for every other CA. Correct?
c. Future: The 9 months validity requirement is basically the full extent of the proposed sanctions, I won't have to create new root or sub CAs and Chrome won't decide to distrust my certs in a year or two. This is not a road to complete or partial distrust unless I fuck up again. Correct?

2. EV:
Not much I can do, as a punishment my existing and new EV certs are effectively DV certs in Chrome58+ for at least a year. I could, however, work with another EV-enabled CA to keep providing EV services. If they perform the validation, that should be allowed. Correct? What if they (cross-)sign an EV subCA of mine with valid audits and stuff? Not allowed?

Message has been deleted

pzb...@gmail.com

unread,
Mar 24, 2017, 9:32:10 AM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org


On Thursday, March 23, 2017 at 9:09:57 PM UTC-7, Ryan Sleevi wrote:
On Thu, Mar 23, 2017 at 2:07 PM, <ben.m...@gmail.com> wrote:
(just to be super clear I'm writing this in a personal capacity and not representing the thoughts of my employer)

Did you guys consider something where there would be an escalating level of distrust (eg, in one version you get a warning rather than a green lock, the next release a red not secure, then a intersticial)? This would allow attentive operators who might not hear about this change otherwise to figure out that there is an issue before their site breaks completely. It also gives all parties involved a small number of times where they can promote this change.

Yes, this was considered. This is similar to the approach used with SHA-1 deprecation, but in this case, may cause undesired security risk, in that such certificates continue to be used, and thus private information (such as cookies) are still transmitted, and privileged features (such as geolocation) may still be enabled. This proposal was an attempt to balance those security concerns with the interoperability and compatibility risk.

I realize API changes usually don't have corresponding UX, but as was called out in the initial email this is using the blink intent process for something that is not technically Blink.

To that end, I think that this this intent seems to be unclear on the resulting user experiences.  There are a variety of gradations of trust reduction available, ranging from simply treating as neutral (currently the ⓘ) to affirmatively insecure (currently red struck through https) to bypassable interstitial to non-bypassable interstitial.    Can you clarity which level of trust is proposed for certificates exceeding the defined maximum validity periods?  Is there any intent to vary this level of trust across the proposed schedule?

Thanks,
Peter

Rick Byers

unread,
Mar 24, 2017, 9:42:10 AM3/24/17
to pzb...@gmail.com, blink-dev, awha...@chromium.org, Ryan Sleevi
I'm following this thread with great interest and trying to build up the context in order to form a well-reasoned opinion.  In the interim I just wanted to clarify one point below:

On Fri, Mar 24, 2017 at 9:32 AM, <pzb...@gmail.com> wrote:

On Thursday, March 23, 2017 at 9:09:57 PM UTC-7, Ryan Sleevi wrote:
On Thu, Mar 23, 2017 at 2:07 PM, <ben.m...@gmail.com> wrote:
(just to be super clear I'm writing this in a personal capacity and not representing the thoughts of my employer)

Did you guys consider something where there would be an escalating level of distrust (eg, in one version you get a warning rather than a green lock, the next release a red not secure, then a intersticial)? This would allow attentive operators who might not hear about this change otherwise to figure out that there is an issue before their site breaks completely. It also gives all parties involved a small number of times where they can promote this change.

Yes, this was considered. This is similar to the approach used with SHA-1 deprecation, but in this case, may cause undesired security risk, in that such certificates continue to be used, and thus private information (such as cookies) are still transmitted, and privileged features (such as geolocation) may still be enabled. This proposal was an attempt to balance those security concerns with the interoperability and compatibility risk.

I realize API changes usually don't have corresponding UX, but as was called out in the initial email this is using the blink intent process for something that is not technically Blink.

Note that for quite some time now we've been using this web platform change process for most chromium changes things that impact web developers, regardless of where the code happens to be located (the division between blink and content is pretty artificial these days).   Here's a few recent examples which I believe are not (or at least primarily not) blink:


That said, I think we're all trying to figure out if this web platform change process will provide value to the web community for tricky PKI-decisions like this.  But at a minimum, the extra transparency and community involvement seems clearly valuable.

lei...@gmail.com

unread,
Mar 24, 2017, 10:17:58 AM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
Are there any plans to put the "View certificate" button back where it belongs: "Click on the padlock, click on 'view certificate'"?

Half of the times F12 invokes a debugger, and it is counter-intuitive.not just following the padlock.

crypt...@gmail.com

unread,
Mar 24, 2017, 11:08:55 AM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org
Disgusting abuse of your power!  You are punishing 30,000 websites, 99.9%+ of whom are completely legitimate, in order to exact revenge against Symantec.

LEAVE THE INNOCENT BYSTANDERS ALONE!!!!

I propose you block all *new* Symantec certificates until they go back and re-validate (AT THEIR EXPENSE) all the 30,000 websites, and revoke any that are found incorrect.

Be responsible with the power you have, and mindful of the massive collateral damage your actions cause!

You've already just destroyed wosign and startssl wreaking havoc across their entire user base: ***WE*** SUFFER when *you* attack CAs... so STOP IT!!!!

fabri...@gmail.com

unread,
Mar 24, 2017, 12:10:02 PM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org, crypt...@gmail.com


yes, I agree with this, doesn't sound much fair what your proposed solution. Are you aware this move will affect thousands of e-commerce sites and related businesses??! Please, ponder your decision carefully and try to find the best compromise for everyone here.

tbar...@encirca.com

unread,
Mar 24, 2017, 12:23:08 PM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org, crypt...@gmail.com, fabri...@gmail.com
Actually,  this impacts ALL existing EV certs from Symantec.  Not just the 30,000 that were improperly issued.  right?

ad...@adamcaudill.com

unread,
Mar 24, 2017, 12:35:35 PM3/24/17
to blink-dev, awha...@chromium.org, rsl...@chromium.org, crypt...@gmail.com
(Standard disclaimer: speaking on my own behalf, not that of my employer, etc.)

> Disgusting abuse of your power!  You are punishing 30,000 websites [...snip...]

Browsers have a substantial responsibility to protect their users, limiting validity and otherwise placing restrictions on a CA is an important tool to minimizing the harm that a CA can do. Browsers, on behalf of users, make decisions about which CAs can be trusted and which ones can't - strict rules and requirements are placed on CAs to ensure that they can be trusted, if these rules are violated, that trust relationship is damaged. If a browser loses faith in a CA, and the controls they have in place to ensure that these requirements are met, they have a responsibility to their users to take action.

If there are signs that controls are defective, if there are signs that a significant number of certificates were issued in violation of these rules, if there is reason to believe that the CA didn't discover issues when they should have, or ignored issues they did discover - there is good reason to lose faith in that CA and all of the certificates they've issued.

> You've already just destroyed wosign and startssl [...snip...]

WoSign/StartSSL showed blatant disregard for the requirements placed on CAs, and in doing so exposed not just their customers to additional risk, but all users of browsers that trusted them. The browsers (not just Chrome) took action to protect their users from such reckless action.

Removing trust, or restricting trust, in a CA is a serious action, with impact to many; for those that operate TLS servers, that impact is often additional work and financial costs, for users (the group browsers are primarily responsible to), the impact is a reduction in risk. CAs that fail their customers by not living up to their obligations should bear the burden of the problems they've introduced, not the browsers that call them out on it. In the case of WoSign, they knowingly performed actions that violated their obligations - they created the situation, they are responsible for failing their customers.

If you want to be mad at someone, be mad at CAs that violate the requirements that keep users safe.
 

guillaume...@gmail.com

unread,
Mar 24, 2017, 12:48:07 PM3/24/17
to blink-dev
If I understand correctly all existing Symantec-issued certificates with a 2 years validity period will be untrusted with September 12th Chrome release, whatever their Not Before date is. Am I right?

I yes, will that generate a big red untrusted certificate error, or will it just not show a green padlock ?

cc1265...@gmail.com

unread,
Mar 24, 2017, 12:48:24 PM3/24/17