Continued Operation of Logs with Planned Turn Down Dates

[Bailey Basile]

All,

In iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13, Apple introduced a new requirement for Extended Validation (EV) certificates to be Certificate Transparency (CT) qualified. If an otherwise valid EV certificate did not have Signed Certificate Timestamps (SCTs) from a qualified CT log, that certificate could pass a trust evaluation but would not have an EV indicator in the returned trust evaluation information. A number of critical Apple services on devices running these software versions require the EV indicator for the TLS server certificate to successfully connect to those services.

The qualified CT log public keys were built into the system image of all devices and could be updated only by a Software Update. Beginning in iOS 11.3, watchOS 4.3, tvOS 11.3, and macOS 10.13.4, the set of qualified CT logs were made updatable out-of-band of a Software Update.

Users still running the software versions prior to this change — users who have chosen not to update — have a static log list with no mechanism to disable the CT checks for EV certificates. All of these users have software update alerts (e.g. badged Settings apps) advising them to update to newer releases for security and other improvements — releases which are capable of updating the CT log list for the EV checks.

All of the remaining operational logs qualified on iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13 had been planned to reject new entries later this year. Those logs and corresponding proposed shutdown dates are:

• Google's Pilot Log (ct.googleapis.com/pilot/) — 1 May 2019
• Google’s Rocketeer Log (ct.googleapis.com/rocketeer/) —1 June 2019
• Google’s Skydiver Log (ct.googleapis.com/skydiver/) —1 July 2019
• Google’s Icarus Log (ct.googleapis.com/icarus/) —1 August 2019
• DigiCert Log Server (ct1.digicert-ct.com/log/) — 6 May 2019

We have asked Google and DigiCert to continue operation of these logs beyond their planned shutdown. Doing so will permit Apple to support all of our users on iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13 for the foreseeable future.

We at Apple strongly believe in the power of Certificate Transparency to improve the security of the Internet for our users and intend to continue to be active participants in this ecosystem. We hope the community is sympathetic to these users’ choice not to update and therefore their need for continued operation of these logs.

[Peter Bowen]

On Tuesday, April 9, 2019 at 10:04:52 AM UTC-7, Bailey Basile wrote:

In iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13, Apple introduced a new requirement for Extended Validation (EV) certificates to be Certificate Transparency (CT) qualified. If an otherwise valid EV certificate did not have Signed Certificate Timestamps (SCTs) from a qualified CT log, that certificate could pass a trust evaluation but would not have an EV indicator in the returned trust evaluation information. A number of critical Apple services on devices running these software versions require the EV indicator for the TLS server certificate to successfully connect to those services.

The qualified CT log public keys were built into the system image of all devices and could be updated only by a Software Update. Beginning in iOS 11.3, watchOS 4.3, tvOS 11.3, and macOS 10.13.4, the set of qualified CT logs were made updatable out-of-band of a Software Update.

Users still running the software versions prior to this change — users who have chosen not to update — have a static log list with no mechanism to disable the CT checks for EV certificates. All of these users have software update alerts (e.g. badged Settings apps) advising them to update to newer releases for security and other improvements — releases which are capable of updating the CT log list for the EV checks.

All of the remaining operational logs qualified on iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13 had been planned to reject new entries later this year. Those logs and corresponding proposed shutdown dates are:

• Google's Pilot Log (ct.googleapis.com/pilot/) — 1 May 2019
• Google’s Rocketeer Log (ct.googleapis.com/rocketeer/) —1 June 2019
• Google’s Skydiver Log (ct.googleapis.com/skydiver/) —1 July 2019
• Google’s Icarus Log (ct.googleapis.com/icarus/) —1 August 2019
• DigiCert Log Server (ct1.digicert-ct.com/log/) — 6 May 2019

What are the specific SCT requirements for the OSes listed?  How can SCTs be delivered, are there diversity requirements, and how many SCTs are required per certificate?

[Paul Adare]

https://support.apple.com/en-ca/HT205280  

--
You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.
To post to this group, send email to ct-p...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/89718af8-08b9-42c6-a7fb-347892729286%40chromium.org.

--

Paul Adare

[Bailey Basile]

The affected OS versions enforce the same requirements for the EV indicator as listed in our policy documentation (https://support.apple.com/en-us/HT205280). However, these software versions were also affected by a bug in the TLS stack that prevents them from using of SCTs delivered via the TLS extension; they do support embedded SCTs and SCTs delivered via stapled OCSP responses.

[Andrew Ayer]

I'm supportive of finding a way out of this predicament, but I think it's
important to find a solution that doesn't require other participants
in the ecosystem to pay excessive costs for mistakes they didn't make.

Already it's expensive to monitor several of these logs since they're
huge. Furthermore, they are disproportionately expensive compared
to their value, since they consist mostly of expired certificates:

Rocketeer: 649 million certificates total, 78% expired
Pilot: 581 million certificates total, 76% expired
Icarus: 558 million certificates, 84% expired
Skydiver: 159 million certificates, 35% expired
DigiCert 1: 13 million certificates, 60% expired

(Source: Merkle Town, 2019-04-10)

This is why it's important for the sustainability of the ecosystem to
phase out non-sharded logs. If these logs continue operating, their
size will grow without bound, as they simultaneously get less and less
valuable to the ecosystem. Monitors and log operators will continue paying
increasing costs, despite not being responsible for this predicament.

There is also a risk of creating a moral hazard - if an ecosystem
participant's mistake causes others to pay but not them, what's the
incentive to be careful?

I believe we can do better.

I propose that on the planned shut down dates, these logs switch to
accepting only the certificates necessary for Apple's EV-only services.
The easiest way to accomplish this within the framework of RFC6962 is for
the certificates to be issued from a dedicated intermediate CA which is
the only trust anchor accepted by the logs. This will significantly slow
the growth of these logs. More importantly, it will allow UAs/monitors
to maintain the current schedule of rejecting/removing these logs after
825 days, when all TLS server certificates in the logs (besides Apple's)
have expired.

After the logs are rejected, Apple will need to send sufficient SCTs
from both the set of legacy logs and the set of still-qualified logs for
their certificates to work in both legacy Apple OSes and up-to-date UAs.

Although this plan requires more work from Apple than simply continuing
to use these logs without any change, it causes minimal impact to
other ecosystem participants:

UAs: no impact. They can still reject these logs 825 days after their
planned shutdown dates.

Monitors: minimal impact. They can still stop monitoring these logs
after UAs reject them 825 days after their planned shutdown dates.
The logs will grow in the meantime, but only slightly since the logs
only accept Apple's critical EV certificates.

CAs/site operators: no impact. CAs/site operators already needed to stop
submitting to these logs prior to their shutdown dates. Since the logs
will only accept Apple certificates, there is no risk of a CA/site
operator mistakenly submitting to the log and getting an SCT that will
go bad in the future.

Log operators unfortunately have to continue operating these logs.
However, the growth of the logs will be slow, which in addition to
limiting costs, reduces the likelihood of log failure (e.g. due to
excessive submission rates). That said, I don't think any log operator
should be obligated to continue operating these logs once they're
rejected by up-to-date UAs.

Regards,
Andrew

[Mads Henriksveen]

Representing a CA using the Google CT-logs, I was surprised when the topic of shutting down Google CT-logs came up in this thread.

 

I got some additional information from Devon O'Brian (thank you Devon!) and understands that a shutdown notification for these logs has not been posted to this list. This has surely been communicated somewhere, but I have missed it - and I assume this might be the same for other CAs.

 

I am concerned about how CAs using CT-logs should be made aware of such planned shutdowns of logs. For some other CT-logs being shut down in the past, we have got a shutdown notification directly from the CT-log operator, but not in this case. 

 

Is there some agreed on method for how CT-log operators announces that they plan to shut down their CT-logs?

 

Regards

Mads 

[Kat Joyce]

Hi all,

We initially posted our Log turn down dates to the Xenon Chrome bug (https://crbug.com/833350#c6), as wanting Xenon included in Chrome was part of the motivation for turning down some of our older non-time-sharded Logs.  However, we should have also posted these plans here, to ct-policy, and possibly also on the Chrome bugs for the Logs that we were actually planning to shut down, and we did not, which was our mistake.  I'd say at the very least, Log shutdown plans should be posted on this mailing list.

However, given the situation with Apple, it is likely we will no longer be turning down these Logs on the dates specified in that bug comment.  We are currently working out exactly what our plans for these Logs are now, and we will post these plans to this list as soon as we have agreed upon them.

Apologies again for the confusion caused by us not posting more widely about the initial turn down plans.  I agree that it would be good to have an accepted method of informing the community of upcoming plans to shut down Logs, perhaps that is something we can discuss?  Is posting to ct-policy enough?

Kat

--

You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.
To post to this group, send email to ct-p...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/0311425f-7a9a-407a-bd43-606d824bae47%40chromium.org.

[Doug Beattie (Globalsign)]

Hi Kat,

Is there an update on if/when Google will be turning down some of their logs?

To unsubscribe from this group and stop receiving emails from it, send an email to ct-p...@chromium.org.

[Doug Beattie]

Hi Bailey,

I'd like to be sure I understand the timeline and process for Apple OSs getting new CT log lists.  If I got any of the below assumptions incorrect, please let me know.

Prior to the following releases, Apple would trust SSL certificates but not display the EV Green bar if it didn't receive a sufficient number of SCTs:

• iOS 11.3, watchOS 4.3, tvOS 11.3, and macOS 10.13.4

For Apple versions after this:

• the SSL certificate will not be trusted unless a sufficient number of SCTs are presented
• Apple will update the CT log list out-of-band of a Software Update.

In another email you recently announced the addition of new CT logs.  Can you provide some guidance on when we can start using these new CT logs and be confident that all of the Apple OSs listed above will have received the updated set of CT logs?

Doug

--

You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.

To post to this group, send email to ct-p...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/f8d5e80c-1597-4958-b575-a1b72ae4857e%40chromium.org.

[Kat Joyce]

Hi Doug,

Thanks for chasing us up on this, I hadn't realised quite how much time had passed since my previous message!  We're not quite ready for an official announcement, but I'm happy to offer an update on what we're thinking :)

We are leaning towards switching our non-temporally-sharded Logs to only accept certificates chaining to the roots Apple requires.  However, before finalizing and announcing any concrete plans, details and change over dates we are waiting on confirmation from Chrome that doing this is okay with them, and will not affect the Logs' qualified status (which I believe they will comment on in this thread?)  Once we have heard from them giving us the go-ahead (assuming that is the decision they make!) we will post our exact plans and dates on ct-policy, and on the relevant Chromium bugs.  Until you hear otherwise, all of our our Logs will remain up and running, with the set of accepted roots remaining in line with those in the major trust stores.

That all being said, we still recommend CAs switch over to use our temporally-sharded Logs, Argon and Xenon.  Using temporally sharded Logs has a whole host of benefits, including predictable Log turn down times, manageable Log sizes etc, and these Logs will not be affected by any of the previously mentioned potential root-change/turn-down plans.

Thanks again for chasing this, and please do continue to do so  if it appears that progress isn't being made!

Kat

To unsubscribe from this group and stop receiving emails from it, send an email to ct-policy+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/a0830de7-ef04-4ff1-b014-d20627ca3a88%40chromium.org.

[Bailey Basile]

Doug,

When we post a log list indicating that logs are “usable”, all devices that enforce CT for successful TLS Server connections have those logs. 

When the logs transition from “pending” to “qualified”, we are deploying those newly qualified logs and will transition the log to the “usable” state after successful deployment. This deployment process takes upwards of 74 days, including a 60-day period after which the software will stop enforcing CT if it has not successfully checked in with Apple to fetch an updated log list.

These states are documented here: https://support.apple.com/en-us/HT209255.

We began enforcing CT for TLS Server certificates in iOS 12.1.1, watchOS 5.1.2, tvOS 12.1.1, and macOS 10.14.2.

iOS 11.0, watchOS 4.0, tvOS 11.0, and macOS 10.13.0 began using CT for the “EV Green bar” (but not for successful TLS connections); however, the log list was not updatable out of band of a software update until iOS 11.3, watchOS 4.3, tvOS 11.3, and macOS 10.13.4.

Bailey

To unsubscribe from this group and stop receiving emails from it, send an email to ct-p...@chromium.org.

[Julie Olson]

Hi Bailey,

I was going through the list of Apple logs (https://valid.apple.com/ct/log_list/current_log_list.json) and noticed a few older ones that are still marked as "usable":

• "Google 'Argon 2017' log"
• "Google 'Argon 2018' log" 
• "Google 'Xenon 2018' log"
• "DigiCert 'Yeti 2018' log"

What is the timeline to updating this JSON list against changes made in Chrome? Or are these logs still technically usable?

Thanks,

Julie.

[Apple Certification Authority]

Earlier in this thread, Andrew Ayer suggested a strategy [1] to minimize the impact to the CT ecosystem while supporting Apple’s need to use legacy CT Logs. Subsequently, on July 12, Google explained their approach [2] to manage their CT Logs in relationship to Apple’s legacy clients.

Both proposals relied on limiting the trust anchors from which the CT logs would accept certificates. In order to facilitate moving forward this way, Apple is limiting EV SSL certificate issuance to three roots [3] from its current vendor, DigiCert, for the foreseeable future. Furthermore, in order to facilitate a more flexible configuration that allows SCTs from legacy and modern CT logs, new Intermediate CAs [3] were issued end of last year and are the only active issuers for Apple EV SSL certificates at this time. 

Apple will continue working closely with Google and DigiCert to minimize impact to the overall community. We appreciate all the comments, suggestions and discussion.

[1] https://groups.google.com/a/chromium.org/d/msg/ct-policy/i1NFmE7txNE/wwrhMtvtBQAJ

[2] https://groups.google.com/a/chromium.org/forum/#!topic/ct-policy/cizrYW7dxac

[3] New subordinate CAs
Issuing Root: DigiCert High Assurance EV Root CA
Intermediate CA: https://crt.sh/?id=2116242360

Issuing Root: DigiCert Global Root G2
Intermediate CA: https://crt.sh/?id=2116244402

Issuing Root: DigiCert Global Root G3
Intermediate CA: https://crt.sh/?id=2116242353

[Andrew Ayer]

This is great! Apple - thanks for taking this step to minimize impact
to the CT ecosystem.

Log operators - when do you anticipate limiting the accepted trust
anchors of the legacy logs to this set of intermediate CAs?
Rocketeer is nearing a billion entries, and over the last week, someone
has been submitting a bunch of long-expired certificates to Rocketeer,
making this log even more of a drag on the ecosystem. It would be
great to slow down the growth of these huge legacy logs before
they get even bigger.

Regards,
Andrew

On Tue, 21 Jan 2020 17:08:22 -0800 (PST)
"'Apple Certification Authority' via Certificate Transparency Policy"

> --
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group. To unsubscribe from
> this group and stop receiving emails from it, send an email to

> ct-policy+...@chromium.org. To view this discussion on the
> web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/3305bf06-1915-488e-bf94-52ae23cac1bd%40chromium.org.

[Kat Joyce]

Hi Andrew,

A quick note to acknowledge this and let you know it's on our radar.  We are currently clarifying a few details with Apple, then we will be posting a planned timeline for changes to our non-temporally sharded Logs.

Thanks for everyone's patience with this!

Kat

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/ct-policy/20200128161938.dd8fdfe0670dd512b1b52248%40andrewayer.name.

[Kat Joyce]

We have just posted an announcement with our plans for our legacy Logs.  Please let us know on that thread if you have any thoughts or feedback!

Thanks,

Kat

[Clint]

I couldn't get to it through that link, so if anyone else encountered that the announcement can be found here: https://groups.google.com/a/chromium.org/d/topic/ct-policy/iOg8Jqc0XxU/discussion

Cheers!

-Clint

> ct-p...@chromium.org. To view this discussion on the

> web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/3305bf06-1915-488e-bf94-52ae23cac1bd%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Certificate Transparency Policy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ct-p...@chromium.org.