Dynamically-updatable CT Log Lists in Chrome & Changes to CT Enforcement Window
Devon O'Brien
Hello ct-policy@,
We’re announcing some changes to how Chrome clients obtain updated CT Log lists, the schema we use to express these lists, and how the CT enforcement window in Chrome is calculated. Most of these changes were incrementally rolled out in Chrome 94 to detect any early issues; however, most up-to-date Chrome clients should now have seamlessly transitioned to the new mechanism. We expect these changes to be generally uncontroversial; however, we always appreciate discussion and feedback from the CT community about changes such as these.
How did Chrome originally obtain and use CT Log lists?Google publishes the current CT Log list using the v2 JSON schema at https://www.gstatic.com/ct/log_list/v2/log_list.json. Changes to this log list were propagated to the CT Log list in Chromium source, which has long been deemed the authoritative source of truth for this information.
When a release of Chrome was built, the log_list.json in Chromium source was translated into C++ to be used when evaluating (certificate, SCTs)-tuples for CT Compliance. This release of Chrome would then enforce CT for certificates issued from default-trusted CAs for 10 weeks (70 days) after its build date. This 10 week cutoff for CT enforcement allows the ecosystem to reliably use new CT logs without having to wait for extremely out-of-date clients to update, and it matches the behavior of several other security features in Chrome such as key pinning and HSTS.
Changes to how Chrome obtains CT Log listsIn order to more rapidly propagate additions and removals to Chrome clients as well as provide log list updates to very out-of-date installs, since version 94, Chrome has moved to a rapid distribution and update model that takes advantage of our Component Updater infrastructure.
Under this new model:
Changes to the CT Log List are first made in a file hosted on internal Google servers, and are automatically propagated to log_list.json and all_logs_list.json, which are publicly-accessible at https://www.gstatic.com/ct/log_list/v3/log_list.json and https://www.gstatic.com/ct/log_list/v3/all_logs_list.json respectively.
Internally, these changes are also propagated to the servers that handle Google’s CT Log mirrors, SCT Auditing, and CT Log compliance monitoring as well as the Component Updater infrastructure.
Several times a day, Chrome will query the Component Updater servers, looking for an updated CT Log List. When a newer version of the list than Chrome’s current version is available, a differential update will be produced and sent to the client over a secure channel.
Changes to log_list_schema.json
In addition to changing how Chrome obtains updated CT Log lists, we are updating the JSON schema of the list itself to accommodate upcoming changes in Chrome’s CT enforcement behavior. The updated schema can be found at https://www.gstatic.com/ct/log_list/v3/log_list_schema.json. To allow for tools and processes to be updated to the new locations and schemas, Google will continue to publish and update the existing log lists at the current /v2/ location until we are confident the transition has passed seamlessly.
The new v3 schema will add the following fields:
log_list_timestamp
Every time a change is made to log_list.json, the log_list_timestamp field will be updated to the current time. Starting in version 94, Chrome uses this timestamp to determine the CT enforcement window. Even if no change is made, log_list.json will be updated automatically every 24 hours, incrementing the minor version by one and updating this timestamp. Additional information on the use of this timestamp can be found in the sections below.
version
The version field is already specified in the v2 schema; however, Chrome’s v3 schema makes this field required and will still represent the version as a string of the form “major.minor”, where both major and minor are non-negative numerical values.
Whenever a substantive change to log_list.json is made (adding a CT Log, removing a CT Log, or an existing CT Log changing state), the major version will increment by one, the minor version will be reset to 0, and as with all changes to the list, the log_list_timestamp will be updated to the current time.
Additionally, when a new major version is released, it will be archived in its own subdirectory labeled with the major version. For example, when version “5.0” is released, it will be made available at https://www.gstatic.com/ct/log_list/v3/5/log_list.json, even after the non-archived version is updated.
Once every 24 hours, log_list.json is updated automatically, incrementing the minor version by one and updating the timestamp. Previous minor versions are not archived since they only differ by the log_list_timestamp, but contain identical lists of CT Logs.
previous_operators
In rare circumstances, a CT Log will change operators from one organization to another. In order to track which operator ran a given CT Log at any point in the log’s history, we are adding an optional previous_operators field to each log within the list.
The previous_operators field is an array that contains a list of the names of any previous log operator(s) accompanied by the final timestamp that the log was run by that operator. The previous_operators field will only be present for logs that have had at least one operator change during its lifetime.
This field will play a major role in an upcoming announcement regarding changes to the requirement that certificates are logged to a Google log. Stay tuned for more on this!
Changes to how Chrome determines its CT enforcement windowWhen evaluating a certificate for CT Compliance, Chrome 94 and up first checks to see if it has a CT Log list with a recent timestamp (within the past 70 days), and will enforce the CT Policy checks; otherwise, it will not enforce CT.
Each new release of Chrome will include an updated CT Log list with a fresh timestamp to base the initial CT enforcement window off of. Several times a day, Chrome contacts the Component Updater servers, looking for an updated log list. If one is available, the server will produce a differential update and deliver it to Chrome over a secure channel. When the list is successfully updated, Chrome will use the new timestamp to determine the new start of the CT enforcement window.
Out-of-date Chrome clients that are unable to dynamically obtain an updated CT Log list within the 70 day CT enforcement window will disable CT enforcement, as is already done today. As a result of this change, we expect to see an increase in the number of Chrome clients enforcing CT, as very out-of-date Chrome installs will now have a way to safely extend or re-enable CT enforcement without updating.
Andrew Ayer
As a monitor operator, I appreciate hearing these details about the
auto-propagation, and having log_list_timestamp in log_list.json, as it
provides assurance that I'm monitoring an up-to-date set of logs.
Thanks!
I noticed that v3/all_logs_list.json has two entries for "Trust Asia
CT2021" (log ID qNxS9j1rJCXlMeN89ORKcU8UKiCAOw0E0uLuBmR5SiM=), one
Pending and one Usable.
I look forward to the upcoming announcement!
Regards,
Andrew
On Mon, 1 Nov 2021 10:17:26 -0700 (PDT)
"'Devon O'Brien' via Certificate Transparency Policy"
>
>
> Hello ct-policy@,
>
> We’re announcing some changes to how Chrome clients obtain updated CT
> Log lists, the schema we use to express these lists, and how the CT
> enforcement window in Chrome is calculated. Most of these changes
> were incrementally rolled out in Chrome 94 to detect any early
> issues; however, most up-to-date Chrome clients should now have
> seamlessly transitioned to the new mechanism. We expect these changes
> to be generally uncontroversial; however, we always appreciate
> discussion and feedback from the CT community about changes such as
> Google publishes the current CT Log list using the v2 JSON schema
> log list were propagated to the CT Log list in Chromium source
> information.
>
> When a release of Chrome was built, the log_list.json in Chromium
> source was translated into C++ to be used when evaluating
> (certificate, SCTs)-tuples for CT Compliance. This release of Chrome
> would then enforce CT for certificates issued from default-trusted
> CAs for 10 weeks (70 days) after its build date. This 10 week cutoff
> for CT enforcement allows the ecosystem to reliably use new CT logs
> without having to wait for extremely out-of-date clients to update,
> and it matches the behavior of several other security features in
> CT Log lists*
> In order to more rapidly propagate additions and removals to Chrome
> clients as well as provide log list updates to very out-of-date
> installs, since version 94, Chrome has moved to a rapid distribution
> and update model that takes advantage of our Component Updater
> [image: CT Log List Publishing Diagram - External.jpg]
>
> Under this new model:
>
> Changes to the CT Log List are first made in a file hosted on
> internal Google servers, and are automatically propagated to
> log_list.json and all_logs_list.json, which are publicly-accessible
> at https://www.gstatic.com/ct/log_list/v3/log_list.json and
> https://www.gstatic.com/ct/log_list/v3/all_logs_list.json
> respectively.
> Internally, these changes are also propagated to the servers that
> handle Google’s CT Log mirrors, SCT Auditing, and CT Log compliance
> monitoring as well as the Component Updater infrastructure.
> Several times a day, Chrome will query the Component Updater
> servers, looking for an updated CT Log List. When a newer version of
> the list than Chrome’s current version is available, a differential
> update will be produced and sent to the client over a secure channel.
>
>
> When evaluating a certificate for CT Compliance, Chrome 94 and up
> first checks to see if it has a CT Log list with a recent timestamp
> (within the past 70 days), and will enforce the CT Policy checks;
> otherwise, it will not enforce CT.
>
> Each new release of Chrome will include an updated CT Log list with a
> fresh timestamp to base the initial CT enforcement window off of.
> Several times a day, Chrome contacts the Component Updater
> server will produce a differential update and deliver it to Chrome
> over a secure channel. When the list is successfully updated, Chrome
> will use the new timestamp to determine the new start of the CT
> enforcement window.
>
> Out-of-date Chrome clients that are unable to dynamically obtain an
> updated CT Log list within the 70 day CT enforcement window will
> disable CT enforcement, as is already done today. As a result of this
> change, we expect to see an increase in the number of Chrome clients
> enforcing CT, as very out-of-date Chrome installs will now have a way
> to safely extend or re-enable CT enforcement without updating.
>
> You received this message because you are subscribed to the Google
> Groups "Certificate Transparency Policy" group. To unsubscribe from
> this group and stop receiving emails from it, send an email to
> ct-policy+...@chromium.org. To view this discussion on the
> web visit
> https://groups.google.com/a/chromium.org/d/msgid/ct-policy/f2958124-f679-406d-8bca-a063b7d37c3an%40chromium.org.
Devon O'Brien
Andrew Ayer
I'm seeing a new issue with
https://www.gstatic.com/ct/log_list/v3/all_logs_list.json - the log ID
for "DigiCert Yeti2022-2 Log" is malformed:
"log_id": "BZwB0yDgB4QTlYBJjRF8kDJmr69yULWvO0akPhGEDUo=s",
(Note the trailing s at the end)
Regards,
Andrew
On Thu, 4 Nov 2021 09:25:34 -0700 (PDT)