All,
This matter involves a replacement of the GlobalSign Root CA - R2 (the “R2 Root”). The R2 Root is currently owned by Google Trust Services (“GTS”). The R2 Root was originally signed in 2006 using SHA1WithRSA and has a validity period of 12/15/2006 to 12/15/2021. Information about the original CA certificate for the R2 Root can be obtained here: https://crt.sh/?id=14. The R2 Root still supports a number of legacy clients.
Of note, this original root CA certificate had key usage bits for Certificate Signing and CRL Signing (hexadecimal 06), but not the digitalSignature key usage bit. For OCSP signing, a CA certificate needs to have the digitalSignature key usage bit, along with the other two key usage bits (hexadecimal 86). (Section 7.1.2.1.b of the Baseline Requirements states, “If the Root CA Private Key is used for signing OCSP responses, then the digitalSignature bit MUST be set.”)
In order to comply, last year, GTS re-signed the R2 Root with the digitalSignature key usage bit. See https://crt.sh/?id=3448659678 It did the same with five other root CA certificates. The only changes made were for that one keyUsage bit and the certificate serial number.
We are currently processing an inclusion request for these GTS replacement CA certificates (https://ccadb-public.secure.force.com/mozilla/PrintViewForCase?CaseNumber=00000666). However, of note, in re-signing the R2 root, GTS used the SHA1WithRSA algorithm because that is the signature algorithm that was used for the original CA certificate. The other new CA certificates were re-signed with their respective signature algorithms (which happened not to be SHA1WithRSA). Whether this re-signing is a violation of section 5.1.3 of the Mozilla Root Store Policy (MRSP) has been alluded to. https://bugzilla.mozilla.org/show_bug.cgi?id=1652581#c22. As most of you know, policies applicable to signing with SHA1 have changed overtime.
GTS would like a decision that section 5.1.3. allows this type of root CA certificate modification because if it is not allowed, then there is no other practical solution to address new requirements for older roots that support legacy clients that might have trouble with SHA2.
Here is the relevant language from MRSP section 5.1.3:
CAs MAY sign SHA-1 hashes over intermediate certificates which chain up to roots in Mozilla's program only if the certificate to be signed is a duplicate of an existing SHA-1 intermediate certificate with the only changes being all of:
· a new key (of the same size);
· a new serial number (of the same length);
· the addition of an EKU and/or a pathlen constraint to meet the requirements outlined above. …
CAs MUST NOT sign SHA-1 hashes over other data, including CT pre-certificates.
Note: Section 7.1.3 of the Baseline Requirements previously stated, “This Section 7.1.3 does not apply to Root CA or CA cross certificates. CAs MAY continue to use their existing SHA-1 Root Certificates.” Section 7.1.3.2.1 with similar language to MRSP section 5.1.3 above was added last year after GTS had re-signed the R2 Root, but at the time it complied with the Baseline Requirements.
While MRSP section 5.1.3 addresses the use of SHA1 with end entity certificates and intermediate CA certificates, it does not expressly address certificate modifications for roots. (The re-signed root also happens to be an intermediate CA certificate under the MRSP because it “chains up to a root in Mozilla’s program”. However, the third bullet of the exception above is only for an EKU or pathlen constraint and not for a change in the key usage bit.) It has been raised that the phrase “CAs MUST NOT sign SHA-1 hashes over other data” precludes the use of SHA1 when signing anything beyond what is allowed by section 5.1.3, including the modification of a root certificate. This is an issue of interpreting the meaning of "other data" in the context of MRSP section 5.1.3.
Still, with the foregoing in mind, I am currently inclined to continue processing the GTS inclusion request with this SHA1-signed certificate for the R2 Root because (1) it would only replace the existing SHA1-signed certificate; (2) GTS was attempting to resolve an issue with one of the requirements using a “minimally invasive” approach (i.e. only changing the key usage bit and the serial number); (3) the new CA certificate is still valid only until December 15, 2021; (4) the public key has not changed and browser root stores rely mainly on the public key and not the signature algorithm, which is one of the factors for why SHA1 roots have been allowed to remain in root stores; and (5) applying the MRSP in this manner would be consistent with the then-existing Baseline Requirements that did not apply to roots.
What are your thoughts?
Thanks in advance,
Ben
--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/20210411041352.63b3826a%40totoro.tlrmx.org.
All, we appreciate the comments and want to address some issues raised thus far.
Before taking this action, GTS consulted with root program contacts for all the root programs we participate in, notifying them of our intent to modify the affected roots in this way. In these communications, we stressed the fact that GS R2 is very tenured and important to keep active for the many devices that do not update their roots, as there were questions about why we were not simply retiring the root. We, perhaps mistakenly, believed the implications of modifying a SHA1 root were clear, but perhaps they were not. Only after the fact were these issues raised. After extensive discussion, Mozilla and GTS agreed that getting community input made sense, thus Ben's post.
We are glad to see that at least some community members see a need for policy modification. There is no such thing as an exception in this space, but we're hoping that there is agreement that it makes sense to modify existing policy to allow this narrow use case. In retrospect, it seems clear that wider community input would have been beneficial here, but we want to be clear that GTS did not take unilateral action and had consulted a number of key community stakeholders before conducting the root modification ceremony.
Finally, it looks like there may be a bit of confusion about the extent to which SHA1 signing is being discussed here. GTS does not issue SHA1 leaf certificates and never will. A single modification event for a tenured root was the only use of SHA1. Sadly, there are many embedded devices that do not update their root stores and will eventually break when the last of the roots they support hit their end of life. We cannot avoid these devices eventually breaking, but keeping tenured roots valid until their natural expiration delays older devices breaking.
The issues covered here are detailed in the existing incident report (https://bugzilla.mozilla.org/show_bug.cgi?id=1652581) and the replacement tracking bug (https://bugzilla.mozilla.org/show_bug.cgi?id=1675821) if the community feels another report focused specifically on the modification is warranted, we will happily provide one.
All, we appreciate the comments and want to address some issues raised thus far.
Before taking this action, GTS consulted with root program contacts for all the root programs we participate in, notifying them of our intent to modify the affected roots in this way.
In these communications, we stressed the fact that GS R2 is very tenured and important to keep active for the many devices that do not update their roots, as there were questions about why we were not simply retiring the root. We, perhaps mistakenly, believed the implications of modifying a SHA1 root were clear, but perhaps they were not.
Finally, it looks like there may be a bit of confusion about the extent to which SHA1 signing is being discussed here. GTS does not issue SHA1 leaf certificates and never will. A single modification event for a tenured root was the only use of SHA1. Sadly, there are many embedded devices that do not update their root stores and will eventually break when the last of the roots they support hit their end of life. We cannot avoid these devices eventually breaking, but keeping tenured roots valid until their natural expiration delays older devices breaking.
The issues covered here are detailed in the existing incident report (https://bugzilla.mozilla.org/show_bug.cgi?id=1652581) and the replacement tracking bug (https://bugzilla.mozilla.org/show_bug.cgi?id=1675821) if the community feels another report focused specifically on the modification is warranted, we will happily provide one.
Let me start with an apology for the delayed response to this thread. I took some time off in April to decompress from the constant work that we have all experienced due to Covid-19, and since back, I have been trying to address a set of urgent matters that came up while I was out including this one.
Given the tone of the thread, I had asked the team to let me respond but overestimated my ability to get to it with the urgency the thread deserved. The fact that I am a critical failure point in the communication process is a problem in itself, and something I will talk more about below.
This is not an excuse, and I want to make it clear that we understand we need to do better in communicating with this community and you have our commitment that we will.
Since there are several topics in play right now, my thinking is that it is best to move the discussions of each into the respective Bugzilla issues. This should enable information related to each to be more easily tracked. For others, not already following in Bugzilla, those issues are:
- digitalSignature KeyUsage not set (https://bugzilla.mozilla.org/show_bug.cgi?id=1652581)
- Failure to provide regular and timely incident updates (https://bugzilla.mozilla.org/show_bug.cgi?id=1708516)
- Forbidden Domain Validation Method 3.2.2.4.10 ( https://bugzilla.mozilla.org/show_bug.cgi?id=1706967)
- Signing SHA-1 Hash for existing CA certificate with changes in Key Usage (https://bugzilla.mozilla.org/show_bug.cgi?id=1709223)
You will notice I have created a third incident to cover the SHA-1 usage in the re-issuance of ‘GlobalSign Root CA - R2’, which was the thread's original topic. I have tried to include the complete picture of what transpired and how the decision was made.
Hopefully, this will allow us to make progress as a community on how to tackle each of these issues thoughtfully and correctly.
As I look at this thread and the incidents in question, I see several areas for Google Trust Services to improve, and I thought I would talk a little about those here.
First, broadly at Google, we see automation as a force multiplier, a way to improve the quality of our services and make things more reproducible. This is why we focus heavily on automating as much as possible, including in non-engineering activities like auditing. For example, we have built extensive automation around streamlining our audits with the hope of moving to a world of being able to continually demonstrate our compliance with ease rather than the current state where the industry relies on point-in-time assessments.
Where Google Trust Services, and me in particular, have had a blind spot, is that we have failed to utilize automation to augment how we interact with the root programs and the communities supporting the WebPKI. The monitoring of these communities is something we believe we can automate, for example by polling Bugzilla and m.d.s.p for conversations relating to our services we can back manual participation with tracking in our internal systems. This will ensure a quicker response and that issues do not fall between the cracks in a manual process. I believe this will help address the latency in communication issues called out by Andrew.
Second, I believe that communications should be public and transparent, however, I do use individual conversations by phone, video, email, or text with many here to form my opinions on topics but have failed to reflect the outcomes of these communications on the list.
To that end, I want to stress to Mozilla and the members of this community, moving forward we will make a concerted effort to come to this list with issues much earlier and not just rely on the list as a communication channel for conversations related to incidents.
Third, just like we have an on-call rotation for security and operational issues, we also have what can be thought of as an on-call rotation for communication with Root Programs and this list on behalf of Google Trust Services. As people's life situations change and their scope of responsibilities evolve, we have failed to grow this on-call rotation, which has contributed to the rushed responses and delays in response.
To remedy this, we will be increasing the number of people involved in these communications, so we are no longer so dependent on a small number of individuals. We hope that this will improve the quality of our communication as well as the speed at which you hear from us.
Thanks for your patience,
Ryan Hurst
Product Management
Google Trust Services