Policy 2.7.1: MRSP Issue #207: Require audit statements to provide information about which CA Locations were audited

[Ben Wilson]

All,

This email is part of the discussion for the next version of the Mozilla
Root Store Policy (MSRP), version 2.7.1, to be published during of Q1-2021.

For audit delays, we currently require that audit statements disclose the
locations that were and were not audited, but that requirement has not been
incorporated yet into the MRSP. See
https://wiki.mozilla.org/CA/Audit_Statements#Minimum_Expectations. That
provision reads as follows:

Disclose each location (at the state/province level) that was included in
the scope of the audit or should have been included in the scope of the
audit, whether the inspection was physically carried out in person at each
location, and which audit criteria were checked (or not checked) at each
location.

- If the CA has more than one location in the same state/province, then
use terminology to clarify the number of facilities in that state/province
and whether or not all of them were audited. For example: "Facility 1 in
Province", "Facility 2 in Province, Facility 3 in Province" *or*
"Primary Facility in Province", "Secondary Facility in Province", "Tertiary
Facility in Province".
- The public audit statement does not need to identify the type of
Facility.
- "Facility" includes: data center locations, registration authority
locations, where IT and business process controls of CA operations are
performed, facility hosting an active HSM with CA private keys,
facility or
bank deposit box storing a deactivated and encrypted copy of a
private key.

It is proposed by Issue #207
<https://github.com/mozilla/pkipolicy/issues/207> that this language
requiring the disclosure of site locations--audited and unaudited--be made
clearly part of the MSRP by reference to the language above.

A similar method of incorporating by reference has been taken in section
2.4 of the MSRP
<https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#24-incidents>
with respect to incident reporting and in section 7.1
<https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#71-inclusions>
with requirements for the CA inclusion process.

It is proposed that we add a new subsection 10 to MRSP section 3.1.4
<https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/#314-public-audit-information>
that would require that audit documentation disclose the facility site
locations that were, or were not, examined.

One concern that has been raised previously is that the Baseline
Requirements do not define "facility site location". However, we believe
that the language above at
https://wiki.mozilla.org/CA/Audit_Statements#Minimum_Expectations
accomplishes that. We're open to suggestions for re-wording parts of it to
make it even better.

Currently, the audit letter template for WebTrust for CAs references the
site location audited (at the level of specificity that is proposed
above). Over this past year, due to COVID, some ETSI attestation letters
have also explained which sites were and were not checked. This approach
seems to work, and the additional information will be beneficial in the
future as we evaluate the security and trust of PKI service providers.

So, for the page cited above, we intend to move "Minimum Expectations" out
from under "Audit Delay" so that it stands separately as a requirement for
disclosing the facility site location. Then we will also revise MRSP
section 3.1.4 by inserting a new subsection 10 to require "facility site
locations that were, or were not, examined" with a hyperlink to the Minimum
Expectations language cited above.

We look forward to your comments and suggestions.

Sincerely yours,

Ben

[Jeff Ward]

Hi Ben. Happy New Year. I have asked the WebTrust Task Force members to provide their comments and Don and I will then provide a more detailed response. I wanted to be sure to get each of the major firms' feedback before responding.

Thank you.

Jeff

[Jeff Ward]

Ben, Don and I offer the following response, which has been vetted through the WebTrust Task Force:

Proposed Requirement

Disclose each location (at the state/province level) that was included in the scope of the audit or should have been included in the scope of the audit, whether the inspection was physically carried out in person at each location, and which audit criteria were checked (or not checked) at each location.

• If the CA has more than one location in the same state/province, then use terminology to clarify the number of facilities in that state/province and whether or not all of them were audited. For example: "Facility 1 in Province", "Facility 2 in Province, Facility 3 in Province" *or*"Primary Facility in Province", "Secondary Facility in Province", "Tertiary Facility in Province".

• The public audit statement does not need to identify the type of Facility.

• "Facility" includes: data center locations, registration authority locations, where IT and business process controls of CA operations are performed, facility hosting an active HSM with CA private keys, facility or bank deposit box storing a deactivated and encrypted copy of a private key.

Task Force Comments on Proposed Requirement
1. Disclosure of each location that was included in the audit. At the present time, both the public WebTrust report, as well as the detailed controls report, require disclosure of each location involved in the audit. The specificity of disclosure can vary, often at the city level or higher, in order to protect the confidentiality of locations required by some CAs.

2. Disclosure of whether the inspection was physically carried out in person at each location. This is not currently provided in the public WebTrust report nor would it likely be included in the future. This detail, however, would likely be provided in a detailed controls report.

Mozilla is aware of the current WebTrust Covid-19 report scenarios where the auditor provides specific qualifications due to inability to physically attend a significant location due to COVID-19 lockdowns. It is preferable to test controls in person at the organization’s location, however that is not a realistic expectation for the foreseeable COVID-19 future.

It should also be noted that, in the scenario where multiple locations exist, an auditor may use sampling to test a sample of the locations where the controls exercised are common throughout the relevant hierarchy under audit (this is common in financial statement audits involving multiple locations). In this scenario some locations are physically audited and the remainder are tested through alternative means.

It should be noted that auditors are able to perform many testing procedures remotely/virtually with the same level of effectiveness as onsite/in-person. For example, auditors have been able to develop alternate procedures by combining virtual tours, log reviews, security camera footage and other testing methods to gain evidence supporting the operating effectiveness of controls without physically visiting a location. Testing physical security, environmental controls, offline ceremonies, and asset management are the exception and typically more effectively tested in-person. If the auditor is not able to obtain sufficient appropriate audit evidence to support a clean audit opinion a qualification and reasons therefore will continue to be set out in the WebTrust report.


3. Definition of Facilities. The term “facility” is not defined in the CA/B Forum guidance (BRs) or formally in any other WebTrust materials. The functional term used in WebTrust (3.1, and 3.4) and in the BRs (5.4.1) is “CA facility”. CA facility is often understood to be a facility securing CA private keys within HSMs. Many CAs adopt this understanding in their respective CP and CPSs. This is an important distinction because section 5.1 of the BRs does not provide any requirements for physical security or environmental requirements. It also does not distinguish between the controls based on the types of facilities. CAs detail the controls requirements of CA facilities but remain silent on controls at “registration authority locations” or “locations where IT and business process controls are performed.” Without consistently defined requirements for locations beyond CA facilities, there is potential for a wide variety of interpretation resulting in inconsistencies.

With many employees working from home due to COVID-19, for example, performing registration authority activities for the CA,
• their homes could be declared to be a facility under the proposed definition,
• It would not make sense to declare such, include the location in an audit report, nor
• expect an auditor to physically visit such (which under COVID-19 lockdown in most cases is illegal).

[Ben Wilson]

Thanks, Jeff. These are useful comments, and I will take them into
consideration in revising our proposal.

On Tue, Jan 12, 2021 at 8:38 AM Jeff Ward via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Sunday, January 3, 2021 at 8:38:05 AM UTC-6, Jeff Ward wrote:

> Ben, Don and I offer the following response, which has been vetted through
> the WebTrust Task Force:
>
> Proposed Requirement

> Disclose each location (at the state/province level) that was included in
> the scope of the audit or should have been included in the scope of the
> audit, whether the inspection was physically carried out in person at each
> location, and which audit criteria were checked (or not checked) at each
> location.

> • If the CA has more than one location in the same state/province,

> then use terminology to clarify the number of facilities in that
> state/province and whether or not all of them were audited. For example:
> "Facility 1 in Province", "Facility 2 in Province, Facility 3 in Province"
> *or*"Primary Facility in Province", "Secondary Facility in Province",
> "Tertiary Facility in Province".
>

> • The public audit statement does not need to identify the type of
> Facility.
>
> • "Facility" includes: data center locations, registration authority

> locations, where IT and business process controls of CA operations are
> performed, facility hosting an active HSM with CA private keys, facility or
> bank deposit box storing a deactivated and encrypted copy of a private key.
>

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>

[Ben Wilson]

The current proposed draft of changes is at
https://github.com/BenWilson-Mozilla/pkipolicy/commit/443b4c5d51500005942a216322480f3a6a273ea2

Right now, I'm considering having subsection of MRSP section 3.1.4 say,
"the CA locations that were or were not audited" - with a hyperlink to
https://wiki.mozilla.org/CA/Audit_Statements#Audited_Locations, and then
elaborating there as needed.

On Wed, Jan 13, 2021 at 10:25 AM Ben Wilson <bwi...@mozilla.com> wrote:

> Thanks, Jeff. These are useful comments, and I will take them into
> consideration in revising our proposal.
>
> On Tue, Jan 12, 2021 at 8:38 AM Jeff Ward via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> On Sunday, January 3, 2021 at 8:38:05 AM UTC-6, Jeff Ward wrote:

>> Ben, Don and I offer the following response, which has been vetted
>> through the WebTrust Task Force:
>>
>> Proposed Requirement

>> Disclose each location (at the state/province level) that was included in
>> the scope of the audit or should have been included in the scope of the
>> audit, whether the inspection was physically carried out in person at each
>> location, and which audit criteria were checked (or not checked) at each
>> location.

>> • If the CA has more than one location in the same state/province,

>> then use terminology to clarify the number of facilities in that
>> state/province and whether or not all of them were audited. For example:
>> "Facility 1 in Province", "Facility 2 in Province, Facility 3 in Province"
>> *or*"Primary Facility in Province", "Secondary Facility in Province",
>> "Tertiary Facility in Province".
>>

>> • The public audit statement does not need to identify the type of
>> Facility.
>>
>> • "Facility" includes: data center locations, registration

>> authority locations, where IT and business process controls of CA
>> operations are performed, facility hosting an active HSM with CA private
>> keys, facility or bank deposit box storing a deactivated and encrypted copy
>> of a private key.
>>