Policy 2.6 Proposal: Require audits back to first issuance

[Wayne Thayer]

Mozilla began requiring BR audits for roots in our program in 2013 [1], but
we have a vague policy statement in section 3.1 regarding audit
requirements prior to inclusion:

Before being included and periodically thereafter, CAs MUST obtain certain
> audits…
>

BR section 8.1 contains the following paragraph:

If the CA does not have a currently valid Audit Report indicating
> compliance with one of the audit schemes listed in Section 8.1, then,
> before issuing Publicly-Trusted Certificates, the CA SHALL successfully
> complete a point-in-time readiness assessment performed in accordance with
> applicable standards under one of the audit schemes listed in Section 8.1.
> The point-in-time readiness assessment SHALL be completed no earlier than
> twelve (12) months prior to issuing Publicly-Trusted Certificates and SHALL
> be followed by a complete audit under such scheme within ninety (90) days
> of issuing the first Publicly-Trusted Certificate.
>

Unfortunately, the definition of Publicly-Trusted Certificates exempts
newly created roots from this requirement, and in practice we have seen
that violating this requirement does not prevent roots from receiving BR
audit statements. We continue to see inclusion requests for roots that do
not have an unbroken chain of BR audits back to first issuance.

I propose that we add a requirement to Mozilla policy section 3.1.3 for
roots to have contiguous audits beginning within 90 days of issuing the
first certificate. I chose 90 days to allow some time for issuing
subordinate CA certificates and test certificates in preparation for the
audit.
.
This is: https://github.com/mozilla/pkipolicy/issues/113

[1] https://wiki.mozilla.org/CA:CertificatePolicyV2.1#Audit_Criteria
[2]
https://groups.google.com/d/msg/mozilla.dev.security.policy/Mezqdljjerc/nIirftRqAgAJ

-------

This is a proposed update to Mozilla's root store policy for version
2.6. Please keep discussion in this group rather than on GitHub. Silence
is consent.

Policy 2.5 (current version):
https://github.com/mozilla/pkipolicy/blob/2.5/rootstore/policy.md

[Ryan Sleevi]

I'm not fully sure I understand the proposal here.

I would think that, for all roots created since 2012, the expectation is
that there is an unbroken series of audits, going from a Point in Time
audit (of the policies and infrastructure) to a Root Key Generation
Ceremony attestation (under the policies and practices) to a Period of Time
audit, with the issuance of any supporting infrastructure appearing between
the RKGC and the PoT and covered by the PoT audit.

Does that match your intent? Assuming I did not botch the audit timing
issues here

[Wayne Thayer]

On Thu, Mar 29, 2018 at 8:57 AM, Ryan Sleevi <ry...@sleevi.com> wrote:

>
> I'm not fully sure I understand the proposal here.
>
> I would think that, for all roots created since 2012, the expectation
>
is that there is an unbroken series of audits, going from a Point in Time
> audit (of the policies and infrastructure) to a Root Key Generation
> Ceremony attestation (under the policies and practices) to a Period of Time
> audit, with the issuance of any supporting infrastructure appearing between
> the RKGC and the PoT and covered by the PoT audit.
>

> This expectation apparently isn't clear given the numerous inclusion
requests - for roots created before and after 2012 - we're seen that are
lacking historic audits - Japan GPKI and ComSign for instance.

I wasn't thinking about requiring the RKGC audit report as part of our
inclusion process, but we probably should (assuming those reports aren't
confidential).

Does that match your intent? Assuming I did not botch the audit timing
> issues here
>

> I think so. My intent is to make it clear that roots must meet our audit
requirements even before they are included, and I'm open to suggestions on
the best way to achieve that in our policy.

[Ryan Sleevi]

On Thu, Mar 29, 2018 at 4:57 PM, Wayne Thayer <wth...@mozilla.com> wrote:

> On Thu, Mar 29, 2018 at 8:57 AM, Ryan Sleevi <ry...@sleevi.com> wrote:
>
>>
>> I'm not fully sure I understand the proposal here.
>>
>> I would think that, for all roots created since 2012, the expectation
>>
> is that there is an unbroken series of audits, going from a Point in Time
>> audit (of the policies and infrastructure) to a Root Key Generation
>> Ceremony attestation (under the policies and practices) to a Period of Time
>> audit, with the issuance of any supporting infrastructure appearing between
>> the RKGC and the PoT and covered by the PoT audit.
>>
>> This expectation apparently isn't clear given the numerous inclusion
> requests - for roots created before and after 2012 - we're seen that are
> lacking historic audits - Japan GPKI and ComSign for instance.
>
> I wasn't thinking about requiring the RKGC audit report as part of our
> inclusion process, but we probably should (assuming those reports aren't
> confidential).
>

I'm not sure the extent for an equivalent under ETSI, but under the
WebTrust side, practitioner guidance and illustrative reports have been
provided at
http://www.webtrust.org/practitioner-qualifications/item64422.aspx

Under the US Reporting - SSAE 18 guidance [1], an illustrative report
conforming to AT-C205 is provided on page 61.
Under the Canada Reporting - CSAE 3000 - 3001 [2], an illustrative report
as an independent assurance attestation engagement is provided on Page 85
Under the International Reporting - ISAE 3000 [3], an illustrative report
conforming to ISAE 3000 is provided on page 68.

My understanding is that these reports, and the international standards
they adhere to, are designed for public reporting.

[1] http://www.webtrust.org/practitioner-qualifications/docs/item85115.pdf
[2] http://www.webtrust.org/practitioner-qualifications/docs/item85114.pdf
[3] http://www.webtrust.org/practitioner-qualifications/docs/item85204.pdf

[crawfo...@gmail.com]

The standard audit cycle, we normally see for a brand new CA, is as follows:

1. Draft CP/CPS
2. Perform RKGC, observed and reported on by auditor
3. Point in time reporting
4. Period of time reporting

Most CAs elect to perform a RKGC before a point in time engagement, because without any CA activities in place most of the WebTrust criteria would not be applicable at the point time of reporting. As an example, much of the following sections would not be applicable:

4.0 – CA Key Lifecycle Management Controls
6.0 – Certificate Lifecycle Management

These sections represent a substantial portion of the WebTrust Principles and Criteria.

[Wayne Thayer]

Thanks Tim and Ryan for the information on WebTrust Root Key Generation
Ceremony audit reports. Can anyone say if an equivalent public-facing
report exists for ETSI audits? If so, I think we should require CAs to
provide these reports with their root inclusion requests.

Regarding the issue of requiring audits from the time of first issuance,
here's a specific proposal:

In section 2.3 (Baseline Requirements Conformance), add a new bullet that
states "Before being included, CAs MUST provide evidence that their root
certificates have continually, from the time of creation, complied with the
then current Mozilla Root Store Policy and CA/Browser Forum Baseline
Requirements."

- Wayne

On Fri, Mar 30, 2018 at 6:45 AM, crawfordtimj--- via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Thursday, March 29, 2018 at 10:59:10 AM UTC-5, Ryan Sleevi wrote:

> The standard audit cycle, we normally see for a brand new CA, is as
> follows:
>
> 1. Draft CP/CPS
> 2. Perform RKGC, observed and reported on by auditor
> 3. Point in time reporting
> 4. Period of time reporting
>
> Most CAs elect to perform a RKGC before a point in time engagement,
> because without any CA activities in place most of the WebTrust criteria
> would not be applicable at the point time of reporting. As an example,
> much of the following sections would not be applicable:
>
> 4.0 – CA Key Lifecycle Management Controls
> 6.0 – Certificate Lifecycle Management
>
> These sections represent a substantial portion of the WebTrust Principles
> and Criteria.

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>

[tom.p...@ualberta.net]

On Monday, April 2, 2018 at 7:12:19 PM UTC-6, Wayne Thayer wrote:
> In section 2.3 (Baseline Requirements Conformance), add a new bullet that
> states "Before being included, CAs MUST provide evidence that their root
> certificates have continually, from the time of creation, complied with the
> then current Mozilla Root Store Policy and CA/Browser Forum Baseline
> Requirements."

When I first read this, I parsed it as saying that the only root needs to comply with the policy at the time of creation (and not at later points in time). I don't have any suggestions on how to make it clear that the root needs to have complied at each time with the policy in force at that time.

-- Tom

[Wayne Thayer]

On Mon, Apr 2, 2018 at 9:28 PM, tom.prince--- via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Monday, April 2, 2018 at 7:12:19 PM UTC-6, Wayne Thayer wrote:

> > In section 2.3 (Baseline Requirements Conformance), add a new bullet that
> > states "Before being included, CAs MUST provide evidence that their root
> > certificates have continually, from the time of creation, complied with
> the
> > then current Mozilla Root Store Policy and CA/Browser Forum Baseline
> > Requirements."
>

> When I first read this, I parsed it as saying that the only root needs to
> comply with the policy at the time of creation (and not at later points in
> time). I don't have any suggestions on how to make it clear that the root
> needs to have complied at each time with the policy in force at that time.

> <https://lists.mozilla.org/listinfo/dev-security-policy>
>

Maybe this is clearer?

In section 2.3 (Baseline Requirements Conformance), add a new bullet that
states "Before being included, CAs MUST provide evidence that their root

certificates have, from the time of creation and continually thereafter,

[m.wied...@tuvit.de]

Hi Wayne

> Can anyone say if an equivalent public-facing
> report exists for ETSI audits? If so, I think we should require CAs to
> provide these reports with their root inclusion requests.

ETSI does require reports on key ceremonies (ETSI EN 319 411-1, 6.5.1 g).
But ETSI does NOT require these reports to be public.

Best regards
Matthias Wiedenhorst

[Wayne Thayer]

Thank you for responding Matthias.

On Wed, Apr 11, 2018 at 10:52 AM, m.wiedenhorst--- via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

>
> Hi Wayne

>
> > Can anyone say if an equivalent public-facing
> > report exists for ETSI audits? If so, I think we should require CAs to
> > provide these reports with their root inclusion requests.
>

> ETSI does require reports on key ceremonies (ETSI EN 319 411-1, 6.5.1 g).
> But ETSI does NOT require these reports to be public.
>

> Does ETSI ALLOW these reports to be public? In other words, could Mozilla
require CAs to publish them?

Best regards
> Matthias Wiedenhorst
>
>

[m.wied...@tuvit.de]

Hi again,

Well, on the one hand ETSI does not mandate anything about these reports being public or non-public. Hence it is not required to make them public, but it would not be forbidden either.

However, on the other hand in practical almost all key ceremony reports that I have either inspected during audits or even co-signed as the independent key ceremony auditor contained very detailed, internal information about the different steps of the performed ceremony and hence would never ever qualify for publication.

Best regards
Matthias Wiedenhorst

[Wayne Thayer]

The proposed language includes the requirement for compliance with both the
BRs and Mozilla policy, so it's a better fit for the section of our policy
titled "Inclusions" than the section titled "Baseline Requirements
Conformance". To close out this discussion, I added the proposed language
to section 7.1:

https://github.com/mozilla/pkipolicy/commit/55929f58da98a7af08fbf4bc2eb4537991de481b

On Wed, Apr 11, 2018 at 11:02 PM, m.wiedenhorst--- via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Hi again,

> Well, on the one hand ETSI does not mandate anything about these reports
> being public or non-public. Hence it is not required to make them public,
> but it would not be forbidden either.
>
> However, on the other hand in practical almost all key ceremony reports
> that I have either inspected during audits or even co-signed as the
> independent key ceremony auditor contained very detailed, internal
> information about the different steps of the performed ceremony and hence
> would never ever qualify for publication.
>

> Based on this feedback, I have decided not to push for a requirement that
CAs provide key generation ceremony audit reports at this time.

- Wayne