Policy 2.6 Proposal: Move Compliance Date into policy

[Wayne Thayer]

Historically, the effective dates of new versions of the policy have been
maintained separately from the policy itself [1]. In our November
Communication, we learned that many CAs weren’t in compliance with policy
version 2.5 despite it having been in effect since June [2]. This proposal
is simply to add the Compliance Date to the policy itself, below the
version number, to make it more visible.

In addition, I propose that we adopt the norm of setting the Compliance
Date to 2 months after the Publication Date, to make it clearer that CAs
are expected to implement whatever changes are necessary no later than the
Compliance Date. This norm would not affect our ability to define more
specific Compliance Dates for specific changes to the policy.

This is: https://github.com/mozilla/pkipolicy/issues/117

[1] https://wiki.mozilla.org/CA/Root_Store_Policy_Archive
[2] https://groups.google.com/d/msg/mozilla.dev.security.policy/Bs3yRryKWFQ/
zJkUtz0GBAAJ

-------

This is a proposed update to Mozilla's root store policy for version
2.6. Please keep discussion in this group rather than on GitHub. Silence
is consent.

Policy 2.5 (current version):
https://github.com/mozilla/pkipolicy/blob/2.5/rootstore/policy.md

[Ryan Sleevi]

On Mon, Mar 19, 2018 at 6:28 PM, Wayne Thayer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Historically, the effective dates of new versions of the policy have been
> maintained separately from the policy itself [1]. In our November
> Communication, we learned that many CAs weren’t in compliance with policy
> version 2.5 despite it having been in effect since June [2]. This proposal
> is simply to add the Compliance Date to the policy itself, below the
> version number, to make it more visible.
>
> In addition, I propose that we adopt the norm of setting the Compliance
> Date to 2 months after the Publication Date, to make it clearer that CAs
> are expected to implement whatever changes are necessary no later than the
> Compliance Date. This norm would not affect our ability to define more
> specific Compliance Dates for specific changes to the policy.
>

Looking through [1], it seems like the Compliance Date has only differed
from the Publication Date once (with 2.0).

It's not clear to me that the 2.5 failure to adoption was related to
ambiguity around compliance dates versus, say, CAs not being in compliance
until directly chastised for non-compliance.

Thus, the deferral of 2 months is not entirely clear as to the reasoning.
Could you speak more to the thinking behind that?

[Wayne Thayer]

On Tue, Mar 20, 2018 at 8:19 AM, Ryan Sleevi <ry...@sleevi.com> wrote:

>
> Looking through [1], it seems like the Compliance Date has only differed
> from the Publication Date once (with 2.0).
>
It's not clear to me that the 2.5 failure to adoption was related to
> ambiguity around compliance dates versus, say, CAs not being in compliance
> until directly chastised for non-compliance.
>

> I believe both causes factored into the 2.5 compliance issues, but agree
that this change only helps with ambiguity around Compliance Dates.

Thus, the deferral of 2 months is not entirely clear as to the reasoning.
> Could you speak more to the thinking behind that?
>

> To begin, I hope we can agree, even if we don't like it, that changes like
this often get done just before the deadline (I believe there is an
incentive for this behavior, but all that is necessary here is to accept
that it happens regularly). Then the question becomes "what is the
deadline?", and of course the answer is the Compliance Date. Then I ask how
the Compliance Date is set and communicated to CAs, and that's where I see
the problem. For the 2.5 version there was a discussion around phase-in
periods for specific changes [1], but as best I can tell the Publication
Date (and hence the Compliance Date) was not communicated in advance [2],
meaning that CAs had no deadline to work toward. In addition, there was no
indication that the policy changes were finalized prior to the Publication
Date, so CAs didn't have a stable policy to implement prior to the
Publication Date, at which point they were expected to already have
complied.

My thinking is that this situation encourages CAs to ignore the Compliance
Date and work on their own schedules, and that communicating a Compliance
Date that is some reasonable amount of time after the Publication Date
clarifies our expectations.

[1]
https://groups.google.com/d/msg/mozilla.dev.security.policy/VFvlhIFFVbA/tFD51RzMAQAJ
[2]
https://groups.google.com/d/msg/mozilla.dev.security.policy/lSyrFEkREZk/9c67Y7bNAQAJ

[Ryan Sleevi]

So, in general, I think Mozilla policy has focused on actions CAs must take
(such as notify Mozilla of incidents), rather than the profile of their
certificates or a chance in scope of the BRs. The 10 blessed methods was an
exception to that, as was the scope of audits for S/MIME - but those were
exceptions, rather than the rule, and called out as such.

I think it's reasonable - especially in light of the discussion being had
regarding 2.6 and 3.2.2.4/3.2.2.5 - that when there are restrictions on the
technical actions a CA must take or can use, that there be a phase in time,
and what you say makes sense. But I think that's probably going to be the
exception, and so it may be better to set the Compliance Date ==
Publication Date by default, and then use the information gathering and
discussion phase (which Mozilla policy requires all CAs to monitor) to
identify if there are any surprises regarding phasing in changes.

As a concrete example, consider the policy changes requiring CAs to follow
m.d.s.p. That seems like something perfectly reasonable to expect immediate
compliance to, and indeed, highly desirable (for any incidents that emerge
within those two months until Compliance Date). Naively, it doesn't seem
like it should take two months for a CA to be able to monitor m.d.s.p., or
if it did, that may signal more problematic practices at play.

To be clear, I'm fully onboard with integrating the compliance date to
publication date, to avoid ambiguity. I am just putting forward the
notation that, by default, Compliance Date == Publication Date, and then if
there are reasonable concerns about the impact/challenges that might
present, raised during the discussion, then prior to finalization and
Publication, setting the Compliance Date out 30-60 days from that.

[Wayne Thayer]

> I am specifically thinking of CP/CPS updates, which were a major part of
the problem with version 2.5 compliance. There are a few proposals in the
2.6 queue that also require CP/CPS updates. Would you expect those to
trigger an exception as described above?

As a concrete example, consider the policy changes requiring CAs to follow
> m.d.s.p. That seems like something perfectly reasonable to expect immediate
> compliance to, and indeed, highly desirable (for any incidents that emerge
> within those two months until Compliance Date). Naively, it doesn't seem
> like it should take two months for a CA to be able to monitor m.d.s.p., or
> if it did, that may signal more problematic practices at play.
>

Again, the problem with this is that there is effectively no deadline when
policies are assumed to begin immediately upon publication. The same CAs
who take two months to begin monitoring m.d.s.p. when given a future
deadline are just as likely to begin monitoring it whenever they get around
to implementing the new policy during their next audit cycle if the
compliance deadline has already passed by the time they see that the policy
has been published.

>
>
To be clear, I'm fully onboard with integrating the compliance date to
> publication date, to avoid ambiguity. I am just putting forward the
> notation that, by default, Compliance Date == Publication Date, and then if
> there are reasonable concerns about the impact/challenges that might
> present, raised during the discussion, then prior to finalization and
> Publication, setting the Compliance Date out 30-60 days from that.
>

> I think we both agree that compliance with 2.5 was a problem that we'd
like to address. I've put forth the argument that having Compliance Date >
Publication Date could help to improve that. Do you disagree with my
assertion, or simply feel that on balance it is better to have new policies
enacted sooner but with less compliance? Maybe a better solution would be
to ask CAs to attest to their compliance via a survey each time we publish
a new policy version?

[Ryan Sleevi]

On Tue, Mar 20, 2018 at 8:27 PM, Wayne Thayer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> > I think it's reasonable - especially in light of the discussion being had
> > regarding 2.6 and 3.2.2.4/3.2.2.5 - that when there are restrictions on
> > the technical actions a CA must take or can use, that there be a phase in
> > time, and what you say makes sense. But I think that's probably going to
> be
> > the exception, and so it may be better to set the Compliance Date ==
> > Publication Date by default, and then use the information gathering and
> > discussion phase (which Mozilla policy requires all CAs to monitor) to
> > identify if there are any surprises regarding phasing in changes.
> >
> > I am specifically thinking of CP/CPS updates, which were a major part of
> the problem with version 2.5 compliance. There are a few proposals in the
> 2.6 queue that also require CP/CPS updates. Would you expect those to
> trigger an exception as described above?
>

Yup, I think it totally makes sense to phase those in, since they first
need to be finalized (via Publication) before CAs can fully update to be
conformant.

> As a concrete example, consider the policy changes requiring CAs to follow
> > m.d.s.p. That seems like something perfectly reasonable to expect
> immediate
> > compliance to, and indeed, highly desirable (for any incidents that
> emerge
> > within those two months until Compliance Date). Naively, it doesn't seem
> > like it should take two months for a CA to be able to monitor m.d.s.p.,
> or
> > if it did, that may signal more problematic practices at play.
> >
>
> Again, the problem with this is that there is effectively no deadline when
> policies are assumed to begin immediately upon publication. The same CAs
> who take two months to begin monitoring m.d.s.p. when given a future
> deadline are just as likely to begin monitoring it whenever they get around
> to implementing the new policy during their next audit cycle if the
> compliance deadline has already passed by the time they see that the policy
> has been published.
>

I'm not sure I understand this point. The deadline is immediate. I was
trying to highlight that with an immediate deadline, there's no excuse for
taking two months to monitor m.d.s.p. - but that's ok, because it doesn't
seem reasonable to take two months to begin with. Any CA out of compliance
is in an egregious breach of confidence at that point - especially if they
wait until their next audit cycle.

> I think we both agree that compliance with 2.5 was a problem that we'd
> like to address. I've put forth the argument that having Compliance Date >
> Publication Date could help to improve that. Do you disagree with my
> assertion, or simply feel that on balance it is better to have new policies
> enacted sooner but with less compliance? Maybe a better solution would be
> to ask CAs to attest to their compliance via a survey each time we publish
> a new policy version?

I don't think having Compliance Date > Publication Date will, in general,
help for situations of CAs being non-compliant. I think it will help with
some things - if their non-compliance was due to, for example, updating
systems or CP/CPSes - but I think it will harm other things - e.g. CAs
monitoring m.d.s.p. or disclosing certain things.

Regarding CAs attesting to compliance, I think that will help, but not
because of the Compliance Date or Publication Date. Rather, it will help
because no matter what we do or say, there are a subset of CAs who simply
will not and do not spend time following m.d.s.p. and only reply to emails
and only after several emails have gone by and only after being shamed at
the risk of distrust.

[Wayne Thayer]

On Wed, Mar 21, 2018 at 2:43 AM, Ryan Sleevi <ry...@sleevi.com> wrote:

>
>
> On Tue, Mar 20, 2018 at 8:27 PM, Wayne Thayer via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>>
>>
>> > I am specifically thinking of CP/CPS updates, which were a major part of
>> the problem with version 2.5 compliance. There are a few proposals in the
>> 2.6 queue that also require CP/CPS updates. Would you expect those to
>> trigger an exception as described above?
>>
>
> Yup, I think it totally makes sense to phase those in, since they first
> need to be finalized (via Publication) before CAs can fully update to be
> conformant.
>

Assuming those CP/CPS requirements make it into the 2.6 version, I will
plan to propose setting the Compliance Date for this specific version to be
2 months after the Publication Date, while not changing the default of
setting them to the same date for future versions of the policy. I think
that addresses both of our concerns.

I think we both agree that compliance with 2.5 was a problem that we'd
>> like to address. I've put forth the argument that having Compliance Date >
>> Publication Date could help to improve that. Do you disagree with my
>> assertion, or simply feel that on balance it is better to have new
>> policies
>> enacted sooner but with less compliance? Maybe a better solution would be
>> to ask CAs to attest to their compliance via a survey each time we publish
>> a new policy version?
>
>
> I don't think having Compliance Date > Publication Date will, in general,
> help for situations of CAs being non-compliant. I think it will help with
> some things - if their non-compliance was due to, for example, updating
> systems or CP/CPSes - but I think it will harm other things - e.g. CAs
> monitoring m.d.s.p. or disclosing certain things.
>
> Regarding CAs attesting to compliance, I think that will help, but not
> because of the Compliance Date or Publication Date. Rather, it will help
> because no matter what we do or say, there are a subset of CAs who simply
> will not and do not spend time following m.d.s.p. and only reply to emails
> and only after several emails have gone by and only after being shamed at
> the risk of distrust.
>

I agree, and will plan to send a CA Communication when the new version is
published.

[Jakob Bohm]

I would say that a implementation deadline of 0 will be instantly
violated by anyone not already in compliance (by chance).

Something like "follow m.d.s.p" (which includes subscribing to the mail
version if applicable, but also assigning one or more employees to the
task), could be done in a week or so, maybe a month if the deciding
boss is on holiday on the publication date.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Wayne Thayer]

I would summarize this discussion as follows:
- we'll add the Compliance Date to the policy document [1].
- the existing norm of setting the Compliance Date equal to the Publication
Date will remain.
- during final review of new versions of the policy, we will discuss any
changes that need to be made to the Compliance Date for the entire version
or individual changes.

- Wayne

[1]
https://github.com/mozilla/pkipolicy/commit/60e340635e31651208ea1c92f4e37295e6285950#diff-b7447c83436e3b067c828eb47cb80282
)

On Wed, Mar 21, 2018 at 10:09 AM, Jakob Bohm via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On 21/03/2018 10:43, Ryan Sleevi wrote:
>

> I would say that a implementation deadline of 0 will be instantly
> violated by anyone not already in compliance (by chance).
>
> Something like "follow m.d.s.p" (which includes subscribing to the mail
> version if applicable, but also assigning one or more employees to the
> task), could be done in a week or so, maybe a m

> <https://maps.google.com/?q=could+be+done+in+a+week+or+so,+maybe+a+m&entry=gmail&source=g>onth

> if the deciding
> boss is on holiday on the publication date.
>
>
> Enjoy
>
> Jakob
> --
> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
> Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
> This public discussion message is non-binding and may contain errors.
> WiseMo - Remote Service Management for PCs, Phones and Embedded

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>