Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Misissuance/non-compliance remediation timelines
303 views
Skip to first unread message
Paul Kehrer
Feb 6, 2018, 8:03:28 PM2/6/18
to mozilla-dev-s...@lists.mozilla.org
A bit over 5 months ago I reported a series of OCSP responders that were
violating BRs (section 4.9.10) by returning GOOD on unknown serial numbers.
Since that time the majority of those responder endpoints have been fixed,
but several are still non-compliant; either with little communication or
continuing assurances that it will be fixed "soon", where soon is a date
that continuously slides into the future.
At the moment Mozilla possesses very few options when it comes to punitive
action and the lesson some CAs appear to be learning is that as long as you
don't rise to PROCERT levels of malfeasance/incompetence then the maximum
penalty is censure on bugzilla and email threads. Clearly this is not a
deterrent.
So, how long is too long? At what point should a CA incur consequences (and
what form can those consequences take) for failure to remediate despite
being given such immense latitude?
As a straw man: what if we developed a set of technical constraints related
to minimizing risk associated with CAs that are deemed to be acting poorly?
For example, CAs deemed a risk would have their certificate maximum
lifetime constrained to some amount less than the BRs currently allow.
Additional penalties could include removal of EV trust indicators,
constraining trust to a limited set of domains, marking contexts as
insecure, and finally full distrust. Once a CA lands in a risk category
further misissuance would escalate their risk to the community and thus
incur additional constraints. (I'm sure the community can come up with far
better tiers than I have!)
Adding controls like this will require significant time and effort from
Mozilla, but if we want to be able to manage the significant and ongoing
volume of misissuance/non-compliance we're seeing I believe some level of
granularity is needed.
-Paul (reaperhulk)
violating BRs (section 4.9.10) by returning GOOD on unknown serial numbers.
Since that time the majority of those responder endpoints have been fixed,
but several are still non-compliant; either with little communication or
continuing assurances that it will be fixed "soon", where soon is a date
that continuously slides into the future.
At the moment Mozilla possesses very few options when it comes to punitive
action and the lesson some CAs appear to be learning is that as long as you
don't rise to PROCERT levels of malfeasance/incompetence then the maximum
penalty is censure on bugzilla and email threads. Clearly this is not a
deterrent.
So, how long is too long? At what point should a CA incur consequences (and
what form can those consequences take) for failure to remediate despite
being given such immense latitude?
As a straw man: what if we developed a set of technical constraints related
to minimizing risk associated with CAs that are deemed to be acting poorly?
For example, CAs deemed a risk would have their certificate maximum
lifetime constrained to some amount less than the BRs currently allow.
Additional penalties could include removal of EV trust indicators,
constraining trust to a limited set of domains, marking contexts as
insecure, and finally full distrust. Once a CA lands in a risk category
further misissuance would escalate their risk to the community and thus
incur additional constraints. (I'm sure the community can come up with far
better tiers than I have!)
Adding controls like this will require significant time and effort from
Mozilla, but if we want to be able to manage the significant and ongoing
volume of misissuance/non-compliance we're seeing I believe some level of
granularity is needed.
-Paul (reaperhulk)
Tim Hollebeek
Feb 6, 2018, 8:30:45 PM2/6/18
to Paul Kehrer, mozilla-dev-s...@lists.mozilla.org
Paul,
I understand your frustration. I've read some of the recent threads about
"how long does it take to update a CPS?" and clearly there needs to be
some stronger compliance language in either the BRs or Mozilla policy
("CAs MUST be able to update their CPS within 90 days"). And as you note
such policies need to have teeth otherwise there will be some who will
just ignore them.
However negative penalties are not the only thing that should be
considered.
Mozilla should also have some way of recognizing CAs that are performing
above and beyond the minimum requirements. I would love to see Mozilla
encourage CAs to compete to be the best CA in Mozilla's program.
To satisfy both goals, I'd like to suggest an idea I've had for a while: at
some
point in time (annually?), Mozilla should assess their opinion of how well
each CA in the program is performing, and give them a letter grade. This
could include policy improvements like "Two consecutive failing grades,
or three consecutive C or lower grades and you're out of the Mozilla
program."
This would not preclude other actions as Mozilla deems necessary. But it
would provide a regular checkpoint for CAs to understand either "Hey,
you're great, keep up the good work!" or "Meh, we think you're ok." or
"Your performance to date is unacceptable. Get your sh*t together or
you're gone."
-Tim
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
I understand your frustration. I've read some of the recent threads about
"how long does it take to update a CPS?" and clearly there needs to be
some stronger compliance language in either the BRs or Mozilla policy
("CAs MUST be able to update their CPS within 90 days"). And as you note
such policies need to have teeth otherwise there will be some who will
just ignore them.
However negative penalties are not the only thing that should be
considered.
Mozilla should also have some way of recognizing CAs that are performing
above and beyond the minimum requirements. I would love to see Mozilla
encourage CAs to compete to be the best CA in Mozilla's program.
To satisfy both goals, I'd like to suggest an idea I've had for a while: at
some
point in time (annually?), Mozilla should assess their opinion of how well
each CA in the program is performing, and give them a letter grade. This
could include policy improvements like "Two consecutive failing grades,
or three consecutive C or lower grades and you're out of the Mozilla
program."
This would not preclude other actions as Mozilla deems necessary. But it
would provide a regular checkpoint for CAs to understand either "Hey,
you're great, keep up the good work!" or "Meh, we think you're ok." or
"Your performance to date is unacceptable. Get your sh*t together or
you're gone."
-Tim
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
Ryan Sleevi
Feb 6, 2018, 9:25:34 PM2/6/18
to Tim Hollebeek, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
How is this different from the negative penalties Paul mentioned?
Do you see the competition based on being the 'least bad' (i.e. more likely
to get an A because of no issues than a B because of some?)
The "CA Thunderdome" advocate in me wants to know whether you envision this
being graded on a curve ;)
I do want to highlight something else, which is that in today's CA
ecosystem model, the 'cost' of misissuance, malfeasance, or incompetence is
a fully externalized cost (to relying parties, site operators, and root
store administrators) unless/until that cost is recovered by distrusting or
sanctioning the CA (under the assumption that there's an economic value to
trust and that the loss of that trust is thus a loss of value). Perhaps its
time to operate on a more explicit model, in which all CAs are assumed to
have a base cost to the ecosystem that is recovered annually, with there
being further cost recovery in the presence of misissuance, and reduction
in cost if the CAs can demonstrate they've reduced the base ecosystem cost
(for example, shorter lived certs or automation). That seems to more
explicitly formalize some of the assumptions.
Do you see the competition based on being the 'least bad' (i.e. more likely
to get an A because of no issues than a B because of some?)
The "CA Thunderdome" advocate in me wants to know whether you envision this
being graded on a curve ;)
I do want to highlight something else, which is that in today's CA
ecosystem model, the 'cost' of misissuance, malfeasance, or incompetence is
a fully externalized cost (to relying parties, site operators, and root
store administrators) unless/until that cost is recovered by distrusting or
sanctioning the CA (under the assumption that there's an economic value to
trust and that the loss of that trust is thus a loss of value). Perhaps its
time to operate on a more explicit model, in which all CAs are assumed to
have a base cost to the ecosystem that is recovered annually, with there
being further cost recovery in the presence of misissuance, and reduction
in cost if the CAs can demonstrate they've reduced the base ecosystem cost
(for example, shorter lived certs or automation). That seems to more
explicitly formalize some of the assumptions.
Tim Hollebeek
Feb 6, 2018, 10:03:55 PM2/6/18
to ry...@sleevi.com, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
Absolutely not. I view the competition as being based as the “most best”.
You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage.
Sticks are good. Carrots are tasty.
-Tim
You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage.
Sticks are good. Carrots are tasty.
-Tim
Ryan Sleevi
Feb 7, 2018, 12:39:02 AM2/7/18
to Tim Hollebeek, ry...@sleevi.com, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
So your view is the “carrot” is getting to use Mozilla’s brand as an
endorsement, and the “stick” being that if you don’t get that endorsement
for a while, you get kicked out?
The assumption is that the branding of “best” is valuable - presumably,
through the indirect benefit of being able to appeal to customers as “the
highest rated (by Mozilla) CA”.
In practice, much like the CA/Browser Forum indirectly gave birth to the CA
“Security” Council, or the existence of firms like Netcraft or NSS Labs,
the more common outcome seems to be that if you don’t like the rules of the
game you’re playing, you make up your own/redefine them and try to claim
equivalency (much lol “alternative facts”). That is, I’m skeptical of
approaches that attempt to say “most good,” because those seem to encourage
all sorts of games of coming up with their own schemes, while “least bad”
is more actionable - as “most bad” is more likely to receive sanctions.
endorsement, and the “stick” being that if you don’t get that endorsement
for a while, you get kicked out?
The assumption is that the branding of “best” is valuable - presumably,
through the indirect benefit of being able to appeal to customers as “the
highest rated (by Mozilla) CA”.
In practice, much like the CA/Browser Forum indirectly gave birth to the CA
“Security” Council, or the existence of firms like Netcraft or NSS Labs,
the more common outcome seems to be that if you don’t like the rules of the
game you’re playing, you make up your own/redefine them and try to claim
equivalency (much lol “alternative facts”). That is, I’m skeptical of
approaches that attempt to say “most good,” because those seem to encourage
all sorts of games of coming up with their own schemes, while “least bad”
is more actionable - as “most bad” is more likely to receive sanctions.
Tim Hollebeek
Feb 7, 2018, 10:10:37 AM2/7/18
to Ryan Sleevi, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
That’s pretty much exactly not what I said.
From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Tuesday, February 6, 2018 10:38 PM
To: Tim Hollebeek <tim.ho...@digicert.com>
Cc: Paul Kehrer <paul.l...@gmail.com>; mozilla-dev-s...@lists.mozilla.org; ry...@sleevi.com
Subject: Re: Misissuance/non-compliance remediation timelines
So your view is the “carrot” is getting to use Mozilla’s brand as an endorsement, and the “stick” being that if you don’t get that endorsement for a while, you get kicked out?
The assumption is that the branding of “best” is valuable - presumably, through the indirect benefit of being able to appeal to customers as “the highest rated (by Mozilla) CA”.
In practice, much like the CA/Browser Forum indirectly gave birth to the CA “Security” Council, or the existence of firms like Netcraft or NSS Labs, the more common outcome seems to be that if you don’t like the rules of the game you’re playing, you make up your own/redefine them and try to claim equivalency (much lol “alternative facts”). That is, I’m skeptical of approaches that attempt to say “most good,” because those seem to encourage all sorts of games of coming up with their own schemes, while “least bad” is more actionable - as “most bad” is more likely to receive sanctions.
https://lists.mozilla.org/listinfo/dev-security-policy
From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Tuesday, February 6, 2018 10:38 PM
To: Tim Hollebeek <tim.ho...@digicert.com>
Cc: Paul Kehrer <paul.l...@gmail.com>; mozilla-dev-s...@lists.mozilla.org; ry...@sleevi.com
Subject: Re: Misissuance/non-compliance remediation timelines
So your view is the “carrot” is getting to use Mozilla’s brand as an endorsement, and the “stick” being that if you don’t get that endorsement for a while, you get kicked out?
The assumption is that the branding of “best” is valuable - presumably, through the indirect benefit of being able to appeal to customers as “the highest rated (by Mozilla) CA”.
In practice, much like the CA/Browser Forum indirectly gave birth to the CA “Security” Council, or the existence of firms like Netcraft or NSS Labs, the more common outcome seems to be that if you don’t like the rules of the game you’re playing, you make up your own/redefine them and try to claim equivalency (much lol “alternative facts”). That is, I’m skeptical of approaches that attempt to say “most good,” because those seem to encourage all sorts of games of coming up with their own schemes, while “least bad” is more actionable - as “most bad” is more likely to receive sanctions.
On Tue, Feb 6, 2018 at 10:03 PM Tim Hollebeek via dev-security-policy <dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> > wrote:
Absolutely not. I view the competition as being based as the “most best”.
You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage.
Sticks are good. Carrots are tasty.
-Tim
Do you see the competition based on being the 'least bad' (i.e. more likely to get an A because of no issues than a B because of some?)
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
Absolutely not. I view the competition as being based as the “most best”.
You cannot get an “A” (or even A- or B+) without significantly exceeding the minimum requirements, or demonstrating behaviors and practices that, while not required, are behaviors Mozilla wants to encourage.
Sticks are good. Carrots are tasty.
-Tim
Do you see the competition based on being the 'least bad' (i.e. more likely to get an A because of no issues than a B because of some?)
_______________________________________________
dev-security-policy mailing list
https://lists.mozilla.org/listinfo/dev-security-policy
Alex Gaynor
Feb 7, 2018, 10:15:08 AM2/7/18
to Tim Hollebeek, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
Hey Tim,
A piece I think I'm missing is what you see as the incentive for CAs to aim
for an "A" rather than being happy to have a "B". It reminds me of the old
joke: What do you call the Dr^W CA who graduated with a C average? Dr.^W
trusted to assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I
wholeheartedly concur with), it seems the place to focus is the folks who
are getting Ds right now. Therefore I think the essential part of your
email is your agreement that CAs which are persistently low performing need
to be recognized and potentially penalized for the sum total of their
behaviors.
Alex
>
>
A piece I think I'm missing is what you see as the incentive for CAs to aim
for an "A" rather than being happy to have a "B". It reminds me of the old
joke: What do you call the Dr^W CA who graduated with a C average? Dr.^W
trusted to assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I
wholeheartedly concur with), it seems the place to focus is the folks who
are getting Ds right now. Therefore I think the essential part of your
email is your agreement that CAs which are persistently low performing need
to be recognized and potentially penalized for the sum total of their
behaviors.
Alex
> > _______________________________________________
> > dev-security-policy mailing list
> > dev-secur...@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-security-policy
> > dev-security-policy mailing list
> > dev-secur...@lists.mozilla.org
>
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
>
>
Tim Hollebeek
Feb 7, 2018, 11:11:21 AM2/7/18
to Alex Gaynor, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
Alex,
Most CAs probably wouldn’t aim for an A. I don’t think doing this would be a game changer.
However there are some CAs that would. And I think that would be a positive thing, and lead to more innovation in best practices that could become mandatory for everyone over time.
And I don’t disagree with you that action is needed on those who are currently getting Ds. I’m very disturbed by the behavior of about half of the CAs in the industry.
-Tim
From: Alex Gaynor [mailto:aga...@mozilla.com]
Sent: Wednesday, February 7, 2018 8:15 AM
To: Tim Hollebeek <tim.ho...@digicert.com>
Cc: Paul Kehrer <paul.l...@gmail.com>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Misissuance/non-compliance remediation timelines
> dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
https://lists.mozilla.org/listinfo/dev-security-policy
Most CAs probably wouldn’t aim for an A. I don’t think doing this would be a game changer.
However there are some CAs that would. And I think that would be a positive thing, and lead to more innovation in best practices that could become mandatory for everyone over time.
And I don’t disagree with you that action is needed on those who are currently getting Ds. I’m very disturbed by the behavior of about half of the CAs in the industry.
-Tim
From: Alex Gaynor [mailto:aga...@mozilla.com]
Sent: Wednesday, February 7, 2018 8:15 AM
To: Tim Hollebeek <tim.ho...@digicert.com>
Cc: Paul Kehrer <paul.l...@gmail.com>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: Misissuance/non-compliance remediation timelines
Hey Tim,
A piece I think I'm missing is what you see as the incentive for CAs to aim for an "A" rather than being happy to have a "B". It reminds me of the old joke: What do you call the Dr^W CA who graduated with a C average? Dr.^W trusted to assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I wholeheartedly concur with), it seems the place to focus is the folks who are getting Ds right now. Therefore I think the essential part of your email is your agreement that CAs which are persistently low performing need to be recognized and potentially penalized for the sum total of their behaviors.
Alex
A piece I think I'm missing is what you see as the incentive for CAs to aim for an "A" rather than being happy to have a "B". It reminds me of the old joke: What do you call the Dr^W CA who graduated with a C average? Dr.^W trusted to assert www-wide identity :-)
That said, given the issues Paul highlighted in his original mail (which I wholeheartedly concur with), it seems the place to focus is the folks who are getting Ds right now. Therefore I think the essential part of your email is your agreement that CAs which are persistently low performing need to be recognized and potentially penalized for the sum total of their behaviors.
Alex
> https://lists.mozilla.org/listinfo/dev-security-policy
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org>
_______________________________________________
dev-security-policy mailing list
https://lists.mozilla.org/listinfo/dev-security-policy
James Burton
Feb 7, 2018, 2:55:56 PM2/7/18
to Tim Hollebeek, Alex Gaynor, mozilla-dev-s...@lists.mozilla.org, Paul Kehrer
The idea of a grading system being used to judge CAs compliance will be a total disaster. We should instead be focusing our efforts on more transparency.
James
James
Gervase Markham
Feb 8, 2018, 10:44:01 AM2/8/18
to mozilla-dev-s...@lists.mozilla.org
On 07/02/18 15:14, Alex Gaynor wrote:
> That said, given the issues Paul highlighted in his original mail (which I
> wholeheartedly concur with), it seems the place to focus is the folks who
> are getting Ds right now. Therefore I think the essential part of your
> email is your agreement that CAs which are persistently low performing need
> to be recognized and potentially penalized for the sum total of their
> behaviors.
This is, in a reasonably strong sense, what happens now. We require each
> That said, given the issues Paul highlighted in his original mail (which I
> wholeheartedly concur with), it seems the place to focus is the folks who
> are getting Ds right now. Therefore I think the essential part of your
> email is your agreement that CAs which are persistently low performing need
> to be recognized and potentially penalized for the sum total of their
> behaviors.
incident in which a CA is involved to be documented in a public bug, so
all can see the timeline, outcomes, how the CA reacted and other factors
which might be taken into account when determining a CA's competence.
Occasionally, we decide that some CA's list of recent[0] problems is
sufficiently serious[0] that we need to run an investigation. We do so,
and invite the CA to more formally comment on the sum total of the
problems. We assess the responses and the style and level of response,
and make a determination[0]. This is what happened to WoSign, Symantec
and PROCERT:
https://wiki.mozilla.org/CA:WoSign_Issues
https://wiki.mozilla.org/CA:Symantec_Issues
https://wiki.mozilla.org/CA:PROCERT_Issues
I therefore expect and hope that CAs in our program have noted what
happened in those cases, particularly PROCERT (which is probably the
clearest case of simple "general incompetence" that we have had), and
want to make sure they are not next.
Gerv
[0] Yes, this is vague. But so is the concept of "trust".
Wayne Thayer
Feb 8, 2018, 12:24:23 PM2/8/18
to Paul Kehrer, mozilla-dev-security-policy
On Tue, Feb 6, 2018 at 6:03 PM, Paul Kehrer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> So, how long is too long?
>
This is the crux of the issue for me. If a CA (that really should have
dev-secur...@lists.mozilla.org> wrote:
>
> So, how long is too long?
>
stopped responding 'good' for unknown certs back in 2013) needs to select,
purchase, and deploy an entirely new OCSP system, is 5 months a really long
time? From their perspective, probably not.
I don't believe there is a standard answer to this question that can apply
to a whole class of issues, but I do think we could do a better job of
communicating our expectations when a situation like this arises by making
a statement such as 'being a CA that has been granted the public's trust,
Mozilla expects problem X to be resolved in Y days'. Responsible CAs will
meet the deadline and thus distinguish themselves from CAs that simply
aren't taking the problem seriously.
Wayne
Paul Kehrer
Feb 9, 2018, 1:26:00 AM2/9/18
to mozilla-dev-security-policy
On February 9, 2018 at 1:24:12 AM, Wayne Thayer (wth...@mozilla.com) wrote:
On Tue, Feb 6, 2018 at 6:03 PM, Paul Kehrer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> So, how long is too long?
>
This is the crux of the issue for me. If a CA (that really should have
stopped responding 'good' for unknown certs back in 2013) needs to select,
purchase, and deploy an entirely new OCSP system, is 5 months a really long
time? From their perspective, probably not.
I agree that from their perspective that's a short period of time. However,
On Tue, Feb 6, 2018 at 6:03 PM, Paul Kehrer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> So, how long is too long?
>
This is the crux of the issue for me. If a CA (that really should have
stopped responding 'good' for unknown certs back in 2013) needs to select,
purchase, and deploy an entirely new OCSP system, is 5 months a really long
time? From their perspective, probably not.
I strongly believe that asking the public to bear the burden of a CA's own
incompetence is, while historically what has been done, not tenable moving
forward. In the specific case of the OCSP issues I question why we should
give CAs half a year to remediate a fault that had already been a
requirement for 4 years when it was discovered. In many ways a CA's primary
job is knowing and following the rules, so why are we giving CAs who fail
in such colossal fashion a free pass?
I don't believe there is a standard answer to this question that can apply
to a whole class of issues, but I do think we could do a better job of
communicating our expectations when a situation like this arises by making
a statement such as 'being a CA that has been granted the public's trust,
Mozilla expects problem X to be resolved in Y days'. Responsible CAs will
meet the deadline and thus distinguish themselves from CAs that simply
aren't taking the problem seriously.
I believe a graduated notion of penalties and risk mitigation would make it
easier for Mozilla to push CAs. If the only penalty is distrust then
"little" things like a slow but steady trickle of misissued certificates,
operating your OCSP responder out of compliance for 4 years, or failing to
get a BR audit for 3 years after they became required never rise to the
level of a distrust conversation. If, on the other hand, there exists a set
of penalty tiers a CA can be placed on that path relatively quickly.
Instead of a "sudden" (from the perspective of the CA or subscribers who
aren't engaged with policy discussions on mdsp) distrust thread, there is
an escalation that makes everyone aware of a CA's need to shape up.
-Paul
0 new messages