How harsh (in general) should Mozilla be towards CAs?

[Jakob Bohm]

This thread is for the general principles, it takes no stance on any
particular cases, as that would quickly derail the discussion.

Over the years, there has been some variation among participants in how
harshly individual mistakes by CAs should be judged, ranging from "just
file a satisfactory incident report, and all will be fine" to "Any tiny
mistake could legally be construed as violating a formal requirement
that would be much more catastrophic under other circumstances,
therefore the maximum penalty of immediate distrust must be imposed".

I believe some middle ground between those extremes would be better for
all involved (including relying parties/users).

Now obviously there will be cases that uncontroversially fall under one
of the extremes, such as the DigiNotar PKIOverheid disaster in the
Netherlands or the "Stripe in another state" experiments. The problem
is all the cases in the spectrum between the extremes.

I believe that the assessment of cases should be based on a balanced
view of the actual circumstances, and that blindly taking either the
"extremely lenient" or "extremely harsh" stance is unfair for everybody
directly or indirectly affected.

Furthermore, people with some clout tend to shut down all
counterarguments when taking either extreme position, creating situation
there only their own position is heard, making the entire "community"
aspect an illusion.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Ryan Sleevi]

On Thu, Nov 8, 2018 at 5:51 PM Jakob Bohm via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> This thread is for the general principles, it takes no stance on any
> particular cases, as that would quickly derail the discussion.
>
> Over the years, there has been some variation among participants in how
> harshly individual mistakes by CAs should be judged, ranging from "just
> file a satisfactory incident report, and all will be fine" to "Any tiny
> mistake could legally be construed as violating a formal requirement
> that would be much more catastrophic under other circumstances,
> therefore the maximum penalty of immediate distrust must be imposed".
>
> I believe some middle ground between those extremes would be better for
> all involved (including relying parties/users).

Concretely, could you explain what that practically looks like, as you
believe?

Can you also state what you believe were appropriate alternatives raised by
the community, and that were ignored, when considering past incidents?

I ask these, because it’s not reasonable to suggest there’s some
as-of-yet-unmet middle ground without actually defining what you believe to
be examples of both ends of the spectrum are. The reality is that almost
everything done in the past several years has been on the “more lenient
than the middle” in practical terms, yet you’re implying, especially later
in your message, that you believe them to be on some extreme.

Without providing those sorts of concrete examples, it can come off very
shady - like asking “have you stopped beating your wife yet”. It’s
suggestive without being constructive or educational.

I believe that the assessment of cases should be based on a balanced
> view of the actual circumstances, and that blindly taking either the
> "extremely lenient" or "extremely harsh" stance is unfair for everybody
> directly or indirectly affected.

This is a bit leading, or perhaps, misleading. I don’t think anyone here
would disagree with the first half - that’s very much what the process is
currently designed to support and accomplish. Either you’re stating a fact
that everyone agrees with, or you’re presenting it as if somehow you’re
unique in this or perhaps (combined with later remarks) a minority in this
view. The second half, while also agreeable and part of the principles, is
worded in such a way that it suggests you believe those things are
happening. Unfortunately, you don’t actually detail how - it’s just an
implication.

If you believe that extremes are being blindly taken, you should call it
out. That’s part of the community process, designed to get feedback. It may
be that people disagree with you, but that doesn’t mean you can’t or
shouldn’t feel free to call it out. If you find people are constantly
disagreeing with you, that might help provide an opportunity to explore if
maybe you’re the one in the wrong. Either way, the first step to that is to
be direct at it; merely implying things helps no one and hurts real
progress.

Furthermore, people with some clout tend to shut down all
> counterarguments when taking either extreme position, creating situation
> there only their own position is heard, making the entire "community"
> aspect an illusion.

Without wanting to tone police, you could have achieved a lot more without
this closing paragraph. You have intimated as much before, and it’s been
responded to before. Repeating it here undermines it for those who’ve seen
those past discussions, and misleads those who haven’t.

There hasn’t been “shouting” down of arguments; different people have
disagreed with in the past, and presented more or less compelling arguments
for their position. Opinions were heard, facts were considered, and a
result was chosen. Just because some arguments were poor doesn’t mean they
weren’t considered, and just because some priorities were different doesn’t
mean they aren’t still important as well.

I hope you can see how messages like this can result in future arguments
being undermined. On its whole, it’s all fundamentally agreeable - yes, the
process for action is designed to be transparent, designed to consider all
the details so as not to be blind, to consider community feedback to not be
hasty, and to ensure consistency and fairness. Either it’s a position that
adds no value, because it’s restating things, making it easier to ignore
future ideas as being equally reductive and repetitive, or it’s a position
that comes off shady, by trying to hint that these things aren’t happening
without providing concrete examples.

[Jakob Bohm]

On 09/11/2018 07:21, Ryan Sleevi wrote:
> On Thu, Nov 8, 2018 at 5:51 PM Jakob Bohm via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> This thread is for the general principles, it takes no stance on any
>> particular cases, as that would quickly derail the discussion.
>>
>> Over the years, there has been some variation among participants in how
>> harshly individual mistakes by CAs should be judged, ranging from "just
>> file a satisfactory incident report, and all will be fine" to "Any tiny
>> mistake could legally be construed as violating a formal requirement
>> that would be much more catastrophic under other circumstances,
>> therefore the maximum penalty of immediate distrust must be imposed".
>>
>> I believe some middle ground between those extremes would be better for
>> all involved (including relying parties/users).
>
>
> Concretely, could you explain what that practically looks like, as you
> believe?
>

Concrete examples would derail the discussion as stated in my first
paragraph.

> Can you also state what you believe were appropriate alternatives raised by
> the community, and that were ignored, when considering past incidents?
>

I have myself raised such concerns in the past, and been flamed to death
mostly by you.

> I ask these, because it’s not reasonable to suggest there’s some
> as-of-yet-unmet middle ground without actually defining what you believe to
> be examples of both ends of the spectrum are. The reality is that almost
> everything done in the past several years has been on the “more lenient
> than the middle” in practical terms, yet you’re implying, especially later
> in your message, that you believe them to be on some extreme.
>

I provided two uncontroversial examples of the ends of the spectrum that
you conveniently snipped just so you could pretend they weren't here.

I must once again reiterate that trying to turn this into a discussion
of concrete real world middle ground cases would derail the discussion.

> Without providing those sorts of concrete examples, it can come off very
> shady - like asking “have you stopped beating your wife yet”. It’s
> suggestive without being constructive or educational.

Anyone reading the list/newsgroup over the past 12 months should see
plenty of cases where someone argues an issue is minor but is constantly
being told that it isn't for various reasons. There are at least 2 such
discussion threads ongoing right now.

>
> I believe that the assessment of cases should be based on a balanced
>> view of the actual circumstances, and that blindly taking either the
>> "extremely lenient" or "extremely harsh" stance is unfair for everybody
>> directly or indirectly affected.
>
>
> This is a bit leading, or perhaps, misleading. I don’t think anyone here
> would disagree with the first half - that’s very much what the process is
> currently designed to support and accomplish. Either you’re stating a fact
> that everyone agrees with, or you’re presenting it as if somehow you’re
> unique in this or perhaps (combined with later remarks) a minority in this
> view. The second half, while also agreeable and part of the principles, is
> worded in such a way that it suggests you believe those things are
> happening. Unfortunately, you don’t actually detail how - it’s just an
> implication.
>

I explicitly prefixed that paragraph with "I believe" to indicate that
this is my viewpoint, not an objective truth.

> If you believe that extremes are being blindly taken, you should call it
> out. That’s part of the community process, designed to get feedback. It may
> be that people disagree with you, but that doesn’t mean you can’t or
> shouldn’t feel free to call it out. If you find people are constantly
> disagreeing with you, that might help provide an opportunity to explore if
> maybe you’re the one in the wrong. Either way, the first step to that is to
> be direct at it; merely implying things helps no one and hurts real
> progress.
>

You yourself have told me to stay out of threads or to prove the
opposite extreme when I suggested a middle ground might exist.

> Furthermore, people with some clout tend to shut down all
>> counterarguments when taking either extreme position, creating situation
>> there only their own position is heard, making the entire "community"
>> aspect an illusion.
>
>
> Without wanting to tone police, you could have achieved a lot more without
> this closing paragraph. You have intimated as much before, and it’s been
> responded to before. Repeating it here undermines it for those who’ve seen
> those past discussions, and misleads those who haven’t.
>
> There hasn’t been “shouting” down of arguments; different people have
> disagreed with in the past, and presented more or less compelling arguments
> for their position. Opinions were heard, facts were considered, and a
> result was chosen. Just because some arguments were poor doesn’t mean they
> weren’t considered, and just because some priorities were different doesn’t
> mean they aren’t still important as well.
>

You yourself have been very shouting towards me whenever I made such
suggestions. At least one other poster called you out on your ways
within the past few months and got shut down with formalistic arguments
about definitions.

> I hope you can see how messages like this can result in future arguments
> being undermined. On its whole, it’s all fundamentally agreeable - yes, the
> process for action is designed to be transparent, designed to consider all
> the details so as not to be blind, to consider community feedback to not be
> hasty, and to ensure consistency and fairness. Either it’s a position that
> adds no value, because it’s restating things, making it easier to ignore
> future ideas as being equally reductive and repetitive, or it’s a position
> that comes off shady, by trying to hint that these things aren’t happening
> without providing concrete examples.
>

This is about stopping bad practices from causing more damage than it
has already done.

[westm...@gmail.com]

I think that punishments of the CAs for already exists in Mozilla Root Store are very mild, and some CAs often do not pay any attention to this...

[Jakob Bohm]

On 09/11/2018 12:44, westm...@gmail.com wrote:
> I think that punishments of the CAs for already exists in Mozilla Root Store are very mild, and some CAs often do not pay any attention to this...
>

However there are also some very harsh punishments handed out, such as
distrusting some CAs (most notably happened to Symantec and WoSign, but
others are also teetering), and distrusting auditors (most notably
happened to the branch of Ernst & Young that audited the bad parts of
those two).

A line of arguments often seen is that someone failed once to do
<something> completely right, therefore they cannot be trusted to do
anything similar to <something> right at all, therefore they should no
longer be trusted.

Another line of arguments is that if a CA says that the <something> they
did wrong was not that dangerous (because some specific argument for
why it can't be exploited against users), they are accused of not taking
security of the users seriously and should no longer be trusted.

A third line of arguments is that if a certificate was issued in a
slightly wrong way (misspelling some mandatory part etc.), then this
should be subject to the 24 hour revocation deadline for certificates
that were truly misissued to the wrong person.

Each of these arguments for maximum punishment and/or maximum
inconvenience for innocent bystanders is backed by a formal/legal
interpretation of existing rules as making this the only possible
outcome.

[westm...@gmail.com]

If Google had not started the process of Symantec distrust, Mozilla would never have come to this step, I think.

[Hanno Böck]

On Fri, 9 Nov 2018 14:56:41 +0100

Jakob Bohm via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:

> However there are also some very harsh punishments handed out, such as
> distrusting some CAs (most notably happened to Symantec and WoSign,
> but others are also teetering), and distrusting auditors (most notably
> happened to the branch of Ernst & Young that audited the bad parts of
> those two).
>
> A line of arguments often seen is that someone failed once to do
> <something> completely right, therefore they cannot be trusted to do
> anything similar to <something> right at all, therefore they should no
> longer be trusted.

I don't think anyone ever said something like that. Particularly
I'm not aware of any recent incident where a CA failed *once* and got
distrusted.

In the cases you mention - Symantec and Wosign - there were multiple
issues. In both cases there was also plenty of opportunity for the
affected CAs to explain and improve things before a distrust was
even considered. It was repeated failures and a long list of issues
that led to the distrust.


--
Hanno Böck
https://hboeck.de/

mail/jabber: ha...@hboeck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

[Ben Wilson]

Jakob Bohm wrote "Each of these arguments for maximum punishment and/or

maximum inconvenience for innocent bystanders is backed by a formal/legal
interpretation of existing rules as making this the only possible outcome."

I'd agree - heavy-handed, strict enforcement of some rules unnecessarily
punishes the
larger community and harms the Internet environment. I might be wrong, but I
don't
believe that Mozilla or Google has an established set of "Sentencing
Guidelines" or even
a set of procedural rights, but maybe these are needed? And maybe we need
additional
criteria on whether a proposed punishment is not just balanced, but also
whether it considers all of the effects (privacy, security, social, economic,
etc.)
on subscribers and relying parties.

[Ben Wilson]

[Jakob Bohm]

On 09/11/2018 15:52, Hanno Böck wrote:
> On Fri, 9 Nov 2018 14:56:41 +0100
> Jakob Bohm via dev-security-policy
> <dev-secur...@lists.mozilla.org> wrote:
>
>> However there are also some very harsh punishments handed out, such as
>> distrusting some CAs (most notably happened to Symantec and WoSign,
>> but others are also teetering), and distrusting auditors (most notably
>> happened to the branch of Ernst & Young that audited the bad parts of
>> those two).
>>
>> A line of arguments often seen is that someone failed once to do
>> <something> completely right, therefore they cannot be trusted to do
>> anything similar to <something> right at all, therefore they should no
>> longer be trusted.
>
> I don't think anyone ever said something like that. Particularly
> I'm not aware of any recent incident where a CA failed *once* and got
> distrusted.
>

All 3 lines of reasoning I mentioned (with variations from case to case)
can be found in two of the most recent threads on this list.

> In the cases you mention - Symantec and Wosign - there were multiple
> issues. In both cases there was also plenty of opportunity for the
> affected CAs to explain and improve things before a distrust was
> even considered. It was repeated failures and a long list of issues
> that led to the distrust.
>

I am not saying those two didn't deserve it. I was just replying to a
claim that only mild punishments have been used. I also noted that some
other CAs are currently being removed or pressured to remove themselves
for various reasons.

However since the successful distrust of WoSign and Symantec, some here
seem to have gotten "the taste for blood" and are threatening the same
punishments for much smaller issues.

[Wayne Thayer]

I'm not convinced there is an answer here. It seems that most would agree
with the premise that we should consider the circumstances and context for
an issue and make a balanced assessment. That leaves the matter of what
this means in practice up for debate. Often, it appears to be a debate
between a strict application of the rules and the seemingly harmless effect
of the particular violation. Unfortunately the "no harm no foul" logic
leads to (takes us back to?) a situation where the rules are suggestions
and any CA can claim that any violation is okay, so long as they 'assessed
the risk'. So I view the 'balance' being between the 'moral hazard' of
excusing a violation and the belief that a violation is harmless in
practice.

As previously mentioned, a secondary consideration when assessing a
seemingly trivial violation is how it reflects on a CA's ability to follow
any rules. I agree that a single failure doesn't necessarily mean the CA is
incompetent, but when we see a pattern of "minor" violations occurring, we
have a problem. Another factor to consider is how 'new' a particular issue
is. The first CA to be called out for making a mistake is less culpable
than those who have ignored the warnings.

I also believe that violations in the context of a new inclusion request
should be treated more seriously because the costs to the ecosystem are
much lower if problems are addressed before a root is included.

- Wayne

On Fri, Nov 9, 2018 at 7:10 AM Jakob Bohm via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On 09/11/2018 15:52, Hanno Böck wrote:

> > On Fri, 9 Nov 2018 14:56:41 +0100
> > Jakob Bohm via dev-security-policy
> > <dev-secur...@lists.mozilla.org> wrote:
> >
> >> However there are also some very harsh punishments handed out, such as
> >> distrusting some CAs (most notably happened to Symantec and WoSign,
> >> but others are also teetering), and distrusting auditors (most notably
> >> happened to the branch of Ernst & Young that audited the bad parts of
> >> those two).
> >>
> >> A line of arguments often seen is that someone failed once to do
> >> <something> completely right, therefore they cannot be trusted to do
> >> anything similar to <something> right at all, therefore they should no
> >> longer be trusted.
> >
> > I don't think anyone ever said something like that. Particularly
> > I'm not aware of any recent incident where a CA failed *once* and got
> > distrusted.
> >
>

> All 3 lines of reasoning I mentioned (with variations from case to case)
> can be found in two of the most recent threads on this list.
>

> > In the cases you mention - Symantec and Wosign - there were multiple
> > issues. In both cases there was also plenty of opportunity for the
> > affected CAs to explain and improve things before a distrust was
> > even considered. It was repeated failures and a long list of issues
> > that led to the distrust.
> >
>

> I am not saying those two didn't deserve it. I was just replying to a
> claim that only mild punishments have been used. I also noted that some
> other CAs are currently being removed or pressured to remove themselves
> for various reasons.
>
> However since the successful distrust of WoSign and Symantec, some here
> seem to have gotten "the taste for blood" and are threatening the same
> punishments for much smaller issues.
>
>

> Enjoy
>
> Jakob
> --
> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
> Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
> This public discussion message is non-binding and may contain errors.
> WiseMo - Remote Service Management for PCs, Phones and Embedded

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>

[Eric Mill]

On Thu, Nov 8, 2018 at 8:51 PM Jakob Bohm via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Over the years, there has been some variation among participants in how
> harshly individual mistakes by CAs should be judged, ranging from "just
> file a satisfactory incident report, and all will be fine" to "Any tiny
> mistake could legally be construed as violating a formal requirement
> that would be much more catastrophic under other circumstances,
> therefore the maximum penalty of immediate distrust must be imposed".
>

This doesn't seem like an accurate description of the debates within the
Mozilla CA program, or this list, at all. I've never heard anyone make an
assertion that sounds like either extreme.

The long-term participants here, including those who press CAs hard, have
all responded very positively to a timely, detailed incident reports that
properly demonstrate an understanding and addressing of root cause.

There have definitely been quite a few CAs who have had incident reports
dragged out of them, or filed incident reports that addressed surface level
issues without any seeming acknowledgment of the gravity of the issue.

Where incidents with little _immediate_ security impact have occurred (such
as certain kinds of spec non-conformity), they have typically become major
issues not on the depth of perceived impact, but when there is a failure to
acknowledge that poor responses to small issues are highly predictive of
future large issues, or a long-term pattern that demonstrates this
empirically.

The major distrust events of the last few years have all been preceded by
robust discussion and demonstration of long-term issues, and months or
years of poor communication with the community.

In other words, no one has been tossed on a technicality, and I've never
seen any regular member of the community advocate for tossing someone
solely on a technicality.

> Furthermore, people with some clout tend to shut down all
> counterarguments when taking either extreme position, creating situation
> there only their own position is heard, making the entire "community"
> aspect an illusion.
>

This isn't my experience at all. Contributions from community members are
certainly distributed unevenly, but that seems to correspond most closely
to folks for whom participation here is part of their day job. That would
particularly be true for those who have spent years engaging in oversight
of a shifting array of CAs. And since the Mozilla CA Program itself is a CA
oversight program, those members have a very credible claim to represent
the community, even if others don't always have the time or mandate to
devote time to articulating the same arguments.

In general, I don't believe this post is well-grounded in fact, and
presents an inaccurate view of the Mozilla CA program's history. As a
result, I don't think it's likely to produce a constructive discussion.

-- Eric

--
konklone.com | @konklone <https://twitter.com/konklone>