Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Unbelievable!

1,370 views
Skip to first unread message

Eddy Nigg

unread,
Dec 22, 2008, 10:25:23 PM12/22/08
to
https://blog.startcom.org/?p=145

--
Regards

Signer: Eddy Nigg, StartCom Ltd.
Jabber: star...@startcom.org
Blog: https://blog.startcom.org

Robin Alden

unread,
Dec 23, 2008, 1:15:35 AM12/23/08
to mozilla's crypto code discussion list
Eddy,
That reseller's ability to sell Comodo certificates has been
suspended while we investigate why they are apparently not fulfilling their
contractual obligations to us.
We revoked your certificate for mozilla.com.

Regards
Robin Alden
Comodo

> _______________________________________________
> dev-tech-crypto mailing list
> dev-tec...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-crypto

Eddy Nigg

unread,
Dec 23, 2008, 1:33:46 AM12/23/08
to
On 12/23/2008 03:15 AM, Robin Alden:

> Eddy,
> That reseller's ability to sell Comodo certificates has been
> suspended while we investigate why they are apparently not fulfilling their
> contractual obligations to us.

How can you outsource such a critical part as domain control validation
to a reseller is a complete mystery to me! Your controls (if any) have
completly failed.

And apparently if the fish stinks at the tail, chances that the head is
rotten too are pretty high.

> We revoked your certificate for mozilla.com.
>

I suggest to revoke ALL certificates from this reseller and perform an
urgent review of your policies and implementations in relation to
resellers at all. Other steps might be needed as well, you know better
than me.

Robin Alden

unread,
Dec 23, 2008, 1:38:14 AM12/23/08
to mozilla's crypto code discussion list
Eddy,

As I noted in my prior correspondence, Comodo has undertaken an internal
review of the Certstar reseller account. We have informed CertStar that
their email violates their contractual obligation to refrain from sending
unsolicited emails and that their email could be interpreted as misleading
and confusing to the customer. During our review, we discovered that
Certstar had apparently issued a certificate to mozilla.com without
validating control of the domain. We immediately revoked the certificate
(prior to your posting) and have suspended Certstar's reseller activities
until our investigation has been completed. Please let me know if you have
any further problems.

Regards
Robin Alden
Comodo

> -----Original Message-----
> From: dev-tech-crypto-bounces+robin=comod...@lists.mozilla.org
> [mailto:dev-tech-crypto-bounces+robin=comod...@lists.mozilla.org] On
> Behalf Of Eddy Nigg

Eddy Nigg

unread,
Dec 23, 2008, 1:47:16 AM12/23/08
to
On 12/23/2008 03:38 AM, Robin Alden:

> Eddy,
>
> As I noted in my prior correspondence, Comodo has undertaken an internal
> review of the Certstar reseller account. We have informed CertStar that
> their email violates their contractual obligation to refrain from sending
> unsolicited emails and that their email could be interpreted as misleading
> and confusing to the customer. During our review, we discovered that
> Certstar had apparently issued a certificate to mozilla.com without
> validating control of the domain. We immediately revoked the certificate
> (prior to your posting) and have suspended Certstar's reseller activities
> until our investigation has been completed.

A.) The certificate was revoked after posting to this list.
B.) Your CA also issued a certificate to startcom.org without validating
anything.
C.) Yes, I think we have a problem of a wider scale here. This is not
about me personally.

Kyle Hamilton

unread,
Dec 23, 2008, 1:53:07 AM12/23/08
to mozilla's crypto code discussion list
I believe that this is a symptom of faulty internal control, and that
this is direct evidence of not living up to the Mozilla CA policy.

I advocate at least temporarily removing the trust bits from Comodo
until a new external audit can be completed, with an eye toward
ensuring that Comodo, not the reseller, perform the domain
validations.

-Kyle H

Frank Hecker

unread,
Dec 23, 2008, 5:09:50 AM12/23/08
to
Kyle Hamilton wrote:
> I advocate at least temporarily removing the trust bits from Comodo
> until a new external audit can be completed, with an eye toward
> ensuring that Comodo, not the reseller, perform the domain
> validations.

There are two general reasons for pulling a root, to address a clear and
present danger to Mozilla users, and to punish a CA and deter others. My
concern right now is with the former. I see at least three issues in
relation to that:

1. Issuance of further non-validated certs by this reseller. Comodo
seems to have addressed this by suspending the reseller's ability to get
certs issued. (I can testify that this is the case, as I tried to
duplicate Eddy's feat earlier today and got my uploaded CSR rejected.)

2. Potential problems with certs already sold through this reseller.
Comodo should investigate this and take action if needed. (This need not
necessarily require revoking all certificates associated with the
reseller; for example, the existing certs and their associated domains
could be re-validated, the registered domain owners could be notified of
the potential for bogus certs floating around, etc.)

3. Potential problems with other Comodo resellers. I'm not going to tell
Comodo how to operate its reseller network, but they certainly should
take a look at whether and where this might be a problem with other
resellers, and how they could revamp their systems to reduce potential
problems with resellers.

Pulling a Comodo root will knock out Firefox, etc., access to thousands
of SSL sites, maybe tens of thousands. Given the disruption that would
cause, the final decision on this IMO should be made in conjunction with
the Firefox security folks. From my point of view I'd wait on more
information regarding items 2 and 3 above before making a recommendation.

Frank

--
Frank Hecker
hec...@mozillafoundation.org

Kyle Hamilton

unread,
Dec 23, 2008, 7:09:35 AM12/23/08
to mozilla's crypto code discussion list, mozilla-...@mozilla.org
I agree with all three of these issues -- it's not *just* this
reseller, it's potentially the entire reseller network, and it's
entirely possible that the problem exists across multiples.
(Especially if Comodo delegates full Registration Authority capability
without verification, which seems to be the case -- though they could
have simply issued a sub-CA certificate.)

It occurs to me that there is no facility in Firefox or other Mozilla
products to provide an explanatory dialog that there's an issue, and
such a facility would be extremely useful at this point. Being able
to print a message to the user like "The Mozilla Foundation has
identified issues with the trusted root that issued this certificate
which prevent Firefox from being able to guarantee that this is truly
the site to which you intended to go. While it is unlikely that this
is a widespread problem, and an attack would rely on more technical
intrusions into the network, the nature of these issues requires that
you be warned of this circumstance so that you can exercise
appropriate levels of caution. The Mozilla Foundation is working with
the trusted root to resolve these issues." would help a lot.

(I word it like that because in order for an attacker to succeed he
would need to also hijack DNS, or place a entry in the user's hosts
file.)

Of course, this would be an NSS change (the addition of a 'trust
suspended' bit, in addition to the trust flags), a Firefox/Thunderbird
code change (to look for that bit), and a chrome change (to explain
what's going on). Or even a check for the name in the hosts file to
see if it's overridden from DNS. Placing an entry in a user's hosts
file is easier than hijacking DNS, or at least it's supposed to be.
I'm pretty sure we're all aware of the fairly recent cache poisoning
attack, but the people who write BIND pushed out a change to protect
against it very rapidly.

The addition of a 'trust suspended' bit is primarily unlikely to
happen, though, because there's too much inertia in the way things are
to do what needs to be done to fix the authentication system to allow
a root to stay in while the user is potentially at risk.

-Kyle H

patr...@certstar.com

unread,
Dec 23, 2008, 8:48:12 AM12/23/08
to
Hi all,

A glitch in our validation system has today caused a certificate to be
issued to a person who successfully abused our system.

We have now strengthened our domain validation system so that such
abuse cannot happen again. Comodo has handled this issue in a
professional way by invoking the certificate immediately after issuing
and contacting Certstar.

Again, I cannot stress enough how seriously we take this issue and I
would like to apologize to the Mozilla organization for the mis-issue.

--
kind regards,
Patricia, Certstar ApS

Thorsten Becker

unread,
Dec 23, 2008, 8:54:07 AM12/23/08
to
Hi Patricia,

patr...@certstar.com schrieb:

> We have now strengthened our domain validation system so that such
> abuse cannot happen again.

just curious: How do you normally validate domain ownership?

TIA,

Thorsten

Eddy Nigg

unread,
Dec 23, 2008, 9:42:49 AM12/23/08
to
On 12/23/2008 10:48 AM, patr...@certstar.com:

> Hi all,
>
> A glitch in our validation system has today caused a certificate to be
> issued to a person who successfully abused our system.

It's not me who abused your system, it's your company which sent out
illegal, misleading emails to our customers. Since I couldn't find out
who stands behind your site I had to get a certificate. There was no
abuse, there was only use-and-get.

- No affiliation published, the fact that you are a reseller for Comodo
is nowhere noted.
- No subscriber agreement or notice presented to subscribers.
- No control validation of the domain name. Just pay-and-go.

I didn't had to do anything to overcome any validation whatsoever. There
was no abuse!

> We have now strengthened our domain validation system so that such
> abuse cannot happen again. Comodo has handled this issue in a
> professional way by invoking the certificate immediately after issuing
> and contacting Certstar.

Yes, after I posted my article here at the mailing list.

> Again, I cannot stress enough how seriously we take this issue and I
> would like to apologize to the Mozilla organization for the mis-issue.
>


--

Eddy Nigg

unread,
Dec 23, 2008, 10:03:18 AM12/23/08
to
For those interested, Frank opened a bug to investigate this incident:

https://bugzilla.mozilla.org/show_bug.cgi?id=470897

Eddy Nigg

unread,
Dec 23, 2008, 10:39:03 AM12/23/08
to
On 12/23/2008 07:09 AM, Frank Hecker:

> There are two general reasons for pulling a root, to address a clear and
> present danger to Mozilla users, and to punish a CA and deter others. My
> concern right now is with the former. I see at least three issues in
> relation to that:
>
> 1. Issuance of further non-validated certs by this reseller. Comodo
> seems to have addressed this by suspending the reseller's ability to get
> certs issued. (I can testify that this is the case, as I tried to
> duplicate Eddy's feat earlier today and got my uploaded CSR rejected.)

As long as this site keeps operating, our customers are still being let
to believe that they have to renew their certificate with them. This is
only a reminder about how it started at all. CAs and their customers are
still taking damage from the previously sent messages.

>
> 2. Potential problems with certs already sold through this reseller.
> Comodo should investigate this and take action if needed. (This need not
> necessarily require revoking all certificates associated with the
> reseller; for example, the existing certs and their associated domains
> could be re-validated, the registered domain owners could be notified of
> the potential for bogus certs floating around, etc.)

You shouldn't notify the subscribers or domain name owners, but the
relying parties. How to do that is up to you and Comodo I guess.

Comodo not only shouldn't just investigate and take action, Comodo needs
to report publicly about their findings and full report about the
actions taken. This isn't a suggestion of resolution about this
incident, it's the transparency I expect from them at this hour.

> Pulling a Comodo root will knock out Firefox, etc., access to thousands
> of SSL sites, maybe tens of thousands.

I'm not advocating removing their root, however we must assess the risk
which is potentially caused to the relying parties. There may be
thousands of sites which received certificates without validating them.

> Given the disruption that would
> cause, the final decision on this IMO should be made in conjunction with
> the Firefox security folks.

Disabling the trust bits of "AddTrust External CA Root" could be a
temporary measure to prevent damage to relying parties until Mozilla
receives full report and disclosure from Comodo about its resellers and
conclusion of their investigation.

Additionally instead of just yanking a root as a deterrent and
punishment, as you mentioned above, I'd prefer to receive a commitment
from Comodo to address other issues noted during the last discussion -
mainly those listed in the "Problematic Practices" document. Considering
that OCSP in Firefox is set to soft-fail, an issue with their OCSP
server could still circumvent bogus sites for potentially the next five
years. A full review of their current status in NSS and their practices
might be advocated too.

Eddy Nigg

unread,
Dec 23, 2008, 11:36:47 AM12/23/08
to
On 12/23/2008 09:09 AM, Kyle Hamilton:

> (I word it like that because in order for an attacker to succeed he
> would need to also hijack DNS, or place a entry in the user's hosts
> file.)

Or be a WiFi operator. This was the attack vector of
https://bugzilla.mozilla.org/show_bug.cgi?id=460374

>
> Of course, this would be an NSS change (the addition of a 'trust
> suspended' bit,

I think this to be an interesting idea and should be considered.

Frank Hecker

unread,
Dec 23, 2008, 1:05:35 PM12/23/08
to
Eddy Nigg wrote:
> For those interested, Frank opened a bug to investigate this incident:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=470897

Actually Nelson opened this bug.

Kyle Hamilton

unread,
Dec 23, 2008, 1:09:42 PM12/23/08
to mozilla's crypto code discussion list
Patricia, I believe it's important to realize a couple of things:

1) An unsolicited commercial email (UCE) message was sent from your
company to the party in question suggesting that there already existed
a relationship between your company and the party in question. This
is obvious from the verb 'renew' in the original message -- a
non-party to the original certificate can't renew "on behalf of" the
original certificate issuer. If they can, there's a major problem.

1a) The message from your company, and in fact the entire process up
to and including paying for the certificate which your company issued,
did not expressly claim an affiliation, and the party in question went
through the process of "renewal" with the intent of figuring out which
CA's services were being advertised via UCE.

2) The party in question (Eddy Nigg) is the founder of another
Certifying Authority, with a root which participates in the Mozilla
root program and which is also included in the root store.

3) It falls within the concept of "due diligence" for a
security-conscious warden of a public trust to recognize the lack of
actual domain control verification in a trusted certificate issuance
process, and *for the purposes of verification and public
notification* obtain a certificate which could be shown absolutely to
have been issued in violation of the standards of at least one of the
root programs that the issuer (or issuer's trust-delegator)
participates in.

3a) If this had not been identified, someone else would have found it
eventually -- and the reaction to a gross violation of security
standards is to immediately believe that the worst-case scenario has
occurred: that it had already been found and exploited by at least one
other person.

4) The end-users (referred to as "relying parties") are well-served by
this identification of completely bogus credentialing.

5) It's also important to recognize the following (the first comment
on https://bugzilla.mozilla.org/show_bug.cgi?id=470897):

------- Comment #1 From Reed Loden [:reed] 2008-12-23 01:35:37
PST -------

The same company that Eddy was able to get the mozilla.com cert from (Certstar)
has been endlessly spamming webm...@mozilla.org since the beginning of
December complaining that one of our SSL certs had "expired" and needed to be
"renewed" (both of which were false). They have continued to spam us almost
daily. :(

------ end comment ------

Yes, the bare facts are that a user exploited the lack of verification
in your certificate issuance process. However, the lack of
verification in your process violates at least one contractual
obligation that your company is required to uphold, and reduces the
value of both Comodo's root AND the commercial Certifying Authority
structure in general. Because the user founded another commercial CA
which is trusted by Mozilla NSS by-default, this user was operating in
a manner to verify that the commercial CA structure was in fact
secure; the trust afforded this user's CA is demonstrably harmed by
your lack of verification.

I do not take this situation as lightly as you appear to wish that
everyone would. The initial fault is YOUR COMPANY'S (and that fault
reverbrates up to Comodo, since it delegated trust to your company),
and the fact that you're attempting to shift the blame onto the user
shows that you are absolutely untrustworthy to run or be the public
face of any commercial certificate-issuance service.

Because of this, my recommendation that Comodo's trust bits be removed
until a full audit of their practices (and a full audit of all issued
certificates) stands, and I am that much more resolute in my belief.
It may be that the integrity of Comodo's root has been irreparably
damaged by your company's malfeasance; if this is the case, I
certainly hope they rake you over the coals.

-Kyle H

Frank Hecker

unread,
Dec 23, 2008, 1:14:42 PM12/23/08
to
Eddy Nigg wrote:
> Disabling the trust bits of "AddTrust External CA Root" could be a
> temporary measure to prevent damage to relying parties until Mozilla
> receives full report and disclosure from Comodo about its resellers and
> conclusion of their investigation.

Do you mean the UTN-UserFirst-Hardware root? According to the screenshot
on your blog post, that's the root the bogus cert chains up to. Also, if
we were to take action of this general sort (as a hypothetical), what
about adding the PositiveSSL CA cert to NSS with the SSL trust bit
disabled; wouldn't that accomplish the same purpose, without interfering
with other parts of the hierarchy under the UTN-UserFirst-Hardware root?
(I seem to recall we've discussed this sort of thing in the past.)

Also note that any "suspension" of a root would last at last 1-3 months,
since that the typical interval between security updates for Firefox and
other Mozilla-based products.

Eddy Nigg

unread,
Dec 23, 2008, 1:15:56 PM12/23/08
to
On 12/23/2008 03:05 PM, Frank Hecker:

> Eddy Nigg wrote:
>> For those interested, Frank opened a bug to investigate this incident:
>>
>> https://bugzilla.mozilla.org/show_bug.cgi?id=470897
>
> Actually Nelson opened this bug.
>

Thanks for that. More into this story...

...all our employees coming the our network are able to retrieve the
certificates and all the rest from the "control panel" of this site.
Again, no questions asked. Apparently the only things one needs is to
fake our gateway IP address to get there.

It's getting weirder with the second. If they don't shut that site, we
can perhaps just publish the private key for the mozilla.com certificate
as well so everybody can enjoy it.

Kyle Hamilton

unread,
Dec 23, 2008, 1:17:14 PM12/23/08
to mozilla's crypto code discussion list
Either way, it's enough of a problem that a bug needs to be opened,
and it needs to be followed upon sooner than later.

(My opinion, of course. Comodo's gotta be breathing a sigh of relief
that I don't work for the Mozilla Foundation, else I would have yanked
the trust bits first and figured it out later.)

This incident shows anew the lack of policy/procedure of what to do in
the case of a gross breach of the root agreement, though. The entire
point of certificates in the first place is "user security", not "user
convenience". (If this weren't the case, none of us would have any
desire to get certificates in the first place.) Your reaction seems
to be based on placing "user convenience" over "user security", and
(again, my opinion) I don't believe that this is appropriate at all.

-Kyle H

Ian G

unread,
Dec 23, 2008, 1:58:45 PM12/23/08
to mozilla's crypto code discussion list
In the past, lots of good stuff has been done that handles the ascension
to the root list of Mozilla. c.f. the policy. But not so much is
written about *what happens afterwards*. This recent thread has been
such a case, and has afforded an opportunity to make some notes on what
might be suitable as a practices page on the wiki.

The reason for doing this, and adding yet more boring verbage to the
mass of words, is that it can prevent uneccessary legal costs by
providing clear guidepaths, and clear signals of seriousness.

Here's what I see:

1. How to file a dispute against Mozilla and/or a CA. This seems
fairly easy; file a bug in bugzilla and mark it as already described
here: https://wiki.mozilla.org/CA:How_to_apply . (With mods: Severity:
as appropriate.)

2. What is the practice on revocation or un-trusting on roots? This is
is perhaps a headline case of a dispute, so possible merits its own
notes. Frank has suggested the test:

a. there is a clear and present danger to Mozilla users, or
b. to punish a CA, and/or to deter others.

I would suggest that (b) be modifed slightly to give it a basis, which
in this case might be "for breaches of policy or practices". The
Policy, pt 4, gives the authority, as well as some examples, and adds
another test case:

c. where certificates cause technical problems with mozo software.

3. How to resolve a dispute. This is a Mozilla action &
responsibility. Reverse-engineering and referring, I would suggest this
as a teaser:

a. The CA certificate "module owner" at Mozilla foundation is
responsible. Ref, the policy, pt 15.
b. The dispute is investigated and ruled on by module owner.
c. The ruling is listed in the bug report above.
d. Many disputes will be dealt with by communication, and no ruling
will be required. This will create a default "closed, no action" ruling.

4. Finality. What happens if we disagree with the decision of the
module owner? In the policy, it says "CAs or others objecting to a
particular decision may appeal to mozilla.org staff, who will make a
final decision." Ref, policy, pt 15.

I would wonder about this; google suggests that "staff" is as listed here:
http://www.mozilla.org/about/staff
but that seems out of date. Also, due to the absence of this forum in
the public eye, I doubt it musters the credibility we need in dispute
review where the legal and contractual significance is high. E.g., is
there any way we can review the decisions they made in the past?

There are several possibilities:

(i) Ruling is final.
(ii) Mozilla.org staff, policy, pt 15.
(iii) Review by board of Mozilla Foundation.
(iv) Review by some independent party.
(v) Review by forum at law: courts, or Arbitrator.

Personally, I would plumb for (iii) and suggest the Mozo Foundation
board as the next step. It is expensive, but available. The directors
already have fiduciary responsibility, and can thus deal with the
significance. It is also aligned with the review of the manager
concerned, the policy and the general contractual issues.

End! I'd encourage others to comment!

If nobody has any objections, I can add that as a page into the wiki as
an informal document of practices, including or without those questions.


iang

Frank Hecker

unread,
Dec 23, 2008, 2:35:00 PM12/23/08
to
Kyle Hamilton wrote:
> Your reaction seems
> to be based on placing "user convenience" over "user security", and
> (again, my opinion) I don't believe that this is appropriate at all.

My intent is to balance the disruption that would be caused by pulling a
root vs. the actual security threat to users. Right now we have no real
idea as to the extent of the problem (e.g., how many certs might have
been issued without proper validation, how many of those were issued to
malicious actors, etc.).

Frank Hecker

unread,
Dec 23, 2008, 2:48:18 PM12/23/08
to
Ian G wrote:
> In the past, lots of good stuff has been done that handles the ascension
> to the root list of Mozilla. c.f. the policy. But not so much is
> written about *what happens afterwards*.

I don't have time right now to address your whole post, but I did want
to comment on one point.

> 4. Finality. What happens if we disagree with the decision of the
> module owner? In the policy, it says "CAs or others objecting to a
> particular decision may appeal to mozilla.org staff, who will make a
> final decision." Ref, policy, pt 15.
>
> I would wonder about this; google suggests that "staff" is as listed here:
> http://www.mozilla.org/about/staff
> but that seems out of date.

There is no longer a "mozilla.org staff" as traditionally constituted.
From a Mozilla community point of view the closest present-day
equivalent is the system described at

https://wiki.mozilla.org/Module_Owners_Activities_Modules

Basically there is a group of people that oversee the Mozilla module
ownership system and the module owners within it. That group in turn
operates under general policy structures set by Mitchell Baker and
Brendan Eich, and as you note the ultimate legal authority resides in
the board of the Mozilla Foundation (of which Mitchell and Brendan are
members).

For more background on this see

http://www.mozilla.org/hacking/module-ownership
http://blog.lizardwrangler.com/2008/05/30/governance-and-module-ownership/

Gervase Markham

unread,
Dec 23, 2008, 3:42:39 PM12/23/08
to Nelson B Bolyard
Frank Hecker wrote:
> Do you mean the UTN-UserFirst-Hardware root? According to the screenshot
> on your blog post, that's the root the bogus cert chains up to. Also, if
> we were to take action of this general sort (as a hypothetical), what
> about adding the PositiveSSL CA cert to NSS with the SSL trust bit
> disabled; wouldn't that accomplish the same purpose, without interfering
> with other parts of the hierarchy under the UTN-UserFirst-Hardware root?
> (I seem to recall we've discussed this sort of thing in the past.)

This was the first thing that occurred to me as well. Can anyone
(Nelson?) tell us definitively if this is possible? I would assume it
is, but it would be good to get confirmation.

Gerv

Gen Kanai

unread,
Dec 23, 2008, 6:17:18 PM12/23/08
to mozilla's crypto code discussion list
Are we going to receive information from Comodo regarding how many
other Comodo resellers may be in a similar position to Certstar?

Are we going to receive information from Certstar as to how many other
certs may have been issued in error?

How do we verify the claims from Comodo or Certstar?

Gen

Kyle Hamilton

unread,
Dec 23, 2008, 6:30:18 PM12/23/08
to mozilla's crypto code discussion list
These are, of course, the reason why I want the trust bits removed.
It's going to take more than a month to get this all straightened out,
and that's "over a month" that we have no idea if Comodo certificates
can be trusted.

-Kyle H

Frank Hecker

unread,
Dec 23, 2008, 6:43:30 PM12/23/08
to
Gen Kanai wrote:
> Are we going to receive information from Comodo regarding how many other
> Comodo resellers may be in a similar position to Certstar?
>
> Are we going to receive information from Certstar as to how many other
> certs may have been issued in error?

I've asked Robin Alden of Comodo to make an accounting regarding these
two issues. I don't expect to see that immediately (i.e., in the next
day or two), though I also don't expect to wait a month for it (as Kyle
is concerned about).

> How do we verify the claims from Comodo or Certstar?

Some things by their nature we can directly verify, and some we can't.
For example, we can certainly verify how Certstar is operating currently
(as Eddy and I did), and if certs get revoked we can verify that by
downloading the CRLs that get issued.

Hendrik Weimer

unread,
Dec 23, 2008, 7:15:25 PM12/23/08
to mozilla's crypto code discussion list
Frank Hecker <hec...@mozillafoundation.org> writes:

> My intent is to balance the disruption that would be caused by pulling
> a root vs. the actual security threat to users. Right now we have no
> real idea as to the extent of the problem (e.g., how many certs might
> have been issued without proper validation, how many of those were
> issued to malicious actors, etc.).

Isn't that, by itself, a very good reason to take immediate action?
Security should be default-fail rather than default-pass.

Hendrik

Kyle Hamilton

unread,
Dec 23, 2008, 7:23:16 PM12/23/08
to mozilla's crypto code discussion list
On Tue, Dec 23, 2008 at 10:43 AM, Frank Hecker
<hec...@mozillafoundation.org> wrote:
> I've asked Robin Alden of Comodo to make an accounting regarding these two
> issues. I don't expect to see that immediately (i.e., in the next day or
> two), though I also don't expect to wait a month for it (as Kyle is
> concerned about).

Actually, I think it's very important that the accounting include this:

for each name (not just certificate, but name in
subjectAlternativeNames) that has been certified, a connection to the
TLS ports should be made, and the certificate presented by the site
compared against the certificate that Comodo issued. This obviously
won't be a complete verification, but it should give a start to see
how widespread the problem is.

A script to do this could probably be written fairly easily, but
depending on the number of certificates Comodo has issued that are
currently valid (and I'd like to see some hard numbers on that, as
well) it could take a while to run.

From the script, the numbers I'd like to see are: the number of
unreachable/not-answering names/hosts, the number of matching
certificates, and the number of mismatched certificates. From that
output plus Comodo's records, I would also like to see how many
resellers there are and how many of them have sold mismatched
certificates.

I hate to say this, but this IS The Worst-Case Scenario. A CA has
gone rogue and issued certificates that violate its standards, and the
standards of the root programs that it's a part of -- it is true that
Comodo didn't /intend/ to go rogue, but it has, and we can't afford to
let it damage the greater PKI. Since every CA in the root store is
treated the same, there is no differentiation between them -- and this
means that Verisign and Comodo and Thawte and *every* CA share the
same reputation. If one goes rogue, it's exactly the same as if all
of them have gone rogue, in the eye of the end-user.

To put it another way, it's exactly the same problem as putting the
public keys of webservers in DNSSEC. Since the end-system can't know
if a response came from DNSSEC or just unauthenticated DNS, the
end-system can't trust anything (including RFC2538-style CERT records)
that comes from DNS.

THIS is why I want to see greater differentiation in the browser
chrome between CAs, so that one bad apple doesn't spoil the whole root
barrel. THIS is why the argument against changing the chrome (user
convenience) fails.

-Kyle H

Kyle Hamilton

unread,
Dec 23, 2008, 7:27:53 PM12/23/08
to mozilla's crypto code discussion list
I'd rather deal with disruption caused thereby (and, yes, the user
complaints generated thereby -- at least then the end-user would KNOW
that there's a problem that's being dealt with rather than having a
FALSE SENSE OF SECURITY) than let those potential security breaches
continue to wreak their quiet little havoc.

-Kyle H

On Tue, Dec 23, 2008 at 11:15 AM, Hendrik Weimer <hen...@enyo.de> wrote:
> Frank Hecker <hec...@mozillafoundation.org> writes:
>
>> My intent is to balance the disruption that would be caused by pulling
>> a root vs. the actual security threat to users. Right now we have no
>> real idea as to the extent of the problem (e.g., how many certs might
>> have been issued without proper validation, how many of those were
>> issued to malicious actors, etc.).
>

> Isn't that, by itself, a very good reason to take immediate action?
> Security should be default-fail rather than default-pass.
>
> Hendrik

Eddy Nigg

unread,
Dec 23, 2008, 7:49:20 PM12/23/08
to
On 12/23/2008 09:15 PM, Hendrik Weimer:

It should be. I realized that there are more points which will have to
be addressed in due time at Mozilla, which however can wait for now.

Concerning the disruption, Comodo has many roots and the resetting of
this specific root would affect low-assurance sites as far as I know.
The higher validated sites would not be affected. Having said that, the
roots of their low-assurance products have been previously of concern it
must be looked at in the broader context of Comodo's operations. It's
not surprising in itself.

Justin Dolske

unread,
Dec 23, 2008, 8:12:27 PM12/23/08
to
On 12/23/08 11:27 AM, Kyle Hamilton wrote:
> I'd rather deal with disruption caused thereby (and, yes, the user
> complaints generated thereby -- at least then the end-user would KNOW
> that there's a problem that's being dealt with rather than having a
> FALSE SENSE OF SECURITY)

Hmm, would they?

Justin Dolske

unread,
Dec 23, 2008, 8:20:15 PM12/23/08
to

*sigh* I somehow managed to click "Send" mid-message.

Anyway, the rest of my incomplete thought:

...I think there's some risk that if a Firefox update suddenly breaks a
large swath of legitimate SSL sites, that could end up training users to
ignore the problem. I'm not even sure how you'd present this condition
to Joe The User in a comprehensible way.

That said, the Comodo/Certstar is hugely sucky and I would hope there's
something we can do about it that helps users.

Justin

Daniel Veditz

unread,
Dec 23, 2008, 8:23:27 PM12/23/08
to
Frank Hecker wrote:
> Eddy Nigg wrote:
>> Disabling the trust bits of "AddTrust External CA Root" could be a
>> temporary measure to prevent damage to relying parties
>
> Also note that any "suspension" of a root would last at last 1-3 months,
> since that the typical interval between security updates for Firefox and
> other Mozilla-based products.

And we don't have a magic switch we can flip in the office. We'd have to
make the change, test the change, make the builds, ship the builds,
users would have to update (about a week from ship until most users have
the update).

If the sole purpose of the update was to break lots of sites (from the
user's POV) then some number of them disable updates, making them less
secure in the future.

If Comodo is acting in good faith then anything they can do would be
lightyears faster than a client update. If they're not fulfilling their
responsibilities then a permanent removal would make sense, but given
the time scales it's hard to see how a "temporary" month-or-so removal
helps.

Maybe we need to build in something like a CRL that pings back to
Mozilla that would let us revoke roots without having to ship a client
update.

Eddy Nigg

unread,
Dec 23, 2008, 8:43:17 PM12/23/08
to
On 12/23/2008 10:23 PM, Daniel Veditz:

>
> Maybe we need to build in something like a CRL that pings back to
> Mozilla that would let us revoke roots without having to ship a client
> update.

Of course we (@ mozilla) also take our lessons from this event, I'm
sure. Indeed it was previously suggested here and I think we'll have to
look at such options in due time.

Anything other than taking down their site is insufficient (as an
immediate action to prevent further damage), specially in light of
apparently more flaws in their system. Not having done that as I
requested already two days ago isn't exactly comforting. Doesn't show
really cooperation and good faith here...

...they'd do that better now before any more damage happens. I've warned
them and requested to shut them down. My warning was ignored, their bad...

dou...@theros.info

unread,
Dec 23, 2008, 8:44:13 PM12/23/08
to

I, for example, have a ssl cert from comodo reseller, and they DO have
made all the validation steps.

My site, a legitimate one, would be in trouble with this. Are you all
sure that it is a good measure to just knock off the root cert or
security bit?

please, think twice

Christoph Moormann

unread,
Dec 23, 2008, 9:08:08 PM12/23/08
to

I puzzled by the security approach on this issue. There was a severe
security problem, and at least two, maybe many certs are compromised,
who knows by this point.
Do i still have trust in the Certs of Comodo or their signing
resellers? No, until the whole issue is cleared up completely.
Now it's the users duty to check every Cert on the issuer to evaluate
the trustlevel, while his browser states, everything is fine?
This is not about punishing Comodo, their resellers or their
customers, this is about trusting my browser software.

Ian G

unread,
Dec 23, 2008, 9:12:21 PM12/23/08
to mozilla's crypto code discussion list
On 23/12/08 20:23, Kyle Hamilton wrote:
> On Tue, Dec 23, 2008 at 10:43 AM, Frank Hecker
> <hec...@mozillafoundation.org> wrote:
>> I've asked Robin Alden of Comodo to make an accounting regarding these two
>> issues. I don't expect to see that immediately (i.e., in the next day or
>> two), though I also don't expect to wait a month for it (as Kyle is
>> concerned about).
>
> Actually, I think it's very important that the accounting include this:
>
> for each name (not just certificate, but name in
> subjectAlternativeNames) that has been certified, a connection to the
> TLS ports should be made, and the certificate presented by the site
> compared against the certificate that Comodo issued. This obviously
> won't be a complete verification, but it should give a start to see
> how widespread the problem is.


Um. Some questions... On what basis are we asking for this "accounting" ?

Is there a criteria that calls for public breach accounting? Perhaps
Mozilla needs to add it? Most of the criteria call for revocation when
a false cert has been issued, and that seems to have been met right now.

Also, some of the process might instead rest with Comodo's external
auditor or its internal audit process or its internal security process.
Perhaps in the next audit, there will be an examination of the
remedial steps taken, to see if they conform to the security manual, etc.

There are some slippery slope aspects here. If the framework for such
"accounting" is outside Mozilla's area, then care might be needed.
There could be confusion in role and liabilities.

It's nice if Mozilla were minded to do all the audit and security
activities for all, but I'm not sure Frank has time ;) OTOH, if we feel
the need to usurp the CA's processes and the conventional audit
relationship with an "accounting" then perhaps we need to also simplify
some other processes?

Another aspect here is that the goal of any security or governance
process is not to eliminate or stop all breaches. The process is more
about how you keep dangers to a minimum, deal with the breaches that do
happen, and improve.


> I hate to say this, but this IS The Worst-Case Scenario.

Noooo.... With respect, I'd say worst case is that a root has been
breached, and the Carders have got hold of it and are MITMing the major
retail sites. Damages!

Earlier, Frank used the language of "clear and present danger."

* clear: we can measure the costs of it, and cost of defences.
* present: it is happening today, provably.
* danger: it can be shown capable of doing damage, at least in theory

Only the last one is shown. It is a danger in that anyone relying on a
cert wrongly issued could be harmed. But it hasn't actually caused
anyone damages, and nobody has shown it to exist.

This situation is about as harmful as the average exploit demo. In this
particular case it was caught by an insider rather than by an outsider.
However, beyond that, the situation is already sealed in that Comodo
have already taken the urgent triage steps to make sure nobody else gets
one.


> A CA has
> gone rogue and issued certificates that violate its standards, and the
> standards of the root programs that it's a part of -- it is true that
> Comodo didn't /intend/ to go rogue, but it has, and we can't afford to
> let it damage the greater PKI.

This is no different to when Verisign got schnikered into issuing a
couple of Microsoft certs. No damages ever resulted. The system worked
as it was expected to, it seems. Panic is not called for.


> Since every CA in the root store is
> treated the same, there is no differentiation between them -- and this
> means that Verisign and Comodo and Thawte and *every* CA share the
> same reputation. If one goes rogue, it's exactly the same as if all
> of them have gone rogue, in the eye of the end-user.

...

> THIS is why I want to see greater differentiation in the browser
> chrome between CAs, so that one bad apple doesn't spoil the whole root
> barrel. THIS is why the argument against changing the chrome (user
> convenience) fails.


Oh, on that, yes, strongly agree. There's no trust without brand :)

iang

Justin Dolske

unread,
Dec 23, 2008, 9:16:05 PM12/23/08
to
On 12/23/08 12:20 PM, Justin Dolske wrote:

> That said, the Comodo/Certstar is hugely sucky and I would hope there's
> something we can do about it that helps users.

I am just full of fail today: "... the Comodo/Comstar *incident* is
hugely sucky ..."

Justin

Eddy Nigg

unread,
Dec 23, 2008, 9:25:09 PM12/23/08
to
On 12/23/2008 11:12 PM, Ian G:

> Earlier, Frank used the language of "clear and present danger."
>
> * clear: we can measure the costs of it, and cost of defences.
> * present: it is happening today, provably.
> * danger: it can be shown capable of doing damage, at least in theory
>
> Only the last one is shown. It is a danger in that anyone relying on a
> cert wrongly issued could be harmed. But it hasn't actually caused
> anyone damages, and nobody has shown it to exist.

The first and second are valid as well. We can measure the costs, can't we?
The second item shows that it can be happening today, you don't have to
prove it, because I did already. The fact that I disclosed publicly
about it doesn't mean that there aren't scores of others out there.
Considering the aggressive and illegal mailing campaign they operated,
one can expect that there are many certificates issued in this way.

>
> This situation is about as harmful as the average exploit demo. In this
> particular case it was caught by an insider rather than by an outsider.
> However, beyond that, the situation is already sealed in that Comodo
> have already taken the urgent triage steps to make sure nobody else gets
> one.
>

Well, that's exactly the point! I haven't heard back from Robin anymore,
their site is still online as well. Clear statements about how many
certificates are affected (all of the ones issued through them) and
revocations (or at least urgent re-validation within 48 hours and
thereafter revocation) would be satisfactory maybe.

The handling of the situation is insufficient, response and measures are
a joke, communication doesn't exist. Apparently this resellers business
was too good for Comodo.

Paul Hoffman

unread,
Dec 23, 2008, 9:54:22 PM12/23/08
to mozilla's crypto code discussion list
At 11:27 AM -0800 12/23/08, Kyle Hamilton wrote:
>I'd rather deal with disruption caused thereby (and, yes, the user
>complaints generated thereby -- at least then the end-user would KNOW
>that there's a problem that's being dealt with rather than having a
>FALSE SENSE OF SECURITY) than let those potential security breaches
>continue to wreak their quiet little havoc.

And I would not.

Just because a few people loudly proclaim their preferences on either side, it does not mean that their preferences should be acted on in a way that affects millions of Firefox users.

Paul C. Bryan

unread,
Dec 23, 2008, 9:59:57 PM12/23/08
to
> Just because a few people loudly proclaim their preferences on either side, it does not mean that their preferences should be acted on in a way that affects millions of Firefox users.

It was Comodo that affected millions of Firefox users; it's up to
Mozilla to protect those users by failing safe.

Paul C. Bryan

unread,
Dec 23, 2008, 10:05:13 PM12/23/08
to
Presumably it was Comodo that underwent an audit to be added to
Mozilla's roots, and Comodo should not be allowed to delegate trust to
their resellers for domain validation. If, today, trust is delegated
to their resellers, then we can't trust Comodo, period.

Although disruptive, their trust bits should be suspended. The
explanation to users: "The CA purporting to provide assurance about
the site you are trying to visit cannot be trusted. Please contact the
site operator and advise them to find a trustworthy certification
authority."

Yes, perception is that Mozilla releases code expressly to "break"
access to legitimate sites, but this is because a trusted CA has gone
rogue. Users can still jump through hoops to expressly include the
site's certificate and keep going.

The trust model for browsers should be fail-safe, even if this
inconveniences users. Better that than me and countless others
inadvertently exposing my credentials to a site pretending to be my
bank, investment house, government revenue agency, etc.

If Mozilla doesn't pull the trust bits, what's it's accountability for
any breaches that occur due to keeping the bits? With assurance must
come liability, whether from the certification authority, or those who
are implicitly trusted with vetting them.

Kyle Hamilton

unread,
Dec 23, 2008, 10:20:05 PM12/23/08
to mozilla's crypto code discussion list
The only effective and appropriate response to a root that does not
have sufficient internal controls to maintain its own security is to
remove the trust in it. If you've purchased a certificate from them
because it's trusted, and then they lose that trust, I would think
that you should be clamoring for your money back and looking for an
alternate certificate issuer rather than trying to maintain the
problem.

-Kyle H

Frank Hecker

unread,
Dec 23, 2008, 10:20:52 PM12/23/08
to
Eddy Nigg wrote:
> Concerning the disruption, Comodo has many roots and the resetting of
> this specific root would affect low-assurance sites as far as I know.

I don't think that's necessarily true. I don't think it would affect EV
sites (because of the way validation for those sites is special-cased),
but it could affect non-EV sites under other Comodo brands (I mean,
other than the PositiveSSL brand) and it could also affect other CAs
whose CA certs are cross-signed by the UTN-UserFirst-Hardware root.

Eddy Nigg

unread,
Dec 23, 2008, 10:31:13 PM12/23/08
to
On 12/24/2008 12:20 AM, Frank Hecker:

How about ADDING the chained issuing CA certificate to NSS and mark it
deliberately untrusted (no bits turned on)?

But our dilemma here shows clearly that we must have tools to act to
such threats. It's probably one of the lessons to learn from this incident.

Eddy Nigg

unread,
Dec 23, 2008, 10:39:24 PM12/23/08
to
On 12/24/2008 12:05 AM, Paul C. Bryan:

> Presumably it was Comodo that underwent an audit to be added to
> Mozilla's roots, and Comodo should not be allowed to delegate trust to
> their resellers for domain validation. If, today, trust is delegated
> to their resellers, then we can't trust Comodo, period.
>

I would second that and in light of many other problematic practices
which were discovered during their inclusion/update of EV, it's simply
too much. More than 24 hours into this, I've come to the conclusion that
this is a sever incident which requires action. If Robin can assure us
of reasonable actions from their side (as suggested previously by me) it
would serve all participants the best. Inaction and non-cooperation will
leave Mozilla with not much choice I think. Ignorance by Mozilla itself
will hunt it for a long time too. But it must happen now, either way!

(I don't think we have the time to discuss each and every aspect of RA
and reseller responsibilities and what we deem as save, I'm certain
we'll take this issue up (which apparently has about the same
implications as intermediate externally operated CAs))

Kyle Hamilton

unread,
Dec 23, 2008, 10:51:21 PM12/23/08
to mozilla's crypto code discussion list
I believe that Startcom (and other certification authorites in
Mozilla's root program) would likely have cause to bring an action for
the tort of negligence against Mozilla. I feel that this is something
that Mozilla should likely ask its general counsel very quickly.

0) Comodo is plainly found to have derelicted its duty to uphold the
Mozilla CA agreement. As a result, damage to the trust in the PKI
occurs, causing people to look outside of the PKI for solutions to the
problems that encryption and authentication can solve.

1) As a result of this Startcom and other CAs in the PKI have a
reduced market for their products. (damage)

2) Mozilla has the authority and ability to remove CAs from its trust
list if and when they are found to be negligent in their duties. It,
as an organization which attempts to vet the CAs that it includes in
its PSM and which has policies which allow for it to take action
should the policies be breached, has accepted the duty to remain
diligent. (duty, and proximity of duty, since in order to get into
the program in the first place all CAs in the PKI must accede to
Mozilla's demands of trustworthiness.)

3) Mozilla fails to remain diligent, and fails to remove trust bits
upon notice of dereliction. (causation, which leads to the trust in
the PKI being eroded further.)

I am not a lawyer, this is not legal advice, etc. I'm trying to
prevent Mozilla from having problems.

-Kyle H

Where Wolf

unread,
Dec 23, 2008, 11:10:57 PM12/23/08
to
On Dec 23, 5:51 pm, "Kyle Hamilton" <aerow...@gmail.com> wrote:
> I believe that Startcom (and other certification authorites in
> Mozilla's root program) would likely have cause to bring an action for
> the tort of negligence against Mozilla.  I feel that this is something
> that Mozilla should likely ask its general counsel very quickly.
>
Apart from the legal and procedural ramifications of all this, can you
guys publish a manual procedure for those of us who want to remove
Comodo's certificates from our browser installations? Assuming one is
available?

Eddy Nigg

unread,
Dec 23, 2008, 11:16:43 PM12/23/08
to
On 12/24/2008 01:10 AM, Where Wolf:

Select Preferences -> Advanced -> View Certificates -> Authorities.
Search for AddTrust AB -> AddTrust External CA Root and click "Edit".
Remove all Flags.

This would remove the trust from the potentially affected sites and
their certificates. Comodo has many more roots if you are interested,
such as all AddTrust, Comodo CA Limited, The Usertrust Network.

Paul Hoffman

unread,
Dec 23, 2008, 11:23:39 PM12/23/08
to mozilla's crypto code discussion list
At 3:15 PM +0200 12/23/08, Eddy Nigg wrote:
>If they don't shut that site, we can perhaps just publish the private key for the mozilla.com certificate as well so everybody can enjoy it.

It is indeed unbelievable to hear the COO of a CA company making threats like this. I'm sure that making such threats is not covered by Mozilla's inclusion policies, but maybe it should be.

And, yes, I'm serious. Given that Startcom has the ability to issue bogus certificates like the kind that Eddy is threatening, I would think that a public statement like the above is relevant to Mozilla or Microsoft deciding whether or not the organization is trustworthy.

Paul Hoffman

unread,
Dec 23, 2008, 11:27:14 PM12/23/08
to mozilla's crypto code discussion list
At 2:51 PM -0800 12/23/08, Kyle Hamilton wrote:
>I believe that Startcom (and other certification authorites in
>Mozilla's root program) would likely have cause to bring an action for
>the tort of negligence against Mozilla. I feel that this is something
>that Mozilla should likely ask its general counsel very quickly.
>
>. . .

>
>I am not a lawyer, this is not legal advice, etc.

The above is most certainly legal advice.

> I'm trying to
>prevent Mozilla from having problems.

Some people might read it differently.

Where Wolf

unread,
Dec 23, 2008, 11:29:26 PM12/23/08
to

> Select Preferences -> Advanced -> View Certificates -> Authorities.
> Search for AddTrust AB -> AddTrust External CA Root and click "Edit".
> Remove all Flags.
>
> This would remove the trust from the potentially affected sites and
> their certificates. Comodo has many more roots if you are interested,
> such as all AddTrust, Comodo CA Limited, The Usertrust Network.
>
> --
> Regards
>
> Signer: Eddy Nigg, StartCom Ltd.
> Jabber: start...@startcom.org
> Blog:  https://blog.startcom.org

Thanks Eddy. I found the option screen shortly after I posted here,
but I was assuming that the affected certificates would be listed
under Comodo's branch, which apparently wouldn't have done me a bit of
good. Sometimes it pays to ask even when you think you know the answer
(or if you're a teacher or lawyer, ONLY ask if you already know the
answer).

BTW I tracked into this discussion from your blog, so thanks for
bringing this whole thing to my attention as well.

Eddy Nigg

unread,
Dec 23, 2008, 11:45:59 PM12/23/08
to
On 12/24/2008 01:23 AM, Paul Hoffman:

> At 3:15 PM +0200 12/23/08, Eddy Nigg wrote:
>> If they don't shut that site, we can perhaps just publish the private key for the mozilla.com certificate as well so everybody can enjoy it.
>

Paul, you are disappointing me! I have not heard one critical word from
you about this incident, instead you are criticize *me*? C'mon, give me
a break!

I reported that my employees can see the supposedly private control
panel of this reseller - what else is needed to get this site down?

> And, yes, I'm serious. Given that Startcom has the ability to issue bogus certificates like the kind that Eddy is threatening, I would think that a public statement like the above is relevant to Mozilla or Microsoft deciding whether or not the organization is trustworthy.

I don't need the services of Comodo for that, if I would have ever
wanted to that, I could do so long time ago. Everybody in his right mind
understands exactly what I meant to say in this post. If you didn't,
that's your problem.

--
Regards

Signer: Eddy Nigg, StartCom Ltd.

Jabber: star...@startcom.org
Blog: https://blog.startcom.org

patr...@certstar.com

unread,
Dec 23, 2008, 11:58:28 PM12/23/08
to
Dear all,

I just wanted to give you all an update from Certstar. As you all know
we failed to validate a certificate due to a flaw in our system which
is clearly unacceptable.

Having worked intensively with this case I can truly say that Comodo
is indeed taking their responsibility extremely seriously and taking
huge efforts to ensure that safety of their root. Currently we are
going through a number certificate to double ensure that no further
mis-issuances have occurred.

Personally I do feel that it would have been appropriate if we had
been contacted by StartCom Ltd when they found this flow so that it
could have been fixed faster. Being our competitor I am not sure if we
could expect this, but it would indeed have been generous move.

The technical verification procedure has been improved and is now on a
very high security level. Comodo will also review our implementation
to ensure that it comply with all standards and cannot be abused.
Again, I would like to apologize to the community. Nothing does
however indicate that the flow was abused by others – but still we
have made a huge mistake which I sincerely apologize for.


--
kind regards,
Patricia, Certstar ApS

patr...@certstar.com

unread,
Dec 24, 2008, 12:12:49 AM12/24/08
to
On Dec 24, 12:45 am, Eddy Nigg <eddy_n...@startcom.org> wrote:
> I reported that my employees can see the supposedly private control
> panel of this reseller - what else is needed to get this site down?

The certificate has been revoked for quite some time now and is not
available in your control panel.

You must be aware of this as you submitted a complaint to Paypal that
“the certificate purchased was revoked” (you did not get what you paid
for). One could think that the purpose was to get them to suspend our
account as similar complaints was made the day before.

I do not wish avoid engaging in a personal discussion. Instead I would
like to get back to how we can solve this and make sure it does
*never* happen again – for Certstar or other SSL providers.

Paul C. Bryan

unread,
Dec 24, 2008, 12:33:18 AM12/24/08
to
On Dec 23, 3:58 pm, patri...@certstar.com wrote:
> The technical verification procedure has been improved and is now on a
> very high security level. Comodo will also review our implementation
> to ensure that it comply with all standards and cannot be abused.

As far as I know, you're not the party who is responsible, nor the one
to mitigate this issue; that lies with Commodo, and ultimately
Mozilla. Comodo should never have delegated trust to anyone to a third
party to perform such validation. In so doing, they have undermined
trust that can be placed in them.

Christoph Moormann

unread,
Dec 24, 2008, 12:39:21 AM12/24/08
to
On Dec 24, 12:58 am, patri...@certstar.com wrote:

> I just wanted to give you all an update from Certstar. As you all know
> we failed to validate a certificate due to a flaw in our system which
> is clearly unacceptable.

IIIRC you failed to validate at least two certificates, coincidentally
the ones by the thread starter.

> Having worked intensively with this case I can truly say that Comodo
> is indeed taking their responsibility extremely seriously and taking
> huge efforts to ensure that safety of their root. Currently we are
> going through a number certificate to double ensure that no further
> mis-issuances have occurred.

I'm anxious for the results, and i would be suprised if this was an
exception.

>Comodo will also review our implementation
> to ensure that it comply with all standards and cannot be abused.

You call it abuse to check whether someone hands out trusted
unvalidated certificates?
Do you realise that half of your text tries to blame other people.
Go ahead, shoot the messenger.
*shaking his head in disbelief*


Ian G

unread,
Dec 24, 2008, 12:40:36 AM12/24/08
to mozilla's crypto code discussion list
On 23/12/08 22:25, Eddy Nigg wrote:
> On 12/23/2008 11:12 PM, Ian G:
>> Earlier, Frank used the language of "clear and present danger."
>>
>> * clear: we can measure the costs of it, and cost of defences.
>> * present: it is happening today, provably.
>> * danger: it can be shown capable of doing damage, at least in theory
>>
>> Only the last one is shown. It is a danger in that anyone relying on a
>> cert wrongly issued could be harmed. But it hasn't actually caused
>> anyone damages, and nobody has shown it to exist.
>
> The first and second are valid as well. We can measure the costs, can't we?


First point: Who lost money? What are the damages?

You don't count, or more precisely, the money you spent getting the cert
doesn't count; sorry about that :)


> The second item shows that it can be happening today, you don't have to
> prove it, because I did already.


Yes, it's an "exploit" not a "breach".

This is a breach, this proves certs are proved to use bad stuff:

http://blogs.technet.com/mmpc/archive/2008/11/06/malware-and-signed-code.aspx

Indeed, some stats to underscore *presence* ...

"In the first six months of 2008, the MMPC received reports of 22M
instances of distinct malware files, of which about 173,000 were
distinct malware files with code signatures. Of this malware with code
signatures, about 38,000 were not validly code signed, so approximately
*135,000 validly signed malware files* were reported to Microsoft.
Approximately 0.6% of detected malware were validly code signed."

But they don't go far not enough to show the first point. (You might
validly ask why Microsoft did not go that far, and deny us the clarity
in dealing with this situation! Another subject, another day!)


> The fact that I disclosed publicly
> about it doesn't mean that there aren't scores of others out there.


It doesn't mean there aren't and it doesn't mean there are. On the
basis of a reasonable judgement, given the info we have today, there
aren't. Also, by the same logic, there aren't any mice in your
computer, even though you looked yesterday.

Which isn't to say that it won't change tomorrow. Security work is done
with facts, because attackers will use your conjectures against you.

> Considering the aggressive and illegal mailing campaign they operated,
> one can expect that there are many certificates issued in this way.


Sure. All highly targeted to existing owners of SSL certs ... who
surprise, surprise, didn't notice that there was no validation :)

What damages are likely here, to the targeted users? Are there any
"non-targeted users" ?


>> This situation is about as harmful as the average exploit demo. In this
>> particular case it was caught by an insider rather than by an outsider.
>> However, beyond that, the situation is already sealed in that Comodo
>> have already taken the urgent triage steps to make sure nobody else gets
>> one.
>>
>
> Well, that's exactly the point! I haven't heard back from Robin anymore,

> their site is still online as well.Clear statements about how many


> certificates are affected (all of the ones issued through them) and
> revocations (or at least urgent re-validation within 48 hours and
> thereafter revocation) would be satisfactory maybe.

> The handling of the situation is insufficient, response and measures are
> a joke, communication doesn't exist.


Right. Over to Comodo. It's their job, let them do it. Any statements
you desire to them should tie into some criteria, practice or policy.


> Apparently this resellers business
> was too good for Comodo.


Well, concerns on the concept of resellers have been raised before.
This becomes a case in point, which should perhaps allow us to
re-address the issue.

But, CAs issue bad certs, that's a fact, so we can't damn them entirely.
Also, it's their job to deal with it. Adding a lynching party helps
no-one.

iang

Eddy Nigg

unread,
Dec 24, 2008, 12:41:53 AM12/24/08
to patr...@certstar.com
Hi Particia,

On 12/24/2008 01:58 AM, patr...@certstar.com:
> Dear all,
>

Upfront, as such for me, I accept your apology - sincerely. However your
apology is only for half the story, please read on...

>
> Having worked intensively with this case I can truly say that Comodo
> is indeed taking their responsibility extremely seriously and taking
> huge efforts to ensure that safety of their root. Currently we are
> going through a number certificate to double ensure that no further
> mis-issuances have occurred.

Look, the problem isn't your implementations alone, this failure is a
broader set of failures not only by your company, but that of Comodo.
Comodo has clearly failed in their duties, in their concepts, controls
and policies. The responsibility of domain validation shouldn't be with
you, but part of Comodo's procedures and data flow. As much as your
system perhaps failed - you aren't the only one to take blame and
responsibility.

Besides that, I want to repeat that there was no validation performed at
all. There was no such step, not even a hint of it.

>
> Personally I do feel that it would have been appropriate if we had
> been contacted by StartCom Ltd when they found this flow so that it
> could have been fixed faster. Being our competitor I am not sure if we
> could expect this, but it would indeed have been generous move.

I tell you something about generosity hereby. You have spammed, phished
and mislead our customers by sending those emails, with it everything
started. StartCom has contacted you (myself personally) and requested to
talk to the owners of your company and somebody with the name Mark
answered me. I replied with full disclosure about those emails you are
sending and with immediate request to shut down your activities.
Unfortunately *OUR* email was ignored and we never received a reply
thereafter.

My request was reasonable in light of the potential damage you have
caused us and other certification authorities and whoever makes a
mistake has to pay for it. Your site should have been taken down then.

Also Comodo replied laconically to my complaint. In that respect, and
the way Comodo handled this incident, nobody should be surprised that
who make foes, will not be treated friendly either. It would have been
generous from *YOU* to address the first problem appropriately and come
to terms with us when we requested it.

>
> The technical verification procedure has been improved and is now on a
> very high security level. Comodo will also review our implementation
> to ensure that it comply with all standards and cannot be abused.

As long as those validations are performed at a hosted site at some
hosting provider (yes, I made my researches in this respect) and not by
Comodo's infrastructure, there can't be any talk of "improvement",
period. This is simply not serious! I hope that everybody takes note on
this.

Eddy Nigg

unread,
Dec 24, 2008, 1:35:00 AM12/24/08
to
On 12/24/2008 02:40 AM, Ian G:

> You don't count, or more precisely, the money you spent getting the cert
> doesn't count; sorry about that :)

At least I got it refunded ;-)

>
> Well, concerns on the concept of resellers have been raised before. This
> becomes a case in point, which should perhaps allow us to re-address the
> issue.

Yes indeed. When the right time comes, this must be addressed appropriately.

>
> But, CAs issue bad certs, that's a fact, so we can't damn them entirely.
> Also, it's their job to deal with it. Adding a lynching party helps no-one.
>

Nope, and flaws can happen, no system is immune. However in this case
the handling was a sever failure from top to bottom, this isn't a bug or
isolated failure.

David E. Ross

unread,
Dec 24, 2008, 1:45:19 AM12/24/08
to

I've turned off all trust bits for all Comodo certs in my own personal,
home installation of SeaMonkey. That was step one. I might turn some
back on when I see the consequences. So far, however, I have not
encountered any secure site using Comodo.

The bigger issue is what will be done by the Mozilla organization. Now
that Mozill knows about the problem, I believe they have a liability to
their users if they fail to take action to mitigate the risks to those
users. This is analogous to laws in many U.S. states that hold a city
liable for automobile damage from a pothole AFTER the pothole is
reported to the city; there is no liability before knowledge but much
liability after knowledge (even if the city did not cause the pothole).

--
David E. Ross
<http://www.rossde.com/>

Go to Mozdev at <http://www.mozdev.org/> for quick access to
extensions for Firefox, Thunderbird, SeaMonkey, and other
Mozilla-related applications. You can access Mozdev much
more quickly than you can Mozilla Add-Ons.

ro...@comodo.com

unread,
Dec 24, 2008, 1:56:59 AM12/24/08
to
On Dec 23, 5:09 am, Frank Hecker <hec...@mozillafoundation.org> wrote:
> There are two general reasons for pulling a root, to address a clear and
> present danger to Mozilla users, and to punish a CA and deter others. My
> concern right now is with the former. I see at least three issues in
> relation to that:
>
> 1. Issuance of further non-validated certs by this reseller. Comodo
> seems to have addressed this by suspending the reseller's ability to get
> certs issued. (I can testify that this is the case, as I tried to
> duplicate Eddy's feat earlier today and got my uploaded CSR rejected.)
>
> 2. Potential problems with certs already sold through this reseller.
> Comodo should investigate this and take action if needed. (This need not
> necessarily require revoking all certificates associated with the
> reseller; for example, the existing certs and their associated domains
> could be re-validated, the registered domain owners could be notified of
> the potential for bogus certs floating around, etc.)
>
Frank,
We are in the process of reviewing all of the certificates that have
been issued where Certstar served as a RA for Comodo. Fortunately they
have only been involved in the validation of 111 certificates. As I
previously mentioned as soon as we discovered the error with the
Mozilla.com certificate we suspended Certicom’s RA privileges.
Certstar’s RA activities remain suspended.

At this point it appears that the Mozilla and Startcom certificates
were an anomaly, the result of the automatic DCV mechanism being
bypassed by accident. Certstar’s account will remain suspended until
we are satisfied that the certificates issued for Mozilla and Startcom
were as Certstar maintains, the result of unintentional mistakes that
have been corrected with safeguards in place to prevent a
reoccurrence. Should Comodo decide to reinstate Certstar as an RA we
will review each certificate request until we are satisfied that
Certstar has been adequately trained and that yesterday’s mistake
cannot be duplicated.

Comodo has been able to verify that 73 of the 111 orders processed by
Certstar were processed pursuant to the requirements of our CPS and
our webhost RA terms and conditions. We should have our review of the
remaining 38 certificates completed by tomorrow evening. If we are
unable to verify any validation, we will revoke the applicable
certificate.


> 3. Potential problems with other Comodo resellers. I'm not going to tell
> Comodo how to operate its reseller network, but they certainly should
> take a look at whether and where this might be a problem with other
> resellers, and how they could revamp their systems to reduce potential
> problems with resellers.
>
Comodo takes it responsibility to supervise RAs very seriously and we
actively audit their performance. While it is not practical to audit
100% of their work, we audit a representative sample. In the past we
have discovered only a few isolated incidents where our sub CAs or RAs
failed to follow the applicable guidelines. When this has occurred we
have suspended the account, performed additional audit activities and
where appropriate revoked certificates. We are confident that our
existing system works well and are constantly looking for ways to
improve it.

We apologize for Certstar’s mistake and assure you that we will
redouble our self-auditing efforts to insure the problem does not
repeat itself.

Regards
Robin Alden
Comodo

Eddy Nigg

unread,
Dec 24, 2008, 1:59:03 AM12/24/08
to
On 12/24/2008 02:12 AM, patr...@certstar.com:

> On Dec 24, 12:45 am, Eddy Nigg<eddy_n...@startcom.org> wrote:
>> I reported that my employees can see the supposedly private control
>> panel of this reseller - what else is needed to get this site down?
>
> The certificate has been revoked for quite some time now and is not
> available in your control panel.

Nevertheless I saw it with my own eyes at my co-workers PC. And no, it's
not a proxy on our end. This was about 24 hours after I last visited
your site.

>
> You must be aware of this as you submitted a complaint to Paypal that
> “the certificate purchased was revoked” (you did not get what you paid
> for). One could think that the purpose was to get them to suspend our
> account as similar complaints was made the day before.
>

I bought the certificate for proving that Comodo issues unvalidated
certificates after realizing this during my initial attempt about who
stands behind those shameless and fraudulent emails - and got one for
startcom.org, no questions asked.

And yes, I requested from any possible party to shut your operation
down: Godaddy (which also operates a CA), Paypal (which owns a CA),
Servage (your hosting provider), Comodo (of which you are a reseller).

Dan Colascione

unread,
Dec 24, 2008, 2:02:54 AM12/24/08
to
On Dec 23, 8:56 pm, ro...@comodo.com wrote:
> Comodo has been able to verify that 73 of the 111 orders processed by
> Certstar were processed pursuant to the requirements of our CPS and
> our webhost RA terms and conditions.
> [snip]
> In the past we
> have *discovered* only a few isolated incidents where our sub CA*s* or RA*s*

> failed to follow the applicable guidelines.

[emphasis mine]

Frankly, that's even *more* disturbing. It means that there are almost
certainly unverified certificates in the wild, and that this problem
is pervasive.

Paul C. Bryan

unread,
Dec 24, 2008, 2:13:57 AM12/24/08
to
On Dec 23, 5:56 pm, ro...@comodo.com wrote:
> Comodo takes it responsibility to supervise RAs very seriously and we
> actively audit their performance. While it is not practical to audit
> 100% of their work, we audit a representative sample.

By delegating RA functions (including domain verification) to third
parties, you appear to be making them the weakest links in PKI chains
of trust.

> We apologize for Certstar’s mistake and assure you that we will
> redouble our self-auditing efforts to insure the problem does not
> repeat itself.

Some questions:

1. Does Comodo take full responsibility for the actions of its
resellers? If so, how should the repercussions of such failures be to
Comodo?

2. Are resellers subject to the same audits that Comodo presumably had
to undergo to get its root certs added to Mozilla? Who performs, and
who verifies such audits? How often are they performed?

3. Are you willing to openly, continually disclose your list of
resellers, the frequency of audits, audit methodology, and actual
audit reports so that third parties can evaluate whether Comodo is
trustworthy as a CA?

Nelson B Bolyard

unread,
Dec 24, 2008, 2:16:54 AM12/24/08
to mozilla's crypto code discussion list
Ian G wrote, On 2008-12-23 05:58:

> 3. How to resolve a dispute. This is a Mozilla action &
> responsibility. Reverse-engineering and referring, I would suggest this
> as a teaser:
>
> a. The CA certificate "module owner" at Mozilla foundation is
> responsible. Ref, the policy, pt 15.
> b. The dispute is investigated and ruled on by module owner.

For there to be a "CA certificate module owner", there would have to be
a "CA certificate Module". Presently, there is no such module. Maybe
there should be. As you probably know, Mozilla now has "code modules"
and "non code modules". I think the CA certificate module would be a
non-code module. Yeah, ultimately whatever certs are accepted by Mozilla
do get stored in NSS, but the NSS developers just abide with Mozilla's
decisions about that. There's only ever been one real disagreement
between Mozilla and NSS about root CA certs, and that was how the whole
process we have now got started. Since then, it's been smooth sailing.
I expect it will continue to be that for a long time.

Anyway, I would support the creation of a "CA certificate" non-code module.

Eddy Nigg

unread,
Dec 24, 2008, 2:23:11 AM12/24/08
to
On 12/24/2008 04:16 AM, Nelson B Bolyard:

Seconded +1

Eddy Nigg

unread,
Dec 24, 2008, 2:51:53 AM12/24/08
to
My blog article and exposure has provoked somebody to come forward with
additional evidences concerning the reseller activities of Comodo. In
order to protect the innocent I decided to provide this information
confidentially to Frank Hecker for now. Stay tuned.

David E. Ross

unread,
Dec 24, 2008, 3:14:14 AM12/24/08
to

I've already cleared all trust bits for the three Comodo certificates in
my home installation of SeaMonkey. Should I also do the same for the
certificates under those other names? In other words, what have YOU
done with respect to this issue?

Paul Hoffman

unread,
Dec 24, 2008, 3:32:54 AM12/24/08
to mozilla's crypto code discussion list
At 1:45 AM +0200 12/24/08, Eddy Nigg wrote:
>Paul, you are disappointing me! I have not heard one critical word from you about this incident,

What would be added by me joining the choir? Clearly, Comodo made a mistake in trusting (at least) one of its resellers. The mistake was laid bare, and the folks who might remove Comodo from the root pile are following the issue, probably more closely than they are letting on. Do you really think "oh, but if only Paul Hoffman would be critical, then things will really change"?

FWIW, I would be shocked if you could not get the same result (a cert without sufficient checking of the domain) for a lower-profile domain name from at least five other resellers of other CAs in the root pile. You tried to find this one because this particular reseller tried to steal your customers in a slimy fashion, but you could probably find other resellers (possibly even Comodo resellers) who are just as lax.

> instead you are criticize *me*?

Yes.

>C'mon, give me a break!

You are repeatedly using this list as a springboard to criticize a competitor. When you didn't get your way instantly, you made threats against Mozilla, an organization for which many of us have a lot of respect. No break is justified.

>I reported that my employees can see the supposedly private control panel of this reseller - what else is needed to get this site down?

I guess you aren't reading the responses from the people on the thread that might not be as upset as you are. That question was already answered.

>>And, yes, I'm serious. Given that Startcom has the ability to issue bogus certificates like the kind that Eddy is threatening, I would think that a public statement like the above is relevant to Mozilla or Microsoft deciding whether or not the organization is trustworthy.
>
>I don't need the services of Comodo for that, if I would have ever wanted to that, I could do so long time ago.

Yes, exactly. And you, the COO/CTO of a trusted CA, are making public threats that would be the equivalent of that. I understand that you don't think that is a problem; please understand that other might think it is.

Paul Hoffman

unread,
Dec 24, 2008, 3:33:00 AM12/24/08
to mozilla's crypto code discussion list
At 1:16 AM +0200 12/24/08, Eddy Nigg wrote:
>Select Preferences -> Advanced -> View Certificates -> Authorities. Search for AddTrust AB -> AddTrust External CA Root and click "Edit". Remove all Flags.
>
>This would remove the trust from the potentially affected sites and their certificates. Comodo has many more roots if you are interested, such as all AddTrust, Comodo CA Limited, The Usertrust Network.

I'm glad that Eddy was the one who posted this. Doesn't this seem like a better solution than "sue Mozilla for theoretical future damages incurred by using their free software"? Put more rudely, why do you expect Daddy to fix it for you when you can fix it yourself, more easily and more quickly, if you want to?

Eddy Nigg

unread,
Dec 24, 2008, 3:50:23 AM12/24/08
to
On 12/24/2008 05:32 AM, Paul Hoffman:

> At 1:45 AM +0200 12/24/08, Eddy Nigg wrote:
>> Paul, you are disappointing me! I have not heard one critical word from you about this incident,
>
> You tried to find this one because this particular reseller tried to steal your customers in a slimy fashion...

Well, I didn't tried to find the flaw, I tried to find the CA
responsible. It took me a while to recap how I got the cert for my
domain and went like...gosh, this can't be, I must have overlooked
something. I decided to check it out and bingo.

As to my personality, what I wrote in my blog article is what I meant. I
was clearly disappointed, because I'm also a strong supporter of domain
validated certificates and that they have their place in PKI (provided
that they are applied correctly). Obviously, this incident makes it
harder if we don't act accordingly.


> but you could probably find other resellers (possibly even Comodo resellers) who are just as lax.

I really hope not. And it should serve others to re-check their controls
and procedures.

>
> You are repeatedly using this list as a springboard to criticize a competitor.

That's how you look at it. I think I've contributed and proved that I'm
not against our competition, I'm against devaluing our work
collectively. One CA is enough to get us looking for other jobs if you
don't mind.

> When you didn't get your way instantly, you made threats against Mozilla, an organization for which many of us have a lot of respect.

I never threatened Mozilla. If at all, I made clear what the results may
be. Including this specific incident shows clearly that my arguments and
concerns were real in many ways. Here we have a CA which issues domain
validated wild cards up to ten years. Now you tell me how this is OK -
specially in light of what happened.

I care about what I'm doing, yes! And I have no intention apologizing
for it.

> Yes, exactly. And you, the COO/CTO of a trusted CA, are making public threats that would be the equivalent of that. I understand that you don't think that is a problem; please understand that other might think it is.

OK, maybe I should have made my point different.

jack...@gmail.com

unread,
Dec 24, 2008, 4:31:48 AM12/24/08
to
On Dec 22, 7:15 pm, "Robin Alden" <ro...@comodo.com> wrote:
> Eddy,
>         That reseller's ability to sell Comodo certificates has been
> suspended while we investigate why they are apparently not fulfilling their
> contractual obligations to us.
>         We revoked your certificate for mozilla.com.
>
> Regards
> Robin Alden
> Comodo
>
> > -----Original Message-----
> > From: dev-tech-crypto-bounces+robin=comodo....@lists.mozilla.org
> > [mailto:dev-tech-crypto-bounces+robin=comodo....@lists.mozilla.org] On
> > Behalf Of Eddy Nigg
> > Sent: Monday, December 22, 2008 10:25 PM
> > To: dev-tech-cry...@lists.mozilla.org
> > Subject: Unbelievable!
>
> >https://blog.startcom.org/?p=145

>
> > --
> > Regards
>
> > Signer: Eddy Nigg, StartCom Ltd.
> > Jabber: start...@startcom.org
> > Blog:      https://blog.startcom.org
> > _______________________________________________
> > dev-tech-crypto mailing list
> > dev-tech-cry...@lists.mozilla.org
> >https://lists.mozilla.org/listinfo/dev-tech-crypto

I guess Comodo didn't do their own due diligence. How can anyone trust
Comodo?

Gen Kanai

unread,
Dec 24, 2008, 4:54:48 AM12/24/08
to mozilla's crypto code discussion list

Kyle Hamilton

unread,
Dec 24, 2008, 5:20:01 AM12/24/08
to mozilla's crypto code discussion list
On Tue, Dec 23, 2008 at 6:16 PM, Nelson B Bolyard <nel...@bolyard.me> wrote:
>
> Anyway, I would support the creation of a "CA certificate" non-code module.

I think this would be a really good idea. I'm aware that my opinion
carries little weight, but I think that since it relies on business-
and legal-side undertakings, it shouldn't be managed by the coders.

How would this work? Split nssckbi out to be managed by the non-code
module owner, though a coder would need to be enlisted to finalize the
decisions made by that person?

-Kyle H

Ian G

unread,
Dec 24, 2008, 10:53:25 AM12/24/08
to mozilla's crypto code discussion list
On 24/12/08 03:16, Nelson B Bolyard wrote:
> Ian G wrote, On 2008-12-23 05:58:
>
>> 3. How to resolve a dispute. This is a Mozilla action&
>> responsibility. Reverse-engineering and referring, I would suggest this
>> as a teaser:
>>
>> a. The CA certificate "module owner" at Mozilla foundation is
>> responsible. Ref, the policy, pt 15.
>> b. The dispute is investigated and ruled on by module owner.
>
> For there to be a "CA certificate module owner", there would have to be
> a "CA certificate Module". Presently, there is no such module. Maybe
> there should be.

I was searching for the role, and found that term in the policy, I
think. Mentally, I call it "the CA desk" and sometimes use that term.
The earlier text had a term "CA Manager" which is a bit old-fashioned.

It's whatever it ends up being called, the point for me was "one person
at Mozo who knows this stuff, decides!"

> As you probably know, Mozilla now has "code modules"
> and "non code modules". I think the CA certificate module would be a
> non-code module. Yeah, ultimately whatever certs are accepted by Mozilla
> do get stored in NSS, but the NSS developers just abide with Mozilla's
> decisions about that. There's only ever been one real disagreement
> between Mozilla and NSS about root CA certs, and that was how the whole
> process we have now got started. Since then, it's been smooth sailing.
> I expect it will continue to be that for a long time.
>

> Anyway, I would support the creation of a "CA certificate" non-code module.


OK.

To address Frank's comment:

>> 4. Finality. ... In the policy, it says "... mozilla.org staff..."

> There is no longer a "mozilla.org staff" ...

(In which case, a policy change will be needed one day.)

I would let my comment stand. The point here is that if the day-to-day
manager is disputed, an immediate next-desk-warmer isn't going to help
much. Go for the top, hand it to the ultimate responsible party, the
board. The need here is to send a serious signal, short circuit the
process, and avoid any additional layers of bureaucracy.

iang

Frank Hecker

unread,
Dec 24, 2008, 2:17:00 PM12/24/08
to
Gen Kanai wrote:
> More discussion on this topic over at Programming Reddit:
>
> http://www.reddit.com/r/programming/comments/7lb96/ssl_certificate_for_mozillacom_issued_without/

Unfortunately the discussion devolved (as it always does :-) into the
merits of self-signed certificates. Oh well.

Frank

--
Frank Hecker
hec...@mozillafoundation.org

Frank Hecker

unread,
Dec 24, 2008, 3:37:50 PM12/24/08
to
Eddy Nigg wrote:
> My blog article and exposure has provoked somebody to come forward with
> additional evidences concerning the reseller activities of Comodo. In
> order to protect the innocent I decided to provide this information
> confidentially to Frank Hecker for now. Stay tuned.

To expand on what I wrote to Eddy via email, I really don't want to be
dealing with information sent to me in confidence. I'm skeptical about
dealing with reports where the originators of the reports aren't willing
to go on public record with their complaints, especially when there are
possibly mixed motives on the part of the person or organization
reporting the problem. (For example, CAs reporting on either CAs, or
resellers reporting on other resellers.)

Kyle Hamilton

unread,
Dec 24, 2008, 5:14:32 PM12/24/08
to mozilla's crypto code discussion list
On Wed, Dec 24, 2008 at 6:17 AM, Frank Hecker
<hec...@mozillafoundation.org> wrote:
> Gen Kanai wrote:
>>
>> More discussion on this topic over at Programming Reddit:
>>
>>
>> http://www.reddit.com/r/programming/comments/7lb96/ssl_certificate_for_mozillacom_issued_without/
>
> Unfortunately the discussion devolved (as it always does :-) into the merits
> of self-signed certificates. Oh well.
>
> Frank

Honestly, I get more mileage out of doing my own diligence. This is
why I want to have my own root certificate that I can cross-sign the
various commercial CA certificates with, so that I can revoke those
cross-signatures if they turn out to have a problem.

I'd like to see an extension that allows other certificates (for the
same public key) to be included in a certificate (self-signed or not).
This would protect against one of the threats against the OpenPGP
model, would create a means for certifications to be cherry-picked for
any given application, and would allow other identities for that
public key to be added by the identity collector -- but would also
create a linkage back to the identity collector's public key.
(realistically, the only reason for the signature on an external
certificate that contains an extension like this would be because no
X.509 software handles the TBSCertificate structure without the
signature.)

-Kyle H

Paul Hoffman

unread,
Dec 24, 2008, 5:55:34 PM12/24/08
to mozilla's crypto code discussion list
At 9:14 AM -0800 12/24/08, Kyle Hamilton wrote:
>I'd like to see an extension that allows other certificates (for the
>same public key) to be included in a certificate (self-signed or not).

Are you asking for a Mozilla extension or a PKIX extension? If the latter, none is needed: it is already inherent in PKIX. In fact, I am not sure that anything needs to be done by Mozilla. The following should theoretically work:

- Remove all trust anchors one-by-one
- Add your single trust anchor
- Sign the certs of any CA you want
- Add those signed certs to the pre-loaded validation path (not root) cert list

I haven't tried this myself, but it should work. I have been told that something very similar to it works fine in XP/Vista for IE.

--Paul Hoffman

Kyle Hamilton

unread,
Dec 24, 2008, 7:34:55 PM12/24/08
to mozilla's crypto code discussion list
On Tue, Dec 23, 2008 at 5:14 AM, Frank Hecker
<hec...@mozillafoundation.org> wrote:
> Eddy Nigg wrote:
>>
>> Disabling the trust bits of "AddTrust External CA Root" could be a
>> temporary measure to prevent damage to relying parties until Mozilla
>> receives full report and disclosure from Comodo about its resellers and
>> conclusion of their investigation.
>
> Do you mean the UTN-UserFirst-Hardware root? According to the screenshot on
> your blog post, that's the root the bogus cert chains up to. Also, if we
> were to take action of this general sort (as a hypothetical), what about
> adding the PositiveSSL CA cert to NSS with the SSL trust bit disabled;
> wouldn't that accomplish the same purpose, without interfering with other
> parts of the hierarchy under the UTN-UserFirst-Hardware root? (I seem to
> recall we've discussed this sort of thing in the past.)

What is the effect of this problem on the request to enable the
UTN-UserFirst-Hardware root for EV,
https://bugzilla.mozilla.org/show_bug.cgi?id=401587 ?

-Kyle H

Kyle Hamilton

unread,
Dec 24, 2008, 7:35:56 PM12/24/08
to mozilla's crypto code discussion list
In the terminology of ASN.1 and PKIX, I want a standardized PKIX
extension that allows for a SEQUENCE OF Certificate within the
tbsCertificate structure.

I'm trying to figure out how I'm supposed to extract all the
certificates from my database without any version of keytool that I
can find available for OSX 10.5 (Leopard).

-Kyle H

> _______________________________________________
> dev-tech-crypto mailing list
> dev-tec...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-crypto
>

John Nagle

unread,
Dec 24, 2008, 6:41:41 PM12/24/08
to dev-tec...@lists.mozilla.org
As a user of SSL certificates in our SiteTruth system, which
attempts to identify and rate the business behind a web site, we're
concerned about CA reliability and trust. We've been using Mozilla's
approved root cert list for our system, and are considering whether
we should continue to do so. Given the situation described in

"http://markmail.org/message/rje3lssql55l2rev?q=unbelievable"

the use of Mozila's root CA list is now questionable.

There seem to be several problems here.

1. AddTrust, a company which apparently no longer exists, has an approved
root CA certificate. This in itself is troublesome. Comodo does
not seem to have taken on the obligations of AddTrust; see
"http://markmail.org/message/3zr4e5hxwmxjbgnp?q=Comodo+AddTrust".
Yet Comodo is still issuing certificates under the root CA of
this dead company.

2. Comodo is apparently not only allowing resellers like CertStar,
but is allowing them to do their own validation of the legitimacy
of the certificate requestor. Who takes financial responsibility
for such errors? CertStar itself disclaims financial responsibility
at "http://www.certstar.com/terms.html".

3. Microsoft requires an annual audit for root CAs:
"http://technet.microsoft.com/en-us/library/cc751157.aspx".
Mozilla seems willing to accept a one-time audit. That seems
to be why the disappearance of AddTrust wasn't noticed.
Microsoft's audit requirements extend all the way down the
chain of trust. Their policy is:
"The CA must complete an audit and submit audit results to Microsoft
every twelve (12) months. The Audit must cover the full PKI hierarchy
that will be enabled by Microsoft through the assignment of Extended
Key Usages (EKUs). All certificate usages that we enable must be
audited periodically. The audit report must document the full scope
of the PKI hierarchy including any sub-CA that issues a specific type
of certificate covered by an audit." Microsoft uses the standards of
the WebTrust Program for Certification Authorities
("http://www.cica.ca/download.cfm?ci_id=45239&la_id=1&re_id=0")
managed by the Canadian Society of Chartered Accountants. That's
a good guideline for Mozilla to follow.

At this point, I would suggest that the following actions are appropriate
to bring Comodo and Mozilla into compliance with the WebTrust standards:

1. Comodo must undergo an audit to WebTrust standards, and the audit
report must be published. An in-house self-investigation is not
acceptable. The audit must be conducted by a recognized outside
auditing firm.
2. CertStar must separately undergo an audit to WebTrust standards,
and the audit report must be published. An in-house
self-investigation is not acceptable. The audit must be conducted
by a recognized outside auditing firm. CertStar should not be
permitted to issue any new certificates until this process is
complete and the audit results are satisfactory. If this process
is not complete within 3 months, all CertStar-issued certificates
should be revoked.
3. Comodo must disclose the identities of all "resellers" to which
it has outsourced any validation functions.
4. All AddTrust root CA certificates must be phased out. An
expiration date in Q1 or Q2 2009 is suggested. Since no company
stands behind the AddTrust name, it's necessary to force expiration
of that root.
5. The Mozilla Foundation should contact Microsoft's CA Root Certificate
group to coordinate their actions.

John Nagle
SiteTruth

Nelson B Bolyard

unread,
Dec 24, 2008, 9:42:45 PM12/24/08
to mozilla's crypto code discussion list
Kyle Hamilton wrote, On 2008-12-23 21:20:
> On Tue, Dec 23, 2008 at 6:16 PM, Nelson B Bolyard <nel...@bolyard.me> wrote:
>> Anyway, I would support the creation of a "CA certificate" non-code module.
>
> I think this would be a really good idea. I'm aware that my opinion
> carries little weight, but I think that since it relies on business-
> and legal-side undertakings, it shouldn't be managed by the coders.
>
> How would this work? Split nssckbi out to be managed by the non-code
> module owner, though a coder would need to be enlisted to finalize the
> decisions made by that person?

No, it would be a NON-CODE module. It would not contain any code.
Its output would be the list of trusted root certs, perhaps as a web page,
and/or also as a set of requests (in the form of Bugzilla bugs) to have
certs inserted into nssckbi.

nssckbi is just a medium for the conveyance of that list, potentially one
of several. The task for the NSS module owner would be to ensure that
the copy of the list in nssckbi is kept reasonably up to date, and doesn't
differ from the official list (or a very recent version of that list) as of
the date on which it is released.

That's really how things operate now. I'm merely suggesting that that
separation of responsibility be formalized by making the maintenance of
the official CA list be a separate "module".

Nelson B Bolyard

unread,
Dec 24, 2008, 9:46:41 PM12/24/08
to mozilla's crypto code discussion list
Paul Hoffman wrote, On 2008-12-24 09:55:
> At 9:14 AM -0800 12/24/08, Kyle Hamilton wrote:
>> I'd like to see an extension that allows other certificates (for the
>> same public key) to be included in a certificate (self-signed or not).
>
> Are you asking for a Mozilla extension or a PKIX extension? If the
> latter, none is needed: it is already inherent in PKIX. In fact, I am not
> sure that anything needs to be done by Mozilla. The following should
> theoretically work:
>
> - Remove all trust anchors one-by-one
> - Add your single trust anchor
> - Sign the certs of any CA you want
> - Add those signed certs to the pre-loaded validation path (not root)
> cert list
>
> I haven't tried this myself, but it should work. I have been told that
> something very similar to it works fine in XP/Vista for IE.

Of course, that is COMPLETELY equivalent to simply setting trust flags on
the CA certs you want to trust, and removing those flags from the ones you
don't want to trust, which is already a part of Mozilla browsers (and
Netscape browsers, before them) for over 14 years.

Kyle Hamilton

unread,
Dec 24, 2008, 10:36:40 PM12/24/08
to mozilla's crypto code discussion list

To be honest, Mozilla doesn't distribute keytool with Firefox, which
means that I have to try to go into the (unbatchable) interface and
remove the flags one. by. one. by. one. and then select the next
certificate and remove those trust flags, and the next, and the next,
and the next...

...for all hundred or so certs that Firefox includes.

And then, once I DO manage to do that, then with the "new and
improved" user interface updates, I then have to click at least six
times to try to figure out what's going on, and then when I do find a
site that's protected by an unknown CA certificate (OR that I've
removed the trust bits on), I have to do the following:

1) Click 'add an exception'
2) click 'get certificate' (why I should have to do this is beyond me,
since firefox obviously already has the certificate downloaded since
it told me 'sec_error_untrusted_issuer', which it couldn't have known
without the certificate in its possession ANYWAY)
3) click 'view'
4) get the name of the Issuer
5) hope to all the gods that there's enough information in the chain
to figure out what root it's supposed to be going to
6) close the window
7) go into Preferences
8) click Advanced
9) click Encryption
10) click 'View Certificates'
11) Scroll through the list, with each click giving me approximately
0.6 useful results (given the preponderance of 'section headings by
root owner', which by the way doesn't work at all with the Addtrust AB
stuff since those are Comodo roots)
12) find the appropriate root and re-enable it for identification of websites
13) refresh the page.

How 'bout this, Nelson (and I invite Frank and the entire security UI
team to do this, as well): YOU do it. Create a new profile and
manually remove the trust on every CA. Then, browse around, and see
which CAs are actually used by you in your day-to-day browsing,
reenabling them manually (since you're trying to emulate not having
keytool around).

Furthermore, even when keytool IS available, it's entirely likely that
its name conflicts with Java's keytool. (especially on Mac OSX.)

This is completely unworkable, and discourages users that want to from
taking their security into their own hands.

-Kyle H

Kyle Hamilton

unread,
Dec 24, 2008, 10:38:00 PM12/24/08
to mozilla's crypto code discussion list
I'm also going to state that yes, I know this, because I HAVE DONE IT.
And I wouldn't wish that hell on anyone who didn't have a DETAILED
knowledge of how the X.509 model operates, and I wouldn't wish the
user-interface hell on ANYONE.

-Kyle H

Kyle Hamilton

unread,
Dec 24, 2008, 10:42:25 PM12/24/08
to mozilla's crypto code discussion list

Thanks for the explanation.

I do agree that the separation of responsibility would be good, since
Frank (appears to?) does the actual CA approval and you appear to be
the one primarily who implements his directives as regards the updates
to nssckbi?

-Kyle H

Eddy Nigg

unread,
Dec 24, 2008, 10:46:08 PM12/24/08
to
On 12/25/2008 12:36 AM, Kyle Hamilton:

> To be honest, Mozilla doesn't distribute keytool with Firefox, which
> means that I have to try to go into the (unbatchable) interface and
> remove the flags one. by. one. by. one. and then select the next
> certificate and remove those trust flags, and the next, and the next,
> and the next...

Kyle, why don't you blow that libnssckbi away from your Firefox
installation? Would make it easier I think. (Hope I picked the right one
;-) )

--
Regards

Signer: Eddy Nigg, StartCom Ltd.

Jabber: star...@startcom.org
Blog: https://blog.startcom.org

Kyle Hamilton

unread,
Dec 24, 2008, 10:53:08 PM12/24/08
to mozilla's crypto code discussion list
On Wed, Dec 24, 2008 at 2:46 PM, Eddy Nigg <eddy...@startcom.org> wrote:
> On 12/25/2008 12:36 AM, Kyle Hamilton:
>>
>> To be honest, Mozilla doesn't distribute keytool with Firefox, which
>> means that I have to try to go into the (unbatchable) interface and
>> remove the flags one. by. one. by. one. and then select the next
>> certificate and remove those trust flags, and the next, and the next,
>> and the next...
>
> Kyle, why don't you blow that libnssckbi away from your Firefox
> installation? Would make it easier I think. (Hope I picked the right one ;-)
> )

Primarily because I want those certs on one profile, but not another,
and disk space is kind of at a premium right now. :)

(Oh yeah, if one person who uses a computer doesn't want the built-in
roots, but another does, they have to have separate Firefox
installations.)

-Kyle H

Nelson B Bolyard

unread,
Dec 24, 2008, 10:53:39 PM12/24/08
to mozilla's crypto code discussion list
Kyle Hamilton wrote, On 2008-12-24 14:42:

> Thanks for the explanation.
>
> I do agree that the separation of responsibility would be good, since
> Frank (appears to?) does the actual CA approval

Yes

> and you appear to be the one primarily who implements his directives as
> regards the updates to nssckbi?

I did it at one time. Presently, and for the last year or more, keeping
nssckbi up to date with Frank's list has been done by Kai Engert who
also works on PSM, the browser component that interfaces to NSS.

Nelson B Bolyard

unread,
Dec 24, 2008, 11:14:51 PM12/24/08
to mozilla's crypto code discussion list
Kyle Hamilton wrote, On 2008-12-24 14:53:
> On Wed, Dec 24, 2008 at 2:46 PM, Eddy Nigg <eddy...@startcom.org> wrote:
>> On 12/25/2008 12:36 AM, Kyle Hamilton:
>>> To be honest, Mozilla doesn't distribute keytool with Firefox, which
>>> means that I have to try to go into the (unbatchable) interface and
>>> remove the flags one. by. one. by. one. and then select the next
>>> certificate and remove those trust flags, and the next, and the next,
>>> and the next...

What is the relevance of keytool here?
The tool relevant to NSS is certutil. It is scriptable.
It's quite possible to write scripts to use certutil to do mass trust
modification. I've done it. But there's probably a much easier way
than that.

>> Kyle, why don't you blow that libnssckbi away from your Firefox
>> installation? Would make it easier I think. (Hope I picked the right one ;-)
>> )
>

> Primarily because I want those certs on one profile, but not another,
> and disk space is kind of at a premium right now. :)
>
> (Oh yeah, if one person who uses a computer doesn't want the built-in
> roots, but another does, they have to have separate Firefox
> installations.)

No, it is possible to accommodate both with a single installation.
There are several ways to do it. Here's one.
Move the nssckbi file from the Firefox code directory where it normally
lives to the profile directory (where the cert8.db file lives) for the
user profile that wants to use it.

sayrer

unread,
Dec 25, 2008, 12:50:46 AM12/25/08
to
On Dec 23, 10:33 pm, Paul Hoffman <phoff...@proper.com> wrote:
> At 1:16 AM +0200 12/24/08, Eddy Nigg wrote:
>
> >Select Preferences -> Advanced -> View Certificates -> Authorities. Search for AddTrust AB -> AddTrust External CA Root and click "Edit". Remove all Flags.
>
> Put more rudely, why do you expect Daddy to fix it for you when you can fix it yourself, more easily and more quickly, if you want to?

Mozilla is in the business of protecting all users, not just those
with sysadmin levels of skill. I'm not advocating drastic action with
the Comodo roots, but a workarounds of this sort are probably not
distinguishable from total failure when the entire user base is taken
into account.

The truth is that we are basically unable to act without a lot of
collateral damage. We should keep this in mind with future security
technology. Relying on companies willing to take money for doing
absolutely nothing (not even the bare minimum they agreed to) is not a
pleasant thing to do to our users. We didn't learn this lesson with
EV--maybe next time! :)

- Rob

Paul Hoffman

unread,
Dec 25, 2008, 12:58:48 AM12/25/08
to mozilla's crypto code discussion list
At 1:46 PM -0800 12/24/08, Nelson B Bolyard wrote:
>Paul Hoffman wrote, On 2008-12-24 09:55:
> > - Remove all trust anchors one-by-one
>> - Add your single trust anchor
>> - Sign the certs of any CA you want
>> - Add those signed certs to the pre-loaded validation path (not root)
> > cert list
>
>Of course, that is COMPLETELY equivalent to simply setting trust flags on
>the CA certs you want to trust, and removing those flags from the ones you
>don't want to trust, which is already a part of Mozilla browsers (and
>Netscape browsers, before them) for over 14 years.

Not "COMPLETELY", but close. What I proposed has a signature at the top of the hierarchy, which is what I thought that Kyle was asking for. The result is completely equivalent, but the format is slightly different.

Of course, it is much easier for the people on this list to Insist With Exclamation Marks! that Mozilla fix this for them instead of them doing it themselves, but that problem is at different layer.

Paul Hoffman

unread,
Dec 25, 2008, 12:54:44 AM12/25/08
to mozilla's crypto code discussion list
At 11:35 AM -0800 12/24/08, Kyle Hamilton wrote:
>In the terminology of ASN.1 and PKIX, I want a standardized PKIX
>extension that allows for a SEQUENCE OF Certificate within the
>tbsCertificate structure.

That makes no sense to me, but I would have to see a complete proposal to understand why you would want that.

>I'm trying to figure out how I'm supposed to extract all the
>certificates from my database without any version of keytool that I
>can find available for OSX 10.5 (Leopard).

That's a completely different problem, of course.

Daniel Veditz

unread,
Dec 25, 2008, 7:13:55 AM12/25/08
to mozilla's crypto code discussion list
Paul Hoffman wrote:
> At 1:16 AM +0200 12/24/08, Eddy Nigg wrote:
>> Select Preferences -> Advanced -> View Certificates -> Authorities.
>> Search for AddTrust AB -> AddTrust External CA Root and click
>> "Edit". Remove all Flags.
>
> Doesn't this seem like a better solution than "sue Mozilla for
> theoretical future damages incurred by using their free software"?
> Put more rudely, why do you expect Daddy to fix it for you when you
> can fix it yourself, more easily and more quickly, if you want to?

Isn't that a bit like an auto maker issuing a notice "If you don't want
your car to explode in a fender bender you can crawl under the right
rear and remove the screw marked 34A on the following diagram."

Everyone should be able to do that so I'm sure that gets the auto maker
off the hook, right? Assuming all 200 million Firefox users even hear
about the problem.

Daniel Veditz

unread,
Dec 25, 2008, 9:56:20 AM12/25/08
to
Kyle Hamilton wrote:
> I then have to click at least six
> times to try to figure out what's going on, and then when I do find a
> site that's protected by an unknown CA certificate (OR that I've
> removed the trust bits on), I have to do the following:
>
> 1) Click 'add an exception'
> 2) click 'get certificate' (why I should have to do this is beyond me,
> since firefox obviously already has the certificate downloaded since
> it told me 'sec_error_untrusted_issuer', which it couldn't have known
> without the certificate in its possession ANYWAY)

Not that it changes your point any, but if you set the pref
browser.ssl_override_behavior to 2 then you won't need to "get
Certificate". Firefox 3.1 will have this behavior by default.

If you set browser.xul.error_pages.expert_bad_cert to true then you
won't have to click a link to reveal the "add exception" button, saving
a click at that step, too. Firefox 3.1 won't be adopting that default
though.

Michael Ströder

unread,
Dec 25, 2008, 11:15:42 AM12/25/08
to
Kyle Hamilton wrote:
> (Especially if Comodo delegates full Registration Authority capability
> without verification, which seems to be the case -- though they could
> have simply issued a sub-CA certificate.)

Delegating the RA's tasks is still different from issuing a sub-CA cert
since with a delegated RA the CA can look at all issued EE certs and
revoke some of them if needed. A sub-CA typically runs completely on its
own so the CA could only revoke the sub-CA cert and not certain EE certs.

> It occurs to me that there is no facility in Firefox or other Mozilla
> products to provide an explanatory dialog that there's an issue, and
> such a facility would be extremely useful at this point. Being able
> to print a message to the user like "The Mozilla Foundation has
> identified issues with the trusted root that issued this certificate
> which prevent Firefox from being able to guarantee that this is truly
> the site to which you intended to go. While it is unlikely that this
> is a widespread problem, and an attack would rely on more technical
> intrusions into the network, the nature of these issues requires that
> you be warned of this circumstance so that you can exercise
> appropriate levels of caution. The Mozilla Foundation is working with
> the trusted root to resolve these issues." would help a lot.

Either the trust bits are removed or not. Such a dialogue wouldn't help
at all. It's too complicated.

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 11:21:02 AM12/25/08
to
Eddy Nigg wrote:
> On 12/23/2008 09:09 AM, Kyle Hamilton:
>> Of course, this would be an NSS change (the addition of a 'trust
>> suspended' bit,
>
> I think this to be an interesting idea and should be considered.

I really wonder why there should be one state more. And how is it going
to be set (updated)? A cert is either trusted or not. Period.

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 11:04:51 AM12/25/08
to
Frank Hecker wrote:
> From my point of view I'd wait on more
> information regarding items 2 and 3 above before making a recommendation.

Could you please define a time-frame within Comodo MUST react?

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 11:24:04 AM12/25/08
to
Kyle Hamilton wrote:
> [..many good observations snipped..]
> Because of this, my recommendation that Comodo's trust bits be removed
> until a full audit of their practices (and a full audit of all issued
> certificates) stands, and I am that much more resolute in my belief.

Full ack!

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 12:23:41 PM12/25/08
to
Kyle Hamilton wrote:
> I hate to say this, but this IS The Worst-Case Scenario. A CA has
> gone rogue and issued certificates that violate its standards, and the
> standards of the root programs that it's a part of -- it is true that
> Comodo didn't /intend/ to go rogue, but it has, and we can't afford to
> let it damage the greater PKI. Since every CA in the root store is
> treated the same, there is no differentiation between them -- and this
> means that Verisign and Comodo and Thawte and *every* CA share the
> same reputation. If one goes rogue, it's exactly the same as if all
> of them have gone rogue, in the eye of the end-user.

I fully agree here. That's why I support to remove the trust bit from
the Comodo root CA cert immediately and make them go through the whole
process of applying to be a trusted root CA.

> THIS is why I want to see greater differentiation in the browser
> chrome between CAs, so that one bad apple doesn't spoil the whole root
> barrel.

I don't think that's feasible. Nobody will be able to deal with that
differentiation. That's also the reason why I think that EV certs does
not help. The problem is the lack of auditing CAs and then punishing
rogue CAs.

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 12:33:51 PM12/25/08
to
Justin Dolske wrote:
> ...I think there's some risk that if a Firefox update suddenly breaks a
> large swath of legitimate SSL sites, that could end up training users to
> ignore the problem.

Given the large amount of self-generated server certs this problem
already exists. Ultimately you cannot hold a user back from shooting
himself in the foot.

> I'm not even sure how you'd present this condition
> to Joe The User in a comprehensible way.

I'm pretty sure that if this case is taken to popular news ticker sites
or other publications many users will be aware of this.

Ciao, Michael.

Michael Ströder

unread,
Dec 25, 2008, 12:39:07 PM12/25/08
to
dou...@theros.info wrote:
> I, for example, have a ssl cert from comodo reseller, and they DO have
> made all the validation steps.
>
> My site, a legitimate one, would be in trouble with this. Are you all
> sure that it is a good measure to just knock off the root cert or
> security bit?
>
> please, think twice

Douglas, I understand that this would be a problem for you. But after
thinking twice the larger issue with bad operational practice at
Comodo's CA and their resellers outweigh your personal damage.

Ciao, Michael.

Eddy Nigg

unread,
Dec 25, 2008, 3:30:35 PM12/25/08
to
On 12/25/2008 02:39 PM, Michael Ströder:

If the operations of certstar would have been a glitch and bug in their
validation system and a very isolated event, I would not suggest to take
any actions beyond requesting to have it fixed properly, reviewed and
approved by the Comodo management.

The very fact that there was no validation in place *at all* suggests
however that Comodo hasn't done any review, testing and approval of
their systems. This is beyond the acceptable norm of failures which
certainly can happen - it suggests gross negligence by Comodo.

Secondly, I believe that such crucial parts shouldn't be outsourced to a
third party - an issue we'll have to look at very closely soon here at
Mozilla. More than that, Comodo hasn't any controls in place to prevent
fraudulent or mistaken issuance of certificates of high profile targets.
This is another failure.

Third and as noted, resellers don't have to undergo any or only some
validations - insufficient and not adhering to the Mozilla CA Policy.
The policy is very clear in this respect and Comodo has failed to
disclose this properly during their review this spring.

Douglas, if and when any actions will be taken, you'll be eligible for
compensation by Comodo. You would have to look elsewhere to get a new
certificates maybe. This would be perhaps annoying, however the risk of
real damage to a third party would be much more severe.


--
Regards

Signer: Eddy Nigg, StartCom Ltd.
Jabber: star...@startcom.org
Blog: https://blog.startcom.org

Ian G

unread,
Dec 25, 2008, 3:44:51 PM12/25/08
to mozilla's crypto code discussion list
On 24/12/08 15:17, Frank Hecker wrote:
> Gen Kanai wrote:
>> More discussion on this topic over at Programming Reddit:
>>
>> http://www.reddit.com/r/programming/comments/7lb96/ssl_certificate_for_mozillacom_issued_without/
>
>
> Unfortunately the discussion devolved (as it always does :-) into the
> merits of self-signed certificates. Oh well.

And the merits of mineral water versus municipal water... There are a
lot of people happy with municipal water, and a lot of people happy with
mineral water, and they aren't likely to sit down and share a social
drink :)

iang

Frank Hecker

unread,
Dec 25, 2008, 4:49:33 PM12/25/08
to
Michael Ströder wrote:
> Frank Hecker wrote:
>> From my point of view I'd wait on more
>> information regarding items 2 and 3 above before making a recommendation.
>
> Could you please define a time-frame within Comodo MUST react?

Comodo (in the person of Robin Alden) has already made a reply:

http://groups.google.com/group/mozilla.dev.tech.crypto/msg/b24e70ea2c396bb5

The question is, what else do what want Comodo to do in this case? They
still have some certificates unaccounted for in terms of verifying the
validation, and certainly I'd like to hear the status of that as soon as
possible. Beyond that? It's somewhat of an open question.

Frank

--
Frank Hecker
hec...@mozillafoundation.org

Frank Hecker

unread,
Dec 25, 2008, 5:59:26 PM12/25/08
to
Kyle Hamilton wrote:
> What is the effect of this problem on the request to enable the
> UTN-UserFirst-Hardware root for EV,
> https://bugzilla.mozilla.org/show_bug.cgi?id=401587 ?

I think (but don't have time to confirm right at the moment) that that
request is moot. As far as I know, Comodo EV certificates are working
fine right now even in the absence of the UTN-UserFirst-Hardware root
being enabled for EV. This is due to EV-enabling of the new Comodo EV
root and also IIRC due to code that was added to PSM (?) to specially
handle cases like this where the EV root was cross-signed by a non-EV
legacy root.

(AFAIK this scenario was/is not unique to Comodo. I believe all the
VeriSign EV CAs work this way as well: a newly added and EV-enabled EV
root cross-signed by a non-EV legacy root.)

Michael Ströder

unread,
Dec 25, 2008, 6:16:10 PM12/25/08
to
Frank Hecker wrote:
> Michael Ströder wrote:
>> Frank Hecker wrote:
>>> From my point of view I'd wait on more
>>> information regarding items 2 and 3 above before making a
>>> recommendation.
>>
>> Could you please define a time-frame within Comodo MUST react?
>
> Comodo (in the person of Robin Alden) has already made a reply:
>
> http://groups.google.com/group/mozilla.dev.tech.crypto/msg/b24e70ea2c396bb5

Yes, already saw that in the meantime. But it does not really say much.

> The question is, what else do what want Comodo to do in this case?

I'd like to know whether there are more contractors serving as RA for
Comodo. A list should provided who they are and which measures are taken
for domain validation. What really strikes me is that this case was only
detected by Eddy because of Certstar's spam e-mails.

> They still have some certificates unaccounted for in terms of
> verifying the validation, and certainly I'd like to hear the status
> of that as soon as possible. Beyond that? It's somewhat of an open
> question.

I'd tend to punish a rogue CA by removing their root CA cert from NSS.
Maybe this serves as a good example to other CAs that the Mozilla CA
policy is really enforced. Otherwise nobody will care.

Ciao, Michael.

Kyle Hamilton

unread,
Dec 25, 2008, 8:05:35 PM12/25/08
to mozilla's crypto code discussion list
I've already stated my preference.

To reiterate:

Actually, I think it's very important that the accounting include this:

for each name (not just certificate, but name in
subjectAlternativeNames) that has been certified, a connection to the
TLS ports should be made, and the certificate presented by the site
compared against the certificate that Comodo issued. This obviously
won't be a complete verification, but it should give a start to see
how widespread the problem is.

A script to do this could probably be written fairly easily, but
depending on the number of certificates Comodo has issued that are
currently valid (and I'd like to see some hard numbers on that, as
well) it could take a while to run.

From the script, the numbers I'd like to see are: the number of
unreachable/not-answering names/hosts, the number of matching
certificates, and the number of mismatched certificates. From that
output plus Comodo's records, I would also like to see how many
resellers there are and how many of them have sold mismatched
certificates.

(The 'resellers' I refer to here are 'contracted registration
authorities', not those who make money by funnelling users into
Comodo's pages. I'd also like to know how Robin/Comodo performed the
audit on certificates for proper domain validation -- if they're
relying on the presence or absence of that check-box "I have verified
the domain control in accordance with...", I think the entire audit is
useless and that they should be removed from the root store out of
spite -- for making a mockery of the entire process.)

I do think that Comodo should be required to suspend their
Registration Authority processing until they in-source their
domain-control verification as a condition of staying in Mozilla's
trust list. I also have not heard word 1 about if they use
registration authorites for higher-assurance certificates.

-Kyle H

> _______________________________________________
> dev-tech-crypto mailing list
> dev-tec...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-crypto
>

Kyle Hamilton

unread,
Dec 25, 2008, 8:13:34 PM12/25/08
to mozilla's crypto code discussion list
If Frank's desire to balance user benefit from keeping the root in
with user security by taking the root out is to be upheld, then there
needs to be a way to notify the software user that there is a valid
complaint against the operator of the CA in question.

If it drives business away from the CA in question (because the owner
of the site doesn't want to have to deal with the warning every time
she browses to it using Firefox), well, that's the CA's own fault.
The setting of that bit should encompass the following:

1) A complaint has been made,
2) Mozilla has examined the complaint, and
3) Mozilla has found good cause for believing that the conduct of the
CA has violated its root CA policy.

Thus, such a statement would not (and could not) be made until Mozilla
has done its own due diligence, and thus such a statement would not be
libelous. (I wish that Mozilla's general counsel was on this list.)

-Kyle H

On Thu, Dec 25, 2008 at 3:21 AM, Michael Ströder <mic...@stroeder.com> wrote:
> Eddy Nigg wrote:
>> On 12/23/2008 09:09 AM, Kyle Hamilton:
>>> Of course, this would be an NSS change (the addition of a 'trust
>>> suspended' bit,
>>
>> I think this to be an interesting idea and should be considered.
>
> I really wonder why there should be one state more. And how is it going
> to be set (updated)? A cert is either trusted or not. Period.
>
> Ciao, Michael.

Daniel Veditz

unread,
Dec 25, 2008, 7:13:55 AM12/25/08
to mozilla's crypto code discussion list
Paul Hoffman wrote:
> At 1:16 AM +0200 12/24/08, Eddy Nigg wrote:
>> Select Preferences -> Advanced -> View Certificates -> Authorities.
>> Search for AddTrust AB -> AddTrust External CA Root and click
>> "Edit". Remove all Flags.
>
> Doesn't this seem like a better solution than "sue Mozilla for
> theoretical future damages incurred by using their free software"?
> Put more rudely, why do you expect Daddy to fix it for you when you
> can fix it yourself, more easily and more quickly, if you want to?

Isn't that a bit like an auto maker issuing a notice "If you don't want
your car to explode in a fender bender you can crawl under the right
rear and remove the screw marked 34A on the following diagram."

Everyone should be able to do that so I'm sure that gets the auto maker
off the hook, right? Assuming all 200 million Firefox users even hear
about the problem.

Paul Hoffman

unread,
Dec 25, 2008, 10:14:40 PM12/25/08
to mozilla's crypto code discussion list
At 11:13 PM -0800 12/24/08, Daniel Veditz wrote:
>Paul Hoffman wrote:
>> At 1:16 AM +0200 12/24/08, Eddy Nigg wrote:
>>> Select Preferences -> Advanced -> View Certificates -> Authorities.
>>> Search for AddTrust AB -> AddTrust External CA Root and click
>>> "Edit". Remove all Flags.
>>
>> Doesn't this seem like a better solution than "sue Mozilla for
>> theoretical future damages incurred by using their free software"?
>> Put more rudely, why do you expect Daddy to fix it for you when you
>> can fix it yourself, more easily and more quickly, if you want to?
>
>Isn't that a bit like an auto maker issuing a notice "If you don't want
>your car to explode in a fender bender you can crawl under the right
>rear and remove the screw marked 34A on the following diagram."

It depends on what you mean by "a bit".

I don't remember paying for Firefox. Nor do I remember anything in the software that made any strong assertion of the validity of all the certs that might be issued by any of the CAs in the root pile. Maybe you had a different experience.

--Paul Hoffman

Paul Hoffman

unread,
Dec 25, 2008, 10:24:01 PM12/25/08
to mozilla's crypto code discussion list
At 7:16 PM +0100 12/25/08, Michael Ströder wrote:
>I'd tend to punish a rogue CA by removing their root CA cert from NSS.
>Maybe this serves as a good example to other CAs that the Mozilla CA
>policy is really enforced. Otherwise nobody will care.

This is Firefox we're talking about, not IE. Do you really think that this is going to help end users, or just hurt people who bought certificates from the lax (not rogue) CA?

Like most punishment, the origin is more often the desire of the punisher to feel powerful. In this case, it is also for financial gain by the first one to propose the punishment, of course, but the base desire is the same.

Eddy Nigg

unread,
Dec 25, 2008, 11:17:51 PM12/25/08
to
On 12/25/2008 08:16 PM, Michael Ströder:

>> The question is, what else do what want Comodo to do in this case?
>
> What really strikes me is that this case was only
> detected by Eddy because of Certstar's spam e-mails.
>

Even though I believe that Robin and his crew are really angry with me
right now for disclosing it publicly, this was really the least price
they could pay and best case scenario for this situation. None of the
109 other cert holders suspected that anything might be wrong. I'm
certain that this would not have remained undetected for long and
somebody could have brought some real damage upon all parties involved.

Speaking about Robin, I wouldn't want to be in his shoes right now -
it's more or less one of the nightmares of a CA. On the other hand, if
this is case (it would be for me) than I really anticipate and hope that
some real changes are going to happen. Additionally, whatever actions
are taken against Comodo and whatever lessons they learned, one thing is
clear, that the company which spammed and mislead other CAs customers so
shamelessly has nothing lost in this industry.

Eddy Nigg

unread,
Dec 25, 2008, 11:34:42 PM12/25/08
to
On 12/26/2008 12:24 AM, Paul Hoffman:

> At 7:16 PM +0100 12/25/08, Michael Ströder wrote:
>> I'd tend to punish a rogue CA by removing their root CA cert from NSS.
>> Maybe this serves as a good example to other CAs that the Mozilla CA
>> policy is really enforced. Otherwise nobody will care.
>
> This is Firefox we're talking about, not IE.

Depending on country and audience, Internet Explorer has even less
market share than Firefox. We are in 2009, not 2003 if you forgot.

> Do you really think that this is going to help end users,

In the longer term it might. And it really depends on other factors like
how many potentially other certs could have been issued this way.

> or just hurt people who bought certificates from the lax (not rogue) CA?

So? They may claim damage from Comodo. Comodo even lists the compatible
browsers in their CPS [1] under section 2.1.5 CA Root Public Key
Delivery to Subscribers. A CA shouldn't guaranty browser support at any
time (and I doubt if Comodo really did).

>
> In this case, it is also for financial gain by the first one to propose the punishment, of course, but the base desire is the same.

Do you mean me? Go back and read what I really proposed:
http://groups.google.com/group/mozilla.dev.tech.crypto/msg/fb8c1fbd0c219eb4
But perhaps you'd disclose how many Comodo shares you've got? ;-)

[1]
http://www.comodo.com/repository/09_22_2006_Certification_Practice_Statement_v.3.0.pdf

Michael Ströder

unread,
Dec 25, 2008, 11:36:54 PM12/25/08
to
Paul Hoffman wrote:
> At 7:16 PM +0100 12/25/08, Michael Ströder wrote:
>> I'd tend to punish a rogue CA by removing their root CA cert from NSS.
>> Maybe this serves as a good example to other CAs that the Mozilla CA
>> policy is really enforced. Otherwise nobody will care.
>
> This is Firefox we're talking about, not IE. Do you really think that
> this is going to help end users, or just hurt people who bought
> certificates from the lax (not rogue) CA?

PKI is about security. Strange I have to remind you about that. There is
a Mozilla CA policy which was violated possibly causing a risk for
end-users. Mozilla has to give some evidence to the community and CAs
that the policy is enforced.

> Like most punishment, the origin is more often the desire of the
> punisher to feel powerful. In this case, it is also for financial
> gain by the first one to propose the punishment, of course, but the
> base desire is the same.

Personally I have absolutely no benefit from withdrawing the trust flags
from Comodo's root CA cert. So it seems strange to me that you're
accusing me in such an arrogant way. This does not contribute anything
to this discussion.

Ciao, Michael.

Gen Kanai

unread,
Dec 26, 2008, 1:28:40 AM12/26/08
to mozilla's crypto code discussion list

On Dec 26, 2008, at 1:49 AM, Frank Hecker wrote:

> Beyond that? It's somewhat of an open question.
>
> Frank

Mozilla needs to have a concrete policy and procedures in place so
that there is no question as to what the penalties would be for future
actions of this kind.

I personally like John Nagle's proposal from earlier in this thread:

http://groups.google.com/group/mozilla.dev.tech.crypto/msg/9443ba781a669879

I think there are other actions that need to be taken beyond John
Nagle's proposal, but it is a good start.

In addition to John Nagle's propsosal, I believe Mozilla needs to act
early in 2009, and not let weeks or months go by without resolution.
We can discuss the details of what happened and why endlessly to no
benefit of our users.

Gen

Eddy Nigg

unread,
Dec 26, 2008, 2:00:01 AM12/26/08
to
On 12/26/2008 03:28 AM, Gen Kanai:

> I personally like John Nagle's proposal from earlier in this thread:
>
> http://groups.google.com/group/mozilla.dev.tech.crypto/msg/9443ba781a669879
>

Gen, one thing to note, that Comodo most likely performs a yearly
WebTrust audit, though the last one I can see currently is from the
tenth of July 2007.

Also important to note that the audit itself isn't enough - that's why
there is the Mozilla CA Policy which clearly defines the minimal
requirements. (A CAs can pass a WebTrust audit without conforming to
those requirements set up by Mozilla).

As a matter of fact, we are still missing a procedure to make sure that
CAs issuing EV certificates indeed perform the yearly audit as required
by the EV guidelines. Those which don't, have to have EV status removed
as they wouldn't be in compliance with the EV guidelines.

Additionally, Mozilla has no control directly over certificates issued
through certstar, since the certificates are issued from an intermediate
CA certificate of Comodo. It's however possible and relatively easy to
ADD this intermediate to NSS and deliberately mark it untrusted. It
could be a good solution to prevent damage in case there should be more
certificates in the wild (and assuming that resellers certs are issued
through this intermediate).

Incidentally I've brought up the issue of AddTrust and UserSomething CAs
during the review discussion this year. It isn't exactly surprising that
now all those questionable practices come up again, isn't it?! There
were many more concerns brought up, which had no effect whatsoever on
the status of Comodo and their request to upgrade to EV was eventually
approved.

Ian G

unread,
Dec 26, 2008, 2:07:54 AM12/26/08
to mozilla's crypto code discussion list
On 26/12/08 00:36, Michael Ströder wrote:
> Paul Hoffman wrote:
>> At 7:16 PM +0100 12/25/08, Michael Ströder wrote:
>>> I'd tend to punish a rogue CA by removing their root CA cert from NSS.


I do not see a rogue CA. The evidence of the posts here suggests a flaw
leading to false certs was found and such certs were issued; and that
the CA responded when made aware.

What is rogue about that? Are you saying they didn't respond?


>>> Maybe this serves as a good example to other CAs that the Mozilla CA
>>> policy is really enforced. Otherwise nobody will care.
>> This is Firefox we're talking about, not IE. Do you really think that
>> this is going to help end users, or just hurt people who bought
>> certificates from the lax (not rogue) CA?
>
> PKI is about security.


Security is risks and costs. In this case, there is low risk and zero
costs. Perhaps because the problem was caught early on, but security is
about real hard facts not conjecture.

(If you want real hard costs and losses and grief, check out phishing.
Where's the lynch mob when we are dealing with phishing, I wonder?)


> Strange I have to remind you about that. There is
> a Mozilla CA policy which was violated possibly causing a risk for
> end-users. Mozilla has to give some evidence to the community and CAs
> that the policy is enforced.


But it has! Mozilla talked to the CA. The CA is dealing with it.
There are emails to that extent, posted here.

What else is necessary? And more importantly, why?

iang, curiosity mode switched to hard-ON.

Kyle Hamilton

unread,
Dec 26, 2008, 10:16:08 AM12/26/08
to mozilla's crypto code discussion list
https://bugzilla.mozilla.org/show_bug.cgi?id=426575

UTN-UserFIRST-Hardware is enabled for EV per that bug.

-Kyle H

ro...@comodo.com

unread,
Dec 26, 2008, 11:28:06 AM12/26/08
to
On Dec 25, 4:49 pm, Frank Hecker <hec...@mozillafoundation.org> wrote:
> Michael Ströder wrote:
> > Could you please define a time-frame within Comodo MUST react?
>
> Comodo (in the person of Robin Alden) has already made a reply:
>
> http://groups.google.com/group/mozilla.dev.tech.crypto/msg/b24e70ea2c396bb5
>
> The question is, what else do what want Comodo to do in this case? They
> still have some certificates unaccounted for in terms of verifying the
> validation, and certainly I'd like to hear the status of that as soon as
> possible. Beyond that? It's somewhat of an open question.
>
Frank,
We have finished our initial investigation on the certificates
issued by Certstar.

Of the 111 orders that had been placed through Certstar there remain
13 orders for which we have still not been able to gather adequate
evidence of the applicant having domain control. We have therefore,
regrettably, revoked the certificates on those orders.

Of those 13 orders, 2 were placed by Eddy, for startcom.org and
www.mozilla.com, as he has already described. As we previously
stated, the certificate for www.mozilla.com was revoked shortly after
it was issued.

Of the remaining 11 orders we do not have any reason to believe that
any were placed fraudulently and had we not set such a short timescale
to effect the (re-)validation and had it not been over this Holiday
season we believe that we would have been able to achieve validation
of domain control for all 11.

Certstar have passed to us all of their records for these customers
and we will ensure that they are contacted for 2 purposes:
a) to check if it would have been possible to complete the re-
validation
b) to offer a replacement certificate.
Some of the orders have either been charged-back or refunded. We have
to accept the possibility that some of those customers will not assist
us in re-validation.

Regards
Robin Alden
Comodo

Frank Hecker

unread,
Dec 26, 2008, 3:07:57 PM12/26/08
to
Kyle Hamilton wrote:
> https://bugzilla.mozilla.org/show_bug.cgi?id=426575
>
> UTN-UserFIRST-Hardware is enabled for EV per that bug.

My apologies, you are right and my recollection was wrong.

Eddy Nigg

unread,
Dec 26, 2008, 3:10:56 PM12/26/08
to
On 12/26/2008 01:28 PM, ro...@comodo.com:

> www.mozilla.com, as he has already described. As we previously
> stated, the certificate for www.mozilla.com was revoked shortly after
> it was issued.

It would behoove yourself if you'd stick with the facts at least. You
keep claiming that you detected it and revoked the certificate.

All times in GMT:
- Dec 22 19:47 Certificate issued to mozilla.com
- Dec 22 22:25 The Story broke at dev.tech.crypto
- Dec 22 22:46 Certificate revoked (thanks for not fixing the time in
the CRL)

Additionally I really hope that your agreement with certstar allows you
to sue the h*** out of them. I could have received any certificate for
any domain I'd have been willing to pay $ 45 and nothing would have
prevented that. If that's not clearly acknowledged I'll feel myself
forced to publish more material I have at my disposal. The mail for the
verisign domain was just a small sample.

Since this company uses a domain resembling that of ours which we
operate already for four years and which has a very high ranking at
Google, I'm insisting that this company closes their web site. We
received already calls from people confusing us with them.

- *certstar.com* as opposed to *cert.startcom*.org

Also in light of the damage caused to us and other certification
authorities due to their attempts to mislead our customers, it's the
least to have them shut down immediately. Failing to do so will make it
only worse.

You also claimed that this was an anomaly and that the validation
mechanism was bypassed by accident. If this is the case would you agree
with me that such events are very unlikely to reoccur? If that's
correct, whatever actions or recommendations are decided in relation to
*this* event, I will suggest to Frank that any re-occurrence would have
to have appropriate consequences. If it's not correct, I expect you from
notifying about the actions you are performing within a reasonable
time-frame to prevent such re-occurrence.

Paul C. Bryan

unread,
Dec 26, 2008, 7:29:49 PM12/26/08
to
Dear Robin:

You have not yet responded to my questions. I believe they are
reasonable. Will you answer them in this forum?

Yours truly,

Paul C. Bryan

ro...@comodo.com

unread,
Dec 26, 2008, 9:10:33 PM12/26/08
to
On Dec 24, 2:13 am, "Paul C. Bryan" <em...@pbryan.net> wrote:
> On Dec 23, 5:56 pm, ro...@comodo.com wrote:
> Some questions:
>
> 1. Does Comodo take full responsibility for the actions of its
> resellers? If so, how should the repercussions of such failures be to
> Comodo?
Comodo accepts responsibility for the work of its RAs in the
validation that they do leading to the issuance of certificates under
our root certificates.

>
> 2. Are resellers subject to the same audits that Comodo presumably had
> to undergo to get its root certs added to Mozilla? Who performs, and
> who verifies such audits? How often are they performed?
No, the RAs are not subject to the same audits as Comodo. Comodo
undergoes an annual external audit to maintain our Webtrust
certification for CAs.
http://www.cica.ca/download.cfm?ci_id=45239&la_id=1&re_id=0
https://cert.webtrust.org/ViewSeal?id=804

>
> 3. Are you willing to openly, continually disclose your list of
> resellers, the frequency of audits, audit methodology, and actual
> audit reports so that third parties can evaluate whether Comodo is
> trustworthy as a CA?
That is a question combined with an assertion.
To the question: on a unilateral basis, no, Comodo wouldn't reveal
that level of detail of our internal operation. If all CAs were
required to provide the information, either to retain Webtrust
certification or to gain or retain access to the root program of a
major browser or other platform, then we would reconsider.
To the assertion that this is a pre-requisite for a CA to be
trustworthy: I am not aware that it is Mozilla's policy to require
this information to be disclosed.

Kyle Hamilton

unread,
Dec 26, 2008, 9:38:48 PM12/26/08
to mozilla's crypto code discussion list
See, Robin, my thought is this:

You've already shown that it's possible for the RA function to bypass
all controls. At this point, because they're not subject to the same
audits that Comodo is, and because the last WebTrust audit that anyone
here can find any record of is in 2007, I find it difficult to believe
that you have the "annual external audit".

The internal controls are supposed to prevent this kind of
mis-issuance. Because they didn't, they throw all the audits that you
have provided into doubt. Because of this, there is no trust that I
have in Comodo's operation.

Further, this is a problematic practice (delegation of Registration
Authority function, as opposed to simple "reseller" role) that has
been shown to cast doubt on the entire domain.

The Registration Authority function is terribly security-sensitive.
If the whistle had not been blown by someone who knew where to post to
kick the beehive, would this have been detected? Since the RAs aren't
audited (which is, by the way, a TERRIBLY dangerous practice, as
you're seeing), and your statements about "a representative sample of
certificate requests are reviewed" suggesting that they're not even
properly audited by your internal controls...

It is not necessarily a requirement for reseller information to be
disclosed. However, we're trying to evaluate your company's continued
trustworthiness, and (at least at the moment) I can't find anything
there to trust. I'm willing to allow your eleven roots to stay in the
root store with trust bits removed until you provide documentation and
an update to your agreement with your RAs to require on-site audits at
least annually (even if done by your internal auditors) -- the only
alternative at this point is to completely remove your roots from the
program.

I would like to know how you're going about ensuring that none of your
other RAs are subject to the same 'glitch' in their signup processes.
I'd like to hear that you're being proactive about this issue.

Unfortunately, I'm not hearing such.

-Kyle H

Eddy Nigg

unread,
Dec 26, 2008, 9:52:13 PM12/26/08
to
On 12/26/2008 11:38 PM, Kyle Hamilton:

> You've already shown that it's possible for the RA function to bypass
> all controls. At this point, because they're not subject to the same
> audits that Comodo is, and because the last WebTrust audit that anyone
> here can find any record of is in 2007, I find it difficult to believe
> that you have the "annual external audit".

Robin just gave us the link for the current WebTrust report. As I
mentioned previously Comodo does perform a yearly audit. However, since
no directory exists at WebTrust, the last audit I could find was the one
published at the "Pending" page.

Here the new link: https://cert.webtrust.org/SealFile?seal=804&file=pdf

Kyle Hamilton

unread,
Dec 26, 2008, 10:07:43 PM12/26/08
to mozilla's crypto code discussion list
On Fri, Dec 26, 2008 at 1:52 PM, Eddy Nigg <eddy...@startcom.org> wrote:
> On 12/26/2008 11:38 PM, Kyle Hamilton:
>>
>> You've already shown that it's possible for the RA function to bypass
>> all controls. At this point, because they're not subject to the same
>> audits that Comodo is, and because the last WebTrust audit that anyone
>> here can find any record of is in 2007, I find it difficult to believe
>> that you have the "annual external audit".
>
> Robin just gave us the link for the current WebTrust report. As I mentioned
> previously Comodo does perform a yearly audit. However, since no directory
> exists at WebTrust, the last audit I could find was the one published at the
> "Pending" page.
>
> Here the new link: https://cert.webtrust.org/SealFile?seal=804&file=pdf

My apologies, I didn't see this. I retract my disbelief.

Paul C. Bryan

unread,
Dec 26, 2008, 10:18:53 PM12/26/08
to
Thanks for your response Robin.

On Dec 26, 1:10 pm, ro...@comodo.com wrote:

> Comodo accepts responsibility for the work of its RAs in the
> validation that they do leading to the issuance of certificates under
> our root certificates.

You failed to answer the other half of this question. What should the
repercussions of such failures as this be for Comodo? Simply hoping
you follow-up with your resellers (as has so far been the case with
Certstar) is not an acceptable remedy in my opinion.

> No, the RAs are not subject to the same audits as Comodo.  Comodo
> undergoes an annual external audit to maintain our Webtrust
> certification for CAs.

How can the results of the Comodo audits be considered valid if Comodo
outsources portions of its functions to third parties, that are not
subject to the same audits?

> http://www.cica.ca/download.cfm?ci_id=45239&la_id=1&re_id=0https://cert.webtrust.org/ViewSeal?id=804

This link responds with an error result.

> That is a question combined with an assertion.

Indeed, which I'll address below.

> To the question: on a unilateral basis, no, Comodo wouldn't reveal
> that level of detail of our internal operation.  If all CAs were
> required to provide the information, either to retain Webtrust
> certification or to gain or retain access to the root program of a
> major browser or other platform, then we would reconsider.

As I have mentioned in previous postings, a trust chain is only as
strong as its weakest link. Comodo has added extra links in its chain,
in the form of resellers whom it trusts to peform DV. If those links
in the chain are not disclosed, and not subject to the same audits as
the party applying for trust certification, then the integrity of the
chain cannot be established. I expect that no other CAs are delegating
their RA/DV functionality to third parties. If they are, then they're
in the same boat as Comodo.

> To the assertion that this is a pre-requisite for a CA to be
> trustworthy: I am not aware that it is Mozilla's policy to require
> this information to be disclosed.

I can't see how a CA can be considered trustworthy by anyone if it
outsources portions of its core operations to undisclosed parties, and
those parties are not subject to the same criteria, inspection and
audits as the CA itself.

Paul C. Bryan

unread,
Dec 26, 2008, 10:23:11 PM12/26/08
to
On Dec 26, 2:18 pm, "Paul C. Bryan" <em...@pbryan.net> wrote:

> This link responds with an error result.

Apologies. Disregard my statement about the link error. I realized
it's two links. I will now go drink some more coffee to increase my
alertness level.

Ian G

unread,
Dec 26, 2008, 10:54:51 PM12/26/08
to mozilla's crypto code discussion list
On 26/12/08 22:38, Kyle Hamilton wrote:
> See, Robin, my thought is this:
>
> You've already shown that it's possible for the RA function to bypass
> all controls. At this point, because they're not subject to the same
> audits that Comodo is, and because the last WebTrust audit that anyone
> here can find any record of is in 2007, I find it difficult to believe
> that you have the "annual external audit".
>
> The internal controls are supposed to prevent this kind of
> mis-issuance. Because they didn't, they throw all the audits that you
> have provided into doubt. Because of this, there is no trust that I
> have in Comodo's operation.

The internal controls are not supposed to "prevent mis-issuance". This
is a gross consumer simplification, and has no place here. The controls
are meant to reduce the likelihood of them, make them discoverable, and
deal with them when they happen.

We can no more "prevent" bad certs than we can stop the winter from
coming. The point is to put in place economically reasonable policies
and practices that meet an appropriate balance of security versus cost.

If there has been a case where a particular instance has swayed and
delivered too much convenience, for too high a security risk, then the
systems will deal with it.

So far the systems are dealing with it. Check the facts: CA was
notified. Reseller frozen. Certs revoked. Internal audits are
checking. External audit might get involved. This is what the systems
are supposed to do.

To all: Although we might in other contexts promote the use of open
forums for open governance purposes -- analysis and discussion of the
properties of providers by open parties -- *this public lynching is not
that*.

It is neither informed, nor professional.

Mozilla runs a process where there is a one week period of public
scrutiny of a CA. During that time, we could reasonably argue that
people here are invited to state their fears. We might consider
discussions to be more "priviledged" such as in parliament.

However, outside that week, there is no such protection. Where people
in this group have crossed the line, and made actionable statements,
and/or done actionable harm to a business or individual, they should
note: it is unlikely that Mozilla, or the community, or the businesses
as a whole will, can or should protect them. And where corporates are
forced to be quiet for fear of reputational damage, then it is up to the
rest of us to seek professionalism and self-governance.

The process of recovery from this hack is not an open nor public
process. CAs, as businesses, and audits, as governance are not
generally public affairs.

If you wish to advance these into the open, by all means do, but first,
establish a policy and a practice. Let's establish guidelines on
reasonable behaviour so that criticism can be seen in a narrow context,
and can be protected and informed.

Elsewise, it is unbalanced. You can talk unprofessionally, but others
are forced to remain silent. Any comments are wasted, discussion is
fruitless at best, and at worst, the mob will have their way.

iang

Ian G

unread,
Dec 26, 2008, 11:12:39 PM12/26/08
to mozilla's crypto code discussion list
On 26/12/08 02:28, Gen Kanai wrote:
>
> On Dec 26, 2008, at 1:49 AM, Frank Hecker wrote:
>
>> Beyond that? It's somewhat of an open question.
>>
>> Frank
>
> Mozilla needs to have a concrete policy and procedures in place so that
> there is no question as to what the penalties would be for future
> actions of this kind.


Penalties ... tough talk, but what does it really mean?

Basically, all that a vendor can do is to drop the root. (Ok, we can
fiddle with the trust bits, but it seems a little but like fiddling to me.)

In short, it is DROP or NIX. Can we say, "blunt weapon" ? Either the
vendor is small, so it matters not, or the vendor is huge, and it
matters a great deal. (In that latter case, it then matters a great
deal to the CA. It could be a deal killer. E.g., bankrupcy. Which
means, Mozilla has to get that *right* or it faces another issue,
further downstream. Deep pockets plus aggressive liquidator equals
not-nice maths.)

How does Mozo make the "right" decision here ? Part of the problem in
making it "right" whatever that means is that according to classical
browser PKI it is not the responsibility of Mozo or any other vendor to
do anything, let alone deciding what "right" is.

Classically, this is the policy, in a nutshell:

CA sets up.
CA gets audited.
some technical things are checked...
root is added.

It is that second part that is the clue: the audit. It is the audit's
area to check whether the CA is following some sort of policy or
practice or compliance.

So, if there is a failure, the first question to ask is whether this the
failure is in the Audit's responsibility, or whether it is a vendor
issue? It might be one, or the other, or BOTH. Certainly, in the
current case, the vendor does not have the information to make a
decision, whereas the Auditor might reasonably, having been in there and
kicked the tires?


(Although I think, it is a singular observation: there is no effective
dispute resolution for this case or any other. What does that say?)

iang

Kyle Hamilton

unread,
Dec 26, 2008, 11:15:41 PM12/26/08
to mozilla's crypto code discussion list
On Fri, Dec 26, 2008 at 3:12 PM, Ian G <ia...@iang.org> wrote:
> (Although I think, it is a singular observation: there is no effective
> dispute resolution for this case or any other. What does that say?)

That there is no reason to trust a system without dispute resolution procedures.

-Kyle H

Ian G

unread,
Dec 26, 2008, 11:39:34 PM12/26/08
to mozilla's crypto code discussion list


I would be somewhat sympathetic to this .... but it is kind of damning
to the entire system.

Question is, is there a way of creating a dispute resolution system that
would deal with the entire problem space?

Let's say you and I are in dispute, and the damages are N bucks. We
need someone to tell us who did what, and who does what, and who pays what.

We could envisage a forum of dispute resolution where we both agree to
enter because we are "inside this space" already. And agree to those
liabilities. That might work between us, but it doesn't scale.

E.g., it can't apply easily to a Firefox end-user who hasn't agreed to
this forum. So, if a Firefox user was ripped off because of a cert,
then she would be shut out. Alternatively, if she libelled a CA by
claiming insecurities in a public blog, the CA would not be able to get
satisfaction because she wouldn't recognise the forum.

However, maybe a partial solution is better than none? Anything might
be better than what we've got now...

iang

Eddy Nigg

unread,
Dec 26, 2008, 11:53:56 PM12/26/08
to
On 12/27/2008 12:54 AM, Ian G:

>
> We can no more "prevent" bad certs than we can stop the winter from
> coming. The point is to put in place economically reasonable policies
> and practices that meet an appropriate balance of security versus cost.
>

Yeah right! It really depends what the right balance is, ehhh?!

> So far the systems are dealing with it. Check the facts: CA was
> notified. Reseller frozen. Certs revoked. Internal audits are checking.
> External audit might get involved. This is what the systems are supposed
> to do.
>

The story starts before that. You are just seeing the tail, I'm seeing
what preceded to that - or better, what did not happen and should have.

That's not up to an internal audit as if it were a well hidden bug in
one of Comodo's system that somebody succeeded to crack. But maybe Robin
can explain to us which failures happened at their side as they are
taking supervision of RAs and resellers very seriously. But that's most
likely something which we'll never know.

> However, outside that week, there is no such protection. Where people in
> this group have crossed the line, and made actionable statements, and/or
> done actionable harm to a business or individual, they should note: it
> is unlikely that Mozilla, or the community, or the businesses as a whole
> will, can or should protect them.

Are you speaking in the name of Mozilla? Or in the name of the
community? Or in the name of which business exactly?

Ian G

unread,
Dec 27, 2008, 12:40:36 AM12/27/08
to mozilla's crypto code discussion list
On 27/12/08 00:53, Eddy Nigg wrote:
> On 12/27/2008 12:54 AM, Ian G:
>>
>> We can no more "prevent" bad certs than we can stop the winter from
>> coming. The point is to put in place economically reasonable policies
>> and practices that meet an appropriate balance of security versus cost.
>>
>
> Yeah right! It really depends what the right balance is, ehhh?!


There is no "right balance" just like there is no world peace. Security
is an economic phenomena, not a beauty pageant.


>> So far the systems are dealing with it. Check the facts: CA was
>> notified. Reseller frozen. Certs revoked. Internal audits are checking.
>> External audit might get involved. This is what the systems are supposed
>> to do.
>>
>
> The story starts before that. You are just seeing the tail, I'm seeing
> what preceded to that - or better, what did not happen and should have.


That "earlier story" has no real place here, IMHO. This is a forum for
the discussion of technical, crypto, root and general PKI issues, by
either dictat or convention. It is not a forum for the airing of
general business complaints.

https://lists.mozilla.org/listinfo/dev-tech-crypto

I elsewhere mentioned there is no general mechanism for dispute
resolution, your "earlier story" might be a case in point. Or might
not. Either way, here is not the place to grumble about practices of
other businesses.


>> However, outside that week, there is no such protection. Where people in
>> this group have crossed the line, and made actionable statements, and/or
>> done actionable harm to a business or individual, they should note: it
>> is unlikely that Mozilla, or the community, or the businesses as a whole
>> will, can or should protect them.
>
> Are you speaking in the name of Mozilla? Or in the name of the
> community? Or in the name of which business exactly?


Having appreciated this point, a more interesting one is whether we as a
community think about opening up the processes for more open governance,
more open scrutiny, more stakeholder checking [1].

There seems to be an emerging consensus that more open is more better,
in general at least.

Would we be in a position to explore a general opening of all auditing
investigations and controls [2] ?

E.g., where Comodo or any CA completes an internal audit and creates a
report to document that audit action, could we expect the CA or the
internal auditor to publish this as a routine action?

iang

[1] My thanks to Robin for underscoring that observation! I had to
kick myself for failing to see it.

[2] Plausibly, such a proposal will not be accepted in time for the
current case to be effected, but that's fine as this is a forum of
improving processes, not dispute resolution.

Paul C. Bryan

unread,
Dec 27, 2008, 1:21:33 AM12/27/08
to
On Dec 26, 4:40 pm, Ian G <i...@iang.org> wrote:

With respect:

> This is a forum for the discussion of technical, crypto, root and general PKI
> issues, by either dictat or convention.  It is not a forum for the airing of general
> business complaints.

Are you characterizing this issue as merely a general business
complaint?

I happen to agree that this should not be the forum for such
discussion, but as you point out, there is no other apparent forum for
dispute resolution. Where should such discussions be taken?

> Having appreciated this point, a more interesting one is whether we as a
> community think about opening up the processes for more open governance,
> more open scrutiny, more stakeholder checking [1].

My first reaction would be: most definitely, +1.

That said, you have classified this discussion as a public lynching of
a CA, called into question the professionalism of those engaging in
such discussion, and identified potential legal liabilities in doing
so.

My question to you would be: assuming the issues you raised are
legitimate (I happen to disagree) how could such an open governance
model cope with the restrictions that would most certainly be
necessary to address the issues you raised?

> Would we be in a position to explore a general opening of all auditing
> investigations and controls [2] ?

+1

Eddy Nigg

unread,
Dec 27, 2008, 1:22:22 AM12/27/08
to
On 12/27/2008 02:40 AM, Ian G:

> On 27/12/08 00:53, Eddy Nigg wrote:
>>
>> Yeah right! It really depends what the right balance is, ehhh?!
>
>
> There is no "right balance" just like there is no world peace. Security
> is an economic phenomena, not a beauty pageant.
>

No, security is an inconvenience, but I've said that earlier already.

>>
>> The story starts before that. You are just seeing the tail, I'm seeing
>> what preceded to that - or better, what did not happen and should have.
>
>
> That "earlier story" has no real place here, IMHO. This is a forum for
> the discussion of technical, crypto, root and general PKI issues, by
> either dictat or convention. It is not a forum for the airing of general
> business complaints.

You don't seem to get it, do you? The story starts before your stating
of the facts you would like us to believe. The story starts with putting
resellers and so-called RAs in charge of validation procedures they have
no clue about and with failing to audit, approving and controlling them,
it's called due diligence. The story continues with failing to prevent
issuance of high-profile target certificates such as Mozilla certainly
is and the story continues with failing to detect them once issued. As I
said, you are only seeing the tail...

>
> There seems to be an emerging consensus that more open is more better,
> in general at least.
>

This might be correct. However generally speaking CP and CP statements,
other documents publicly available in addition to general questioning
(at least during our review procedures at Mozilla) are fairly sufficient.

In relation to Comodo, the writing has been on the wall.

> E.g., where Comodo or any CA completes an internal audit and creates a
> report to document that audit action, could we expect the CA or the
> internal auditor to publish this as a routine action?
>

I don't think we can expect that as a general role.

Eddy Nigg

unread,
Dec 27, 2008, 1:31:56 AM12/27/08
to
On 12/27/2008 03:22 AM, Eddy Nigg:

>
> You don't seem to get it, do you? The story starts before your stating
> of the facts you would like us to believe. The story starts with putting
> resellers and so-called RAs in charge of validation procedures they have
> no clue about and with failing to audit, approving and controlling them,
> it's called due diligence. The story continues with failing to prevent
> issuance of high-profile target certificates such as Mozilla certainly
> is and the story continues with failing to detect them once issued. As I
> said, you are only seeing the tail...
>

And listen up! What's the difference between storing the private key of
mozilla.com at an Internet facing web server [1] and that of having a
so-called RA turned reseller performing [or not performing] domain
validation at a Servage hosting account?

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c13

Nelson B Bolyard

unread,
Dec 27, 2008, 1:38:49 AM12/27/08
to mozilla's crypto code discussion list

Robin, Thanks for that report.

Speaking for myself only, I think that the speed with which you and Comodo
dealt with this issue, once it became publicly known, and the integrity
you & Comodo showed by revoking 11 certs (~10%) whose veracity simply could
not be determined in a timely fashion, is commendable.

Issues remain, and will continue to be discussed about how this situation
came to be, and what new measures should be taken, and by whom, to minimize
the probability of it ever happening again. But one clear conclusion from
this episode is that there is not a single clear line of separation of
responsibilities between CAs and RAs that applies to all CAs. Clearly
several participants in this discussion were surprised that a CA would
delegate the duty of validating domain control to an RA, and some opined
that a CA ought to perform that duty itself. I'm not convinced that's
necessary, but it certainly does seem that a CA firm ought to be prepared
to deal with the possibility that an RA makes a (potentially big) mistake
without sacrificing the CA firm's entire business. The challenge, in the
event of an RA error, is to restore/maintain confidence in the integrity
of the CA's PKI overall, while mitigating the potential damage from dubious
enrollments.

In this case, it was feasible to examine the data for ~90% of the RA's 111
enrollments in a reasonably short time. If the RA had enrolled 10 or 100
times as many, a much larger number (and probably a larger percentage) of
the enrollments might not have been verifiable in such a short time.
I wonder if it would have been perceived as feasible to revoke all the
unverified certs in such a case, and still remain in business.

I personally received numerous requests (mostly privately) asking for ways
for browser users to effectively disable the trust for the certs issued
under the auspices of this particular RA, at least until the veracity of
those certs could be sorted out. As you and I discussed in bug 470897, the
only options available to users, which would effectively distrust the entire
PositiveSSL CA cert (or the entire root with all its subordinate CAs), would
have also effectively distrusted the certs issued under the auspices of
numerous other RAs. Hence that solution would not have been a good fit for
the scope of the problem. Nevertheless, a number of people
expressed to me the view that disabling trust in the subordinate CA that
issued certs for that RA, while too broad of a measure, was nonetheless
preferable to leaving themselves vulnerable to the possibility of false
certificates. I sensed that they wanted the ability to take action that fit
the scope of the potential problem well, and also was potentially
reversible if and when things were finally sorted out (as it now seems they
are). Had there been a separate CA issuing certs for this RA, the ability
to disable trust for that CA cert and the ability to restore that trust
at a later time (such as now) would have been much more satisfying to many,
I believe.

So, I would like to suggest that Comodo consider modifying its practices
somewhat, to reduce the mismatch of scope between subordinate CAs and RAs.
I suggest that Comodo operate a separate subordinate CA for each RA to
whom Comodo has delegated validation duties. I suggest that a new
subordinate CA be created for each such RA, and that all new certs issued
for those RAs be issued from those new single-RA CAs. I am aware of at
least one other commercial CA that operates that way, operating a separate
subordinate CA for each RA to whom they have delegated validation duties.
I believe that is a sound way to minimize the "collateral damage" that
might need to be inflicted, even temporarily, to restore/maintain PKI
integrity in the event of a breach.

This is only my view. I look forward to Frank's thoughts on this subject.

Regards,
/Nelson Bolyard

Kyle Hamilton

unread,
Dec 27, 2008, 1:58:50 AM12/27/08
to mozilla's crypto code discussion list
I am minded of the CRL entry reason "remove from CRL". Does NSS
properly handle that reason-code?

If so, a temporary revocation of all unknown certificates might be a
sound practice, removing them from the CRL as they're found and
verified.

We are running up against problems that are caused by absolute
adherence to inflexible standards, and we're proposing mechanisms
inside of the inflexible standards to deal with them.

If there were a way to, say, have the CA include a unique, opaque code
for the RA that submitted a CSR in the issued certificate, AND if
there were a way to filter based on the content of an extension
(rather than simply leaving it completely opaque, which leads to the
current problem), then there would be no need for a separate CA for
each RA.

-Kyle H

On Fri, Dec 26, 2008 at 5:38 PM, Nelson B Bolyard <nel...@bolyard.me> wrote:
> ro...@comodo.com wrote, On 2008-12-26 03:28:
>

Paul C. Bryan

unread,
Dec 27, 2008, 3:47:58 AM12/27/08
to
On Dec 26, 5:38 pm, Nelson B Bolyard <nel...@bolyard.me> wrote:

> Clearly several participants in this discussion were surprised that a CA would
> delegate the duty of validating domain control to an RA, and some opined
> that a CA ought to perform that duty itself.

I certainly fall in that category.

> I'm not convinced that's necessary, but it certainly does seem that a CA firm
> ought to be prepared to deal with the possibility that an RA makes a (potentially
> big) mistake without sacrificing the CA firm's entire business. The challenge, in
> the event of an RA error, is to restore/maintain confidence in the integrity
> of the CA's PKI overall, while mitigating the potential damage from dubious
> enrollments.

I think I can boil down my concern in this statement:

When trust is being established in a certification authority, trust is
explicitly being placed in its operational practices. It is not being
trusted in its ability to place trust in turn in whomever it may
decide to outsource its operations. By allowing arbitrary parties to
perform critical RA activities (such as DV) the CA is attempting to
extend its operations beyond that which was originally judged.

> So, I would like to suggest that Comodo consider modifying its practices
> somewhat, to reduce the mismatch of scope between subordinate CAs and RAs.
> I suggest that Comodo operate a separate subordinate CA for each RA to
> whom Comodo has delegated validation duties.  I suggest that a new
> subordinate CA be created for each such RA, and that all new certs issued
> for those RAs be issued from those new single-RA CAs.  I am aware of at
> least one other commercial CA that operates that way, operating a separate
> subordinate CA for each RA to whom they have delegated validation duties.
> I believe that is a sound way to minimize the "collateral damage" that
> might need to be inflicted, even temporarily, to restore/maintain PKI
> integrity in the event of a breach.

I believe your suggestion is valid. This seems to fit s. 13 of the
Mozilla CA Certificate policy: "... we recommend that CAs consider
using separate root CA certificates with separate public keys (or
separate intermediate CA certificates with separate public keys under
a single root) when issuing certificates according to different
Certificate Policies, so that we or others may selectively enable or
disable acceptance of certificates issued according to a particular
policy, or may otherwise treat such certificates differently ..."

I believe another valid option would be for the CA to incorporate key
RA duties, namely domain verification. The CA can still have resellers
that initiate registration and collect information. Verification would
remain within the operations of that which is judged in the CA's
conformance to policy.

Ian G

unread,
Dec 27, 2008, 12:16:22 PM12/27/08
to mozilla's crypto code discussion list
On 27/12/08 04:47, Paul C. Bryan wrote:
> On Dec 26, 5:38 pm, Nelson B Bolyard<nel...@bolyard.me> wrote:
>
>> Clearly several participants in this discussion were surprised that a CA would
>> delegate the duty of validating domain control to an RA, and some opined
>> that a CA ought to perform that duty itself.
>
> I certainly fall in that category.


Curious. I thought it was the standard business model of many CAs to
outsource some or all of the functions to the reseller or customer.
Indeed, this is the "Verisign buyout model"; outsource something new,
get huge, get bought out by Verisign.

Has anyone done a survey about how this is done in all CAs?


>> I'm not convinced that's necessary, but it certainly does seem that a CA firm
>> ought to be prepared to deal with the possibility that an RA makes a (potentially
>> big) mistake without sacrificing the CA firm's entire business. The challenge, in
>> the event of an RA error, is to restore/maintain confidence in the integrity
>> of the CA's PKI overall, while mitigating the potential damage from dubious
>> enrollments.
>
> I think I can boil down my concern in this statement:
>
> When trust is being established in a certification authority, trust is
> explicitly being placed in its operational practices. It is not being
> trusted in its ability to place trust in turn in whomever it may
> decide to outsource its operations. By allowing arbitrary parties to
> perform critical RA activities (such as DV) the CA is attempting to
> extend its operations beyond that which was originally judged.


Hold on, that is leaping to far...

All businesses place their all trust in "outsourced" entites. They are
called "employees" and "suppliers" and various other names. This is normal.

However, when a contract is signed or an agreement otherwise reached
(and the latter is more the case here), the responsibility for the
agreement remains in the primary party. Again, this is normal.

Nothing has changed here in the CA business. The CA has reached
agreement on its offered CPS, and the assumptions should hold:

the CPS describes what it does.
the CA takes the responsibility.

It doesn't matter in business principle whether it outsources a function
to a reseller, to its employees or to the government.

Is there a criteria anywhere that says or implies "The CA has not
outsourced critical function X to an external agent?" Can anyone recall
such a statment? This would be a controversial criteria because we know
that a popular incentive is to generate opportunities for business revenues.


>> So, I would like to suggest that Comodo consider modifying its practices
>> somewhat, to reduce the mismatch of scope between subordinate CAs and RAs.
>> I suggest that Comodo operate a separate subordinate CA for each RA to
>> whom Comodo has delegated validation duties. I suggest that a new
>> subordinate CA be created for each such RA, and that all new certs issued
>> for those RAs be issued from those new single-RA CAs. I am aware of at
>> least one other commercial CA that operates that way, operating a separate
>> subordinate CA for each RA to whom they have delegated validation duties.
>> I believe that is a sound way to minimize the "collateral damage" that
>> might need to be inflicted, even temporarily, to restore/maintain PKI
>> integrity in the event of a breach.
>
> I believe your suggestion is valid. This seems to fit s. 13 of the
> Mozilla CA Certificate policy: "... we recommend that CAs consider
> using separate root CA certificates with separate public keys (or
> separate intermediate CA certificates with separate public keys under
> a single root) when issuing certificates according to different
> Certificate Policies, so that we or others may selectively enable or
> disable acceptance of certificates issued according to a particular
> policy, or may otherwise treat such certificates differently ..."
>
> I believe another valid option would be for the CA to incorporate key
> RA duties, namely domain verification. The CA can still have resellers
> that initiate registration and collect information. Verification would
> remain within the operations of that which is judged in the CA's
> conformance to policy.

As advice this would remain fine and standard. However trying to create
some sort of restriction on how these things are done is likely to close
of opportunities to do it better another way, in the future.


iang

Gervase Markham

unread,
Dec 27, 2008, 12:32:56 PM12/27/08
to
Michael Ströder wrote:
> Given the large amount of self-generated server certs this problem
> already exists.

Large number != large % of visits. A million Joe Publics might use the
Internet for 5 years to do their online shopping without once
encountering a self-signed cert or a certificate error. Geeks, on the
other hand, encounter them the first time they visit some major Bugzilla
installations.

Gerv

Gervase Markham

unread,
Dec 27, 2008, 12:34:11 PM12/27/08
to
sayrer wrote:
> The truth is that we are basically unable to act without a lot of
> collateral damage. We should keep this in mind with future security
> technology. Relying on companies willing to take money for doing
> absolutely nothing (not even the bare minimum they agreed to) is not a
> pleasant thing to do to our users. We didn't learn this lesson with
> EV--maybe next time! :)

One of the points of EV was to allow us to act against a CA without
massive collateral damage. We can remove EV status from a root without
disabling the root entirely.

Gerv

Gervase Markham

unread,
Dec 27, 2008, 12:39:41 PM12/27/08
to
Dan Colascione wrote:
> Frankly, that's even *more* disturbing. It means that there are almost
> certainly unverified certificates in the wild, and that this problem
> is pervasive.

You mean, you wouldn't be disturbed at all if Comodo had done loads of
auditing and found absolutely no problems whatsoever?

Gerv

Eddy Nigg

unread,
Dec 27, 2008, 12:43:31 PM12/27/08
to
On 12/27/2008 02:16 PM, Ian G:

> Indeed, this is the "Verisign buyout model"; outsource something new,
> get huge, get bought out by Verisign.

What has that to do exactly with what Paul agreed to?

> It doesn't matter in business principle whether it outsources a function
> to a reseller, to its employees or to the government.

Of course it does. Besides that an employee isn't outsourcing, he is
part of the company. Or one might ask, why are certain functions never
outsourced to a third party? Or perhaps lets start to outsource the CA
root key responsibilities as well then...

>
> Is there a criteria anywhere that says or implies "The CA has not
> outsourced critical function X to an external agent?" Can anyone recall
> such a statment?

Yes, the some extend Mozilla does that already today with the
"Problematic Practices". For example auditing of intermediate CAs
shouldn't be outsourced from the auditor to the CA (it's just the other
way around).

And if there is no such criteria we might still create and adopt it.
This is no precedence, there are other criterion already.

> that a popular incentive is to generate opportunities for business
> revenues.

So? Mozilla really shouldn't care about the business revenues of some
CAs. How is that relevant?

> As advice this would remain fine and standard. However trying to create
> some sort of restriction on how these things are done is likely to close
> of opportunities to do it better another way, in the future.
>

I think what Paul suggested is exactly what any responsible CA should
do. I believe most do exactly that today. Specially in light that it's
a core requirement of the Mozilla CA policy.

Gervase Markham

unread,
Dec 27, 2008, 12:49:12 PM12/27/08
to
Kyle Hamilton wrote:
> How would this work? Split nssckbi out to be managed by the non-code
> module owner, though a coder would need to be enlisted to finalize the
> decisions made by that person?

Non-code module arrangements don't require code changes.

As I understand it, having a "CA Certificate Module" was one of the
example test cases used in the setting up of the non-code module owners
system.
https://wiki.mozilla.org/Modules_Scratchpad

However, it doesn't seem to have made it to this list yet:
https://wiki.mozilla.org/Module_Owners_Activities_Modules

Regardless, Frank is the de facto owner of the module that doesn't exist
yet.

Gerv

Kyle Hamilton

unread,
Dec 27, 2008, 12:50:02 PM12/27/08
to mozilla's crypto code discussion list
I'll also mention that these CAs are supposed to be audited to
"financial services" levels. The root that it chains to is
EV-enabled.

The fact that audits didn't pick up on the discrepancies that Eddy
found between Comodo's CP/CPS and Robin's statements suggests that
Comodo's playing dirty pool, and Frank's letting them get away with
it.

-Kyle H

Eddy Nigg

unread,
Dec 27, 2008, 12:52:14 PM12/27/08
to
On 12/27/2008 02:34 PM, Gervase Markham:

> One of the points of EV was to allow us to act against a CA without
> massive collateral damage. We can remove EV status from a root without
> disabling the root entirely.

Which unfortunately isn't really effective for the issue we are facing
today. Removing EV status would be applicable in case the EV guidelines
wouldn't be fulfilled by a CA. It's absolutely useless otherwise. Or
would you suggest that because a CA doesn't perform its duties for
regular certs to disable EV, even though their EV business practices are
in complete compliance with the EV guidelines?

I think the opposite should be explored more clearly. Disable a root
except in case it's an EV cert. Think about it...

Eddy Nigg

unread,
Dec 27, 2008, 12:57:18 PM12/27/08
to
On 12/27/2008 02:49 PM, Gervase Markham:

> However, it doesn't seem to have made it to this list yet:
> https://wiki.mozilla.org/Module_Owners_Activities_Modules
>

Mitchell and others are on the verge of creating this module now upon
request from Nelson, see mozilla.governance.

Gervase Markham

unread,
Dec 27, 2008, 1:07:40 PM12/27/08
to
Hi John,

You raise some important questions, but it's worth having clarity on a
few matters of fact.

John Nagle wrote:
> 1. AddTrust, a company which apparently no longer exists, has an
> approved
> root CA certificate. This in itself is troublesome.

This is extremely common. Certificates change hands. Failing to honour
root certificates which are no longer owned by the companies which
created them would break a significant proportion of the web. Microsoft
does not have a policy preventing this.

The previous are statements of fact; at this point I express no opinion
as to whether those facts are a good state of affairs.

> Comodo does
> not seem to have taken on the obligations of AddTrust; see
> "http://markmail.org/message/3zr4e5hxwmxjbgnp?q=Comodo+AddTrust".

That is not what that message says. We (Mozilla) would expect Comodo to
be issuing certificates under any root it owns, whether the name on the
root is its own or another's, in compliance with the Mozilla CA policy
and the audits it has passed. If you have evidence to the contrary
(leaving the current situation aside for a moment), present it.

> 2. Comodo is apparently not only allowing resellers like CertStar,
> but is allowing them to do their own validation of the legitimacy
> of the certificate requestor. Who takes financial responsibility
> for such errors? CertStar itself disclaims financial responsibility
> at "http://www.certstar.com/terms.html".

That is a reasonable question.

> 3. Microsoft requires an annual audit for root CAs:
> "http://technet.microsoft.com/en-us/library/cc751157.aspx".
> Mozilla seems willing to accept a one-time audit. That seems
> to be why the disappearance of AddTrust wasn't noticed.

The two things are not connected. There are root certificates in the
store which bear the names of companies which have not existed for quite
some time. We know about this. Knowing about it is not a function of
audit frequency.

> 1. Comodo must undergo an audit to WebTrust standards, and the audit
> report must be published.

As I understand it, Comodo has a current WebTrust audit (confusion in
this thread notwithstanding):
https://cert.webtrust.org/SealFile?seal=804&file=pdf

Gerv

Ian G

unread,
Dec 27, 2008, 1:54:54 PM12/27/08
to mozilla's crypto code discussion list
On 27/12/08 02:21, Paul C. Bryan wrote:
> On Dec 26, 4:40 pm, Ian G<i...@iang.org> wrote:
>
> With respect:
>
>> This is a forum for the discussion of technical, crypto, root and general PKI
>> issues, by either dictat or convention. It is not a forum for the airing of general
>> business complaints.
>
> Are you characterizing this issue as merely a general business
> complaint?


I see two particular complaints. Firstly is the alleged
email-soliciting. This I would characterise as business, and I would
suggest is completely outside scope.

The second is the alleged failure of verification. This is more
particularly of interest to this forum, in principle, as it poses
questions of policy and practice, which the forum has by custom debated.

But, again, the dispute itself is probably outside scope for the moment,
although it may be inside scope of the bugzilla filing (an open question).

These are just my views and my readings of the policy and the
conventions. Others no doubt will chime in.

(BTW, is it verification or validation? As far as I know, there are
specific meanings for these terms, and one is likely wrongly applied.)


> I happen to agree that this should not be the forum for such
> discussion, but as you point out, there is no other apparent forum for
> dispute resolution. Where should such discussions be taken?


For the second dispute, there is a bugzilla filing. We might take that
as a place for the dispute before Mozilla, but this is by no means an
exclusive venue.

The formal forum for complaints is court, and we probably have to
respect that for both of the complaints, at some level. Because of the
nature of court, and because we've probably crossed the rubicon of
actionable acts and statements, everything said here could be presented.
So, in effect, any parties that are named may already be "in court"
and will see their statements presented. Hence my general warning.

That said, I'd say the really pressing question is, where *better* to
take these complaints, and deal with them in a more appropriate forum
(court being expensive, inconveniant, and too general). For that, I
don't know. Bugzilla doesn't begin to answer that question, and while
court will likely conquer the question, it will also likely kill the
solution and some of the participants.

I've had to work with this question in the past. However, the solutions
I have seen only work in certain contexts, and I can't see a way to
scale it to this situation. E.g., I can craft paper solutions, but they
always end up with "and then party X must agree" which nobody wants.


>> Having appreciated this point, a more interesting one is whether we as a
>> community think about opening up the processes for more open governance,
>> more open scrutiny, more stakeholder checking [1].
>
> My first reaction would be: most definitely, +1.
>
> That said, you have classified this discussion as a public lynching of
> a CA, called into question the professionalism of those engaging in
> such discussion, and identified potential legal liabilities in doing
> so.
>
> My question to you would be: assuming the issues you raised are
> legitimate (I happen to disagree) how could such an open governance
> model cope with the restrictions that would most certainly be
> necessary to address the issues you raised?


Good question! What we are seeing now: one side to the complaint feels
bound by certain stated and unstated restrictions, whereas the other
side(s) to the complaint feel(s) that they are free of all these stated
and unstated restrictions.

An open governance model would attempt to address these imbalances.

Firstly, lay out the restrictions of criticism more clearly. For
example, we would perhaps encourage open scrutiny of security models on
an ongoing basis, but we would require that all criticisms be grounded
in the prevailing practices and policies.

That is, if we say "the RA must not be outsourced!" we would have to
ground that as either a comment on the CPS of the CA concerned,
referring to a statement that indeed says "our RA is not outsourced" or
similar ... or as a proposal for a criteria that says "RA is not in
general outsourced" which then becomes a criticism of the mozo policy
and audit criteria, not of any CA.

(Insisting on clear grounding in practices & policies would wipe out
about 80% of the noise we have seen in this thread.)

Secondly, statements should be primarily non-personal. That is, if a
representative of a CA were to call for a report to be delivered on
X,Y,Z, then it could be taken to mean that the same representative of
the CA were proposing that this report be standard for all CAs, and
indeed, we might reasonably ask that representative to provide an example.

Thirdly, if there are to be any "penalties" then they need to equally
bind all. If I decide to claim that a CA is derelict in some duty, and
this should cause loss of something, then if found not to be true, then
the statement has to rebound on me for presenting a false claim.

Fourthly, if there were to be any "serious penalties" they should be
decided in a serious fashion. If not, in a serious dispute, all parties
will be likely exposed to court for frivolous acts. That "serious
method" needs to be written down, be impressive, and be followed.

Commonly this sort of thing is done with a "code of ethics". Sometimes
this is done with an association or industry grouping. However it is
done, there needs some way to deal with the downside as well as the upside.


>> Would we be in a position to explore a general opening of all auditing
>> investigations and controls [2] ?
>
> +1

iang

Michael Ströder

unread,
Dec 27, 2008, 1:48:38 PM12/27/08
to
ro...@comodo.com wrote:
> On Dec 24, 2:13 am, "Paul C. Bryan" <em...@pbryan.net> wrote:
>> 2. Are resellers subject to the same audits that Comodo presumably had
>> to undergo to get its root certs added to Mozilla? Who performs, and
>> who verifies such audits? How often are they performed?
> No, the RAs are not subject to the same audits as Comodo.

And that's a fundamental flaw. If you delegate RA functionality (here
domain validation) to a reseller leading to the reseller being capable
of triggering cert issuance without further validation of the CA the RA
should also be audited just like the CA.

>> 3. Are you willing to openly, continually disclose your list of
>> resellers, the frequency of audits, audit methodology, and actual
>> audit reports so that third parties can evaluate whether Comodo is
>> trustworthy as a CA?


> That is a question combined with an assertion.

> To the question: on a unilateral basis, no, Comodo wouldn't reveal
> that level of detail of our internal operation. If all CAs were
> required to provide the information, either to retain Webtrust
> certification or to gain or retain access to the root program of a
> major browser or other platform, then we would reconsider.

> To the assertion that this is a pre-requisite for a CA to be
> trustworthy: I am not aware that it is Mozilla's policy to require
> this information to be disclosed.

Robin, I agree that all CAs should fullfil the same requirements. And I
suspect your case is not the only problematic case.

So basically we're back at the point which was already raised many times
here. In former discussion people were concerned about the power of RAs
and sub-CAs of trusted root CAs and that this relationship is not
published at all. And as this case shows the concerns are valid.

Ciao, Michael.

Michael Ströder

unread,
Dec 27, 2008, 2:00:58 PM12/27/08
to
Ian G wrote:
> On 26/12/08 00:36, Michael Ströder wrote:
>> Paul Hoffman wrote:
>>> At 7:16 PM +0100 12/25/08, Michael Ströder wrote:
>>>> I'd tend to punish a rogue CA by removing their root CA cert from NSS.
>
> I do not see a rogue CA. The evidence of the posts here suggests a flaw
> leading to false certs was found and such certs were issued; and that
> the CA responded when made aware.
>
> What is rogue about that? Are you saying they didn't respond?

Bear in mind I'm not a native English speaker. After looking up "rogue"
in a dictionary it seems a little bit strong. So thanks for asking back.

Still I think we have a fundamental problem here which was discussed in
theory before many times here. And the follow-ups by Robin, Comodo and
Patricia, Certstar IMO shows that problem cannot be solved in practice
by just fixing a single mistake.

>>>> Maybe this serves as a good example to other CAs that the Mozilla CA
>>>> policy is really enforced. Otherwise nobody will care.
>>> This is Firefox we're talking about, not IE. Do you really think that
>>> this is going to help end users, or just hurt people who bought
>>> certificates from the lax (not rogue) CA?
>>
>> PKI is about security.
>
> Security is risks and costs. In this case, there is low risk and zero
> costs. Perhaps because the problem was caught early on, but security is
> about real hard facts not conjecture.

Ian, we had this point many times. Frankly you cannot really estimate
risks and costs in such cases since you don't know what happens out there.

Bear in mind that even though Mozilla products are provided at no cost
to the end user Mozilla could be made accountable and taken to court in
some countries for things going wrong. IIRC here in Germany you cannot
effectively deny warranty for open source products provided at no cost.
To some extent you have to apply generally accepted rules of technology
in every case.

> (If you want real hard costs and losses and grief, check out phishing.
> Where's the lynch mob when we are dealing with phishing, I wonder?)

If you really want to discredit me or my comments as "lynch mob" we can
simply stop discussing.

Ciao, Michael.

Michael Ströder

unread,
Dec 27, 2008, 2:22:31 PM12/27/08
to
Gervase Markham wrote:
> We (Mozilla) would expect Comodo to be issuing certificates under any
> root it owns, whether the name on the root is its own or another's,
> in compliance with the Mozilla CA policy and the audits it has
> passed.
> [..]

> There are root certificates in the store which bear the names of
> companies which have not existed for quite some time. We know about
> this. Knowing about it is not a function of audit frequency.

Disclaimer: I'm no lawyer. But different national laws might apply here.

Here in Germany we have some obligations for commercial web sites to
really show correct names (of natural or legal persons) and full postal
address so that anybody who wants to take the web site owner to court
can do so. It's called "Impressumspflicht" and it already caused lots of
litigation cases. In this spirit I'm not sure whether there aren't any
legal problems with root CA certs containing issuer names which are not
a valid name of a natural or legal person anymore. Even though such a
name mismatch is not primarily caused by Mozilla the project could be
taken to court because of publishing this false information as "trusted".

Ask your lawyers...

Ciao, Michael.

Gervase Markham

unread,
Dec 27, 2008, 2:23:03 PM12/27/08
to
Eddy Nigg wrote:
> On 12/27/2008 02:34 PM, Gervase Markham:
>> One of the points of EV was to allow us to act against a CA without
>> massive collateral damage. We can remove EV status from a root without
>> disabling the root entirely.
>
> Which unfortunately isn't really effective for the issue we are facing
> today.

No, indeed. My point was just that sayrer said "We didn't learn that
lesson for EV", and I am saying that we did.

> Removing EV status would be applicable in case the EV guidelines
> wouldn't be fulfilled by a CA. It's absolutely useless otherwise. Or
> would you suggest that because a CA doesn't perform its duties for
> regular certs to disable EV, even though their EV business practices are
> in complete compliance with the EV guidelines?

No, I'm not suggesting that.

Gerv

Frank Hecker

unread,
Dec 27, 2008, 2:34:01 PM12/27/08
to
John Nagle wrote:
> As a user of SSL certificates in our SiteTruth system, which
> attempts to identify and rate the business behind a web site, we're
> concerned about CA reliability and trust. We've been using Mozilla's
> approved root cert list for our system, and are considering whether
> we should continue to do so.

As a general point, I have never advocated having downstream licensees
of Mozilla code accept the default NSS root list as is, without doing
some due diligence on their own. There are lots of roots in that list
that are there for legacy reasons, and others that are not necessarily
of general interest (e.g., CAs operating within a single country or
region). I encourage you and other licensees to trim the root list to
meet your own needs and your own assessment of CAs.

> 1. Comodo must undergo an audit to WebTrust standards, and the audit

> report must be published. An in-house self-investigation is not
> acceptable. The audit must be conducted by a recognized outside
> auditing firm.

Comodo already has undergone WebTrust audits, and presumably will do so
again; see for example

https://cert.webtrust.org/SealFile?seal=798&file=pdf
https://cert.webtrust.org/SealFile?seal=804&file=pdf

which I believe are the latest ones. Robin Alden can provide information
on other past, present, and future WebTrust audits of Comodo.

> 2. CertStar must separately undergo an audit to WebTrust standards,


> and the audit report must be published.

Certstar isn't a CA, and thus the WebTrust for CAs criteria are not
necessarily a good fit for it. (Plus the expense of a full WebTrust for
CAs audit is likely an order of magnitude higher than Certstar's
probable revenues.) However it's certainly true that future Comodo
WebTrust audits could and IMO should look at the question of how Comodo
deals with resellers and affiliates, as part of the task of determining
whether Comodo is operating in accordance with its CPS.

Frank

--
Frank Hecker
hec...@mozillafoundation.org

Ian G

unread,
Dec 27, 2008, 2:46:17 PM12/27/08
to mozilla's crypto code discussion list


Where is this documented? I do not recall a mention of this in the
guidelines. It would seem to be a fairly important point!


iang

Ian G

unread,
Dec 27, 2008, 2:51:04 PM12/27/08
to mozilla's crypto code discussion list
On 27/12/08 13:43, Eddy Nigg wrote:
> On 12/27/2008 02:16 PM, Ian G:
>> Indeed, this is the "Verisign buyout model"; outsource something new,
>> get huge, get bought out by Verisign.
>
> What has that to do exactly with what Paul agreed to?
>
>> It doesn't matter in business principle whether it outsources a function
>> to a reseller, to its employees or to the government.
>
> Of course it does. Besides that an employee isn't outsourcing, he is
> part of the company. Or one might ask, why are certain functions never
> outsourced to a third party?


E.g., employees are sometimes subject to various criteria such as
background checking.

> Or perhaps lets start to outsource the CA
> root key responsibilities as well then...


This is already done. It is common practice to outsource the security
model of the root key to something called a HSM which is supplied by a
manufacturer, which again likely outsources its security criteria to
another party, for example CC.


>> Is there a criteria anywhere that says or implies "The CA has not
>> outsourced critical function X to an external agent?" Can anyone recall
>> such a statment?
>
> Yes, the some extend Mozilla does that already today with the
> "Problematic Practices". For example auditing of intermediate CAs
> shouldn't be outsourced from the auditor to the CA (it's just the other
> way around).


They are not criteria nor policy. If in the future they are to become
criteria or policy, let's propose them?


> And if there is no such criteria we might still create and adopt it.
> This is no precedence, there are other criterion already.


Yes, that was the question, to restate it: what criteria or policy exist?

>> that a popular incentive is to generate opportunities for business
>> revenues.
>
> So? Mozilla really shouldn't care about the business revenues of some
> CAs. How is that relevant?


Well, a normal lesson of business is that we can't get business people
to agree to something if their revenues go down... PKI is business only
(a frequent complaint, who speaks for the user?), and Mozilla has to
live in this business world.

Either way, when you get serious and propose a chance to a criteria or
policy, we have to expect that all will consider the revenues question.

Hence, I predict there are very few restrictions on outsourcing.


>... Specially in light that it's a


> core requirement of the Mozilla CA policy.


Well, with respect to desires and so forth, the words that matter are
the ones that are in the policy. It says:

"13. In addition to the requirements outlined above,
*we also recommend that* ..."

If there is a move to make that recommendation into a requirement, let's
hear it.

iang

Michael Ströder

unread,
Dec 27, 2008, 3:18:52 PM12/27/08
to
Ian G wrote:
> On 27/12/08 13:43, Eddy Nigg wrote:
>> So? Mozilla really shouldn't care about the business revenues of some
>> CAs. How is that relevant?
>
> Well, a normal lesson of business is that we can't get business people
> to agree to something if their revenues go down... PKI is business only
> (a frequent complaint, who speaks for the user?), and Mozilla has to
> live in this business world.

Please bear in mind the CAs want to be added to the trusted root CA cert
store to make business. AFAIK they don't pay for that. So Mozilla's
customers are the end users of Firefox etc. not the CAs.

> Either way, when you get serious and propose a chance to a criteria or
> policy, we have to expect that all will consider the revenues question.

It's definitely not Mozilla's task to think about the CAs' business and
whether they have revenues when being added to trusted root CA cert
store or not!

Ciao, Michael.

Michael Ströder

unread,
Dec 27, 2008, 3:10:49 PM12/27/08
to
Frank Hecker wrote:

> John Nagle wrote:
>> 2. CertStar must separately undergo an audit to WebTrust standards,
>> and the audit report must be published.
>
> Certstar isn't a CA, and thus the WebTrust for CAs criteria are not
> necessarily a good fit for it.

If a CA delegates some tasks to a RA the RA, probably a department and
not the whole company, should be certainly part of the CA audit as well.

> (Plus the expense of a full WebTrust for
> CAs audit is likely an order of magnitude higher than Certstar's
> probable revenues.)

It's Comodo's business decision whether they delegate some tasks to an
external RA or not and whether the revenues are worth it. That's IMO out
of scope for Mozilla and its policy regarding trusted root CA certs.

Ciao, Michael.

Florian Weimer

unread,
Dec 27, 2008, 3:38:03 PM12/27/08
to dev-tec...@lists.mozilla.org
* Hendrik Weimer:

> Frank Hecker <hec...@mozillafoundation.org> writes:
>
>> My intent is to balance the disruption that would be caused by pulling
>> a root vs. the actual security threat to users. Right now we have no
>> real idea as to the extent of the problem (e.g., how many certs might
>> have been issued without proper validation, how many of those were
>> issued to malicious actors, etc.).
>
> Isn't that, by itself, a very good reason to take immediate action?
> Security should be default-fail rather than default-pass.

This is not about security, this is about the presence or absence of
an obscure browser warning.

David E. Ross

unread,
Dec 27, 2008, 3:47:12 PM12/27/08
to
On 12/27/2008 5:07 AM, Gervase Markham wrote [in part]:
> Hi John,
>
> You raise some important questions, but it's worth having clarity on a
> few matters of fact.
>
> John Nagle wrote [also in part]:

>> 1. AddTrust, a company which apparently no longer exists, has an
>> approved
>> root CA certificate. This in itself is troublesome.
>
> This is extremely common. Certificates change hands. Failing to honour
> root certificates which are no longer owned by the companies which
> created them would break a significant proportion of the web. Microsoft
> does not have a policy preventing this.

I would sometimes encounter a secure site with a certificate from a root
not in the Mozilla database. The root would be from a CA that no longer
existed. Using the WebTrust list of certified CAs (a list that no
longer appears on the Web), I would be able to trace the changes in
ownership of such CAs and determine for myself whether the root was
indeed certified by WebTrust. It the root were certified by WebTrust,
WebTrust's list would even have a link to the current CA's Web site,
from where I could download and install the root.

This process is no longer available to users, now that WebTrust no
longer maintains a public list of certified CAs.

--
David E. Ross
<http://www.rossde.com/>

Go to Mozdev at <http://www.mozdev.org/> for quick access to
extensions for Firefox, Thunderbird, SeaMonkey, and other
Mozilla-related applications. You can access Mozdev much
more quickly than you can Mozilla Add-Ons.

David E. Ross

unread,
Dec 27, 2008, 4:11:51 PM12/27/08
to
On 12/27/2008 5:48 AM, Michael Ströder wrote [in part]:
> ro...@comodo.com wrote [in part]:

>> On Dec 24, 2:13 am, "Paul C. Bryan" <em...@pbryan.net> wrote:
>>> 2. Are resellers subject to the same audits that Comodo presumably had
>>> to undergo to get its root certs added to Mozilla? Who performs, and
>>> who verifies such audits? How often are they performed?
>> No, the RAs are not subject to the same audits as Comodo.
>
> And that's a fundamental flaw. If you delegate RA functionality (here
> domain validation) to a reseller leading to the reseller being capable
> of triggering cert issuance without further validation of the CA the RA
> should also be audited just like the CA.
>

Instead of auditing RAs and resellers, audit the root CA's process for
ensuring that RAs and resellers comply with the CA's policies (e.g., the
CP). This is what I proposed in a different (but related) thread in
this newsgroup.

The CA approves its RAs and resellers. Thus, the CA should be held
accountable for the actions of its RAs and resellers. If the CA's CP
addresses how accountability is handled (or denies the existence of RAs
or resellers), the CA's outside audit is supposed to review the
implementation of this (along with the implementation of the all rest of
the CP). If this accountability is not addressed in the CP or the way
it is addressed is weak, the CA's root does not belong in the Mozilla
database.

I ask Hecker, Wilson, and any others doing the initial reviews of root
certificates proposed for inclusion in the Mozilla database to give some
attention to this.

Michael Ströder

unread,
Dec 27, 2008, 5:37:43 PM12/27/08
to
Ian G wrote:
> That "earlier story" has no real place here, IMHO. This is a forum for

> the discussion of technical, crypto, root and general PKI issues, by
> either dictat or convention. It is not a forum for the airing of
> general business complaints.

I agree that the effects of this whole story on Startcom's business is
out of scope for this forum and Eddy has to clarify that with his
lawyers. I'm certain Eddy knows that. (And I personally am not
affiliated with Eddy or Startcom.)

But the fact is that Certstar used misleading DNS names for their web
site to trick Starcom's customers to "re-new" certs at their web site.
These server naming tricks are pretty close to what phishers are doing.
Also look at From: google@ in one of Patricia's postings. So I take this
as a strong indication that Certstar has a rather rogue attitude (and in
case of Certstar I mean like this). And discussing the conclusions for
trustworthiness of Comodo is perfectly within the scope of this forum.

> E.g., where Comodo or any CA completes an internal audit and creates a
> report to document that audit action, could we expect the CA or the
> internal auditor to publish this as a routine action?

Personally I have some doubts about auditing reports anyway.

But I believe that bad press and removing the trust flags from a root CA
cert as a result of a CA misbehaving is an appropriate negative
enforcement leading to better results in the long run. Again: If Mozilla
fails to enforce its own policy the Mozilla foundation should better
drop this whole root CA cert store completely.

Ciao, Michael.

Michael Ströder

unread,
Dec 27, 2008, 5:51:33 PM12/27/08
to
Frank Hecker wrote:
> John Nagle wrote:
>> As a user of SSL certificates in our SiteTruth system, which
>> attempts to identify and rate the business behind a web site, we're
>> concerned about CA reliability and trust. We've been using Mozilla's
>> approved root cert list for our system, and are considering whether
>> we should continue to do so.
>
> As a general point, I have never advocated having downstream licensees
> of Mozilla code accept the default NSS root list as is, without doing
> some due diligence on their own. There are lots of roots in that list
> that are there for legacy reasons, and others that are not necessarily
> of general interest (e.g., CAs operating within a single country or
> region). I encourage you and other licensees to trim the root list to
> meet your own needs and your own assessment of CAs.

If e.g. a Linux distributor wants to ship Firefox and trims the list of
pre-installed trusted root CA certs is it still allowed to distribute
the resulting code as Firefox?

Ciao, Michael.

Frank Hecker

unread,
Dec 27, 2008, 6:58:07 PM12/27/08
to
Michael Ströder wrote:
> If e.g. a Linux distributor wants to ship Firefox and trims the list of
> pre-installed trusted root CA certs is it still allowed to distribute
> the resulting code as Firefox?

That's a decision for the people at the Mozilla Corporation who work
with Linux distributors and others who want to distribute
Firefox-branded browsers.

Note that in the past I've proposed having localized versions of Firefox
have slightly different root lists, e.g. to account for national or
regional CAs that have limited or no market share elsewhere in the
world. As it turned out, people didn't want to do that for various
reasons, but I myself am not wedded to the idea that the default root
list should never be changed by downstream distributors.

Eddy Nigg

unread,
Dec 27, 2008, 6:59:13 PM12/27/08
to
On 12/27/2008 05:10 PM, Michael Ströder:

> Frank Hecker wrote:
>> (Plus the expense of a full WebTrust for
>> CAs audit is likely an order of magnitude higher than Certstar's
>> probable revenues.)
>
> It's Comodo's business decision whether they delegate some tasks to an
> external RA or not and whether the revenues are worth it. That's IMO out
> of scope for Mozilla and its policy regarding trusted root CA certs.
>

Certainly! I don't think Frank implied that (if he would, I'd have some
suggestions to make ;-) ), but simply stated the fact that RAs are not
CAs and hence can't perform themselves a WebTrust for CAs audit. They
could be audited nevertheless by an audit firm to a different set of
criterion of course.

Eddy Nigg

unread,
Dec 27, 2008, 7:01:28 PM12/27/08
to
On 12/27/2008 05:38 PM, Florian Weimer:

>> Isn't that, by itself, a very good reason to take immediate action?
>> Security should be default-fail rather than default-pass.
>
> This is not about security, this is about the presence or absence of
> an obscure browser warning.

Huuu? Have you understood the issue at all? I'm not sure...however it's
not about browser warnings. This is about security proper. Or how else
would you explain an MITM attack?

Ian G

unread,
Dec 27, 2008, 7:26:28 PM12/27/08
to mozilla's crypto code discussion list
On 27/12/08 20:01, Eddy Nigg wrote:
> On 12/27/2008 05:38 PM, Florian Weimer:
>>> Isn't that, by itself, a very good reason to take immediate action?
>>> Security should be default-fail rather than default-pass.
>>
>> This is not about security, this is about the presence or absence of
>> an obscure browser warning.
>
> Huuu? Have you understood the issue at all? I'm not sure...however it's
> not about browser warnings. This is about security proper. Or how else
> would you explain an MITM attack?


Security proper is about risks and threats and costs for end-users. Ask
them whether they are worried about an MITM attack :)

Anyway, old debate, not going to be solved today.

iang

Eddy Nigg

unread,
Dec 27, 2008, 7:44:44 PM12/27/08
to
On 12/27/2008 03:07 PM, Gervase Markham:

> This is extremely common. Certificates change hands. Failing to honour
> root certificates which are no longer owned by the companies which
> created them would break a significant proportion of the web. Microsoft
> does not have a policy preventing this.
>

In itself I've raised concern about it previously. If Microsoft is
preventing it or not it isn't really relevant. If we look at the issue
more closely, than we will realize (maybe) that companies can change
hands, but not root certificates. If common policies are applied to
roots as they are applied to end-user (and even intermediate CA)
certificates, than roots which change any of the listed parameters must
be revoked and a new certificate created with the corrected and updated
information. This is a common requirement of digital certificates at large.

In this case, I knew to whom the affected root belonged, even though it
listed an unrelated company from Sweden or Utah. Others would simply not
know. If a user must start researches in a field not familiar to him
and/or has to contact the browser vendor in order to know who the issuer
is, I think we have a problem.

Florian Weimer

unread,
Dec 27, 2008, 8:36:13 PM12/27/08
to mozilla's crypto code discussion list
* Eddy Nigg:

> On 12/27/2008 05:38 PM, Florian Weimer:
>>> Isn't that, by itself, a very good reason to take immediate action?
>>> Security should be default-fail rather than default-pass.
>>
>> This is not about security, this is about the presence or absence of
>> an obscure browser warning.
>
> Huuu? Have you understood the issue at all?

I think so.

> I'm not sure...however it's not about browser warnings. This is
> about security proper.

As a downstream distributor of Mozilla code, I'd hate to roll out
updates (especially security updates) just because CAs start to play
games with each other. This is not about "security proper". You're
trying to pull us into a PR attack on one of your competitors, thereby
willingly reducing confidence in ecommerce. (I'm exaggerating a bit,
of course.)

> Or how else would you explain an MITM attack?

If users edit /etc/hosts to complete the attack, it's their fault.

Even if you've got the certificate, you need to attack IP routing or
DNS. If you can do that, chances are that you can mount this attack
against one of the domain-validating RAs, and still receive a
certificate. So the browser PKI is currently irrelevant for practical
purposes (beyond CA revenues and giving users a warm, fuzzy feeling),
even if everybody follows established RA procedures.

Michael Ströder

unread,
Dec 27, 2008, 9:07:52 PM12/27/08
to
Eddy Nigg wrote:
> On 12/27/2008 05:10 PM, Michael Ströder:
>> Frank Hecker wrote:
>>> (Plus the expense of a full WebTrust for
>>> CAs audit is likely an order of magnitude higher than Certstar's
>>> probable revenues.)
>>
>> It's Comodo's business decision whether they delegate some tasks to an
>> external RA or not and whether the revenues are worth it. That's IMO out
>> of scope for Mozilla and its policy regarding trusted root CA certs.
>>
>
> Certainly! I don't think Frank implied that (if he would, I'd have some
> suggestions to make ;-) ), but simply stated the fact that RAs are not
> CAs and hence can't perform themselves a WebTrust for CAs audit. They
> could be audited nevertheless by an audit firm to a different set of
> criterion of course.

I meant the RA should also be audited during the CA audit.

Ciao, Michael.

Michael Ströder

unread,
Dec 27, 2008, 9:13:56 PM12/27/08
to
Florian Weimer wrote:
> Even if you've got the certificate, you need to attack IP routing or
> DNS. If you can do that, chances are that you can mount this attack
> against one of the domain-validating RAs, and still receive a
> certificate. So the browser PKI is currently irrelevant for practical
> purposes (beyond CA revenues and giving users a warm, fuzzy feeling),
> even if everybody follows established RA procedures.

Oh Florian, come on! You know the MITM techniques within a LAN very
well. So I take your comment simply as a provocation saying that
maintaining a cert store with pre-trusted root CA certs are not worth
the effort at all. But that's also not entirely true.

Ciao, Michael.

Eddy Nigg

unread,
Dec 27, 2008, 9:20:55 PM12/27/08
to
On 12/27/2008 11:07 PM, Michael Ströder:

>
> I meant the RA should also be audited during the CA audit.
>

This in turn would be similar to this
https://wiki.mozilla.org/CA:Problematic_Practices#Allowing_external_entities_to_operate_unconstrained_subordinate_CAs

At this stage I'm not proposing to make the same requirements as for
externally operated intermediate CAs, however I'd maybe suggest to have
critical aspects performed at the CA itself than have it outsourced.
This in order to guaranty adherence to the Mozilla CA Policy section 7.

Eddy Nigg

unread,
Dec 27, 2008, 9:46:49 PM12/27/08
to
On 12/27/2008 10:36 PM, Florian Weimer:

> As a downstream distributor of Mozilla code,

StartCom is also a downstream distributor of Mozilla code...

> I'd hate to roll out updates (especially security updates)

...which happens every two month anyway...

> just because CAs start to play games with each other.

I really hope that nobody sincerely believes that this is a game.
However any other party - and not only competing CAs - are invited to
verify the the implementations of the StartCom CA at any time, heck I'd
even thank you for finding a bug: http://www.startssl.com/

(For every wrongfully issued certificate I'll return to you ten times
the amount you paid for it ;-) )

> This is not about "security proper". You're
> trying to pull us into a PR attack on one of your competitors, thereby
> willingly reducing confidence in ecommerce. (I'm exaggerating a bit,
> of course.)

Exactly the opposite is true. If at all, I'm trying to encourage
responsible competition on *equal* footing without compromising the
security of the relying parties. It needs just *one* CA to devalue the
collective work of browser vendors, certification authorities and
cryptography specialist. Only one! Unfortunately some CAs take their
responsibilities less serious than others - which in turn gives them a
competitive advantage.
Besides that, I'm known to work towards improving the practices of
public certification authorities in order to provide better security on
the Internet.

> If users edit /etc/hosts to complete the attack, it's their fault.

Nobody will do that and this is not how those attacks work. That's only
to easily demonstrate it.

> Even if you've got the certificate, you need to attack IP routing or
> DNS.

This is one way, there are others [1].

> If you can do that, chances are that you can mount this attack
> against one of the domain-validating RAs, and still receive a
> certificate.

CAs (should) have controls in place to prevent that from happening. But
since you mentioned RAs, than yes, it's fairly bad that an RA hosted at
a hosting provider should perform those validations. This is exactly why
we think that this functionally should not be outsourced but performed
at the CA.


> So the browser PKI is currently irrelevant for practical
> purposes (beyond CA revenues and giving users a warm, fuzzy feeling),
> even if everybody follows established RA procedures.

This however is unrelated FUD!

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=460374

Kyle Hamilton

unread,
Dec 27, 2008, 11:56:13 PM12/27/08
to mozilla's crypto code discussion list
I am a user. I am worried about MITM attacks.

Unlike most users, I'm technically and legally savvy enough to know:
1) Why to perform my due diligence
2) How to perform my due diligence
3) How to add the root into my store

However, I have additional problems that I can't deal with through
the standard Mozilla user interface (or any browser that I have access
to's interface, realistically).

For example, I cannot easily see who issued a given certificate, or
what root it chains up to. I cannot apply an attribute to a root
certificate saying "not a financial-services certification authority".
I cannot see details about the chain without having to go through
multiple difficult-to-get-to windows.

If it wasn't already obvious from the past five years that I've been
on this list, I resent the way that Mozilla's developers have chosen
to make it continually more difficult for me to do what I need to do
to ensure my own security, by concealing more and more information
(there was the "blue site name" bar, which was disabled by default in
FF3, which provides one-click access to the information I need --
whereas the lock icon at the bottom requires a double-click).

Further, I resent the fact that there's a "this web site does not
supply identity information" line. THAT IS WHERE I NEED THE SUBJECT
TO BE PRINTED. I honestly don't care one whit that it's not an EV
certificate. I need the Subject, because I need to see at one glance
if it's a "Domain Control Verified" certificate, not have to
double-click the lock and then click "View Certificate". If you want
to point out that this is not extended-validation, that's fine -- but
for the sake of the users, don't try to "protect" them from
"unverified information".

It is my unshakable belief that if a user EVER has to examine the
certificate itself, or go into the interface to do so, the goal of the
user interface (which is to provide information) has failed. This is
NOT, however, a statement that the ability to view the certificate
should be removed! (Especially given Mozilla's track record at
creating useful user interfaces for certificate data presentation --
every time they've done something right, they've gone back two
revisions later, declared it "useless", removed it, and put in
something even more wrong.)

I believe that CA branding on the UI is necessary, so that the user
can do the due diligence which Mozilla is arguably NOT doing on the
user's behalf, no matter that Mozilla appears to claim that they are
by requiring audits to WebTrust criteria as a prerequisite to joining
the "big CAs club" of Mozilla's trust list.

-Kyle H

On Sat, Dec 27, 2008 at 11:26 AM, Ian G <ia...@iang.org> wrote:
> On 27/12/08 20:01, Eddy Nigg wrote:
>>

>> On 12/27/2008 05:38 PM, Florian Weimer:
>>>>
>>>> Isn't that, by itself, a very good reason to take immediate action?
>>>> Security should be default-fail rather than default-pass.
>>>
>>> This is not about security, this is about the presence or absence of
>>> an obscure browser warning.
>>
>> Huuu? Have you understood the issue at all? I'm not sure...however it's
>> not about browser warnings. This is about security proper. Or how else
>> would you explain an MITM attack?
>
>

> Security proper is about risks and threats and costs for end-users. Ask
> them whether they are worried about an MITM attack :)
>
> Anyway, old debate, not going to be solved today.
>
> iang

> _______________________________________________
> dev-tech-crypto mailing list
> dev-tec...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-crypto
>

Nelson B Bolyard

unread,
Dec 28, 2008, 8:20:40 PM12/28/08
to mozilla's crypto code discussion list
Kyle Hamilton wrote, On 2008-12-27 15:56:
> I am a user. I am worried about MITM attacks.
>
> Unlike most users, I'm technically and legally savvy enough to know:
> 1) Why to perform my due diligence
> 2) How to perform my due diligence
> 3) How to add the root into my store
>
> However, I have additional problems that I can't deal with through
> the standard Mozilla user interface (or any browser that I have access
> to's interface, realistically).
>
> For example, I cannot easily see who issued a given certificate, or
> what root it chains up to. I cannot apply an attribute to a root
> certificate saying "not a financial-services certification authority".
> I cannot see details about the chain without having to go through
> multiple difficult-to-get-to windows.

[rest snipped]

Kyle, I think you're making some good points about the PKI/security related
UI in Mozilla products (especially Firefox). I'm afraid this will get lost
in this thread which is otherwise about CA behaviors and responsibilities.
So, I suggest you start a new thread for this topic. Do that by posting
a message which is not a reply to any existing message, but is the first
with a new subject (not reusing a subject in any previously posted message).

When you do that, some may suggest that another of Mozilla's newsgroups and
mailing lists is a better venue for that discussion, and if so, so be it.
But I encourage you to raise the UI issues in a way that is clearly
separated from the current CA brouhaha.

Ben Bucksch

unread,
Dec 30, 2008, 6:47:25 PM12/30/08
to Gervase Markham
On 27.12.2008 13:34, Gervase Markham wrote:
> sayrer wrote:
>
>> The truth is that we are basically unable to act without a lot of
>> collateral damage. We should keep this in mind with future security
>> technology. Relying on companies willing to take money for doing
>> absolutely nothing (not even the bare minimum they agreed to) is not a
>> pleasant thing to do to our users. We didn't learn this lesson with
>> EV--maybe next time! :)
>>
>
> One of the points of EV was to allow us to act against a CA without
> massive collateral damage. We can remove EV status from a root without
> disabling the root entirely.
>

Well, really?

We try to train users to check that the bar is green (on sites where it
was green before), and not use the site when it's merely blue.
Otherwise, EV is useless, as the scammer could get a, say, CertStar
cert, to fake an EV site, right? Only when people start getting
concerned and stop visiting the site when it's truning green->blue is EV
of any use.

So, that means we have the same collateral damage as now.

See thread "Just change expiry time" for an alternative reaction.

Ben

Florian Weimer

unread,
Dec 30, 2008, 9:04:32 PM12/30/08
to mozilla's crypto code discussion list
* Michael Ströder:

> Florian Weimer wrote:
>> Even if you've got the certificate, you need to attack IP routing or
>> DNS. If you can do that, chances are that you can mount this attack
>> against one of the domain-validating RAs, and still receive a
>> certificate. So the browser PKI is currently irrelevant for practical
>> purposes (beyond CA revenues and giving users a warm, fuzzy feeling),
>> even if everybody follows established RA procedures.
>
> Oh Florian, come on! You know the MITM techniques within a LAN very
> well.

BCP 38 requires that active MITM attacks don't work on LANs. LANs
which violate that and are under attack are typically not very usable:
Search engines blocks you due to automated queries, DHCP and DNS
delivers data which is not 100% accurate (with unknown consequences),
you receive even more web ads than usual, rogue PPPoE servers sniff
your credentials, and so on.

In short, I don't think this is the use case to optimize for.

> So I take your comment simply as a provocation saying that
> maintaining a cert store with pre-trusted root CA certs are not
> worth the effort at all. But that's also not entirely true.

If you can't get rid of CAs which are snake oil because they add no
value beyond suppressing the browser warning, the certificate store
serves little purpose beyond CA revenue generation and improving user
experience (the latter isn't a bad thing per se, actually security and
perceived security are both important).

Nelson B Bolyard

unread,
Dec 30, 2008, 9:38:06 PM12/30/08
to mozilla's crypto code discussion list
Florian Weimer wrote, On 2008-12-30 13:04:
> * Michael Ströder:

>
>> Florian Weimer wrote:
>>> Even if you've got the certificate, you need to attack IP routing or
>>> DNS. If you can do that, chances are that you can mount this attack
>>> against one of the domain-validating RAs, and still receive a
>>> certificate. So the browser PKI is currently irrelevant for practical
>>> purposes (beyond CA revenues and giving users a warm, fuzzy feeling),
>>> even if everybody follows established RA procedures.
>> Oh Florian, come on! You know the MITM techniques within a LAN very
>> well.
>
> BCP 38 requires that active MITM attacks don't work on LANs.

Surely you don't really think that's much of a deterrent to attackers?!

> LANs which violate that and are under attack are typically not very usable:

If an attacker wants his attack to be effective, he will be sure that it
does not render the LAN unusable.

> Search engines blocks you due to automated queries, DHCP and DNS
> delivers data which is not 100% accurate (with unknown consequences),
> you receive even more web ads than usual, rogue PPPoE servers sniff
> your credentials, and so on.

Consider the increasingly common case of the "free" wireless access point
set up for the express purpose of MITMing all those who would use it.
Consider the phenomenon of "phorm". Most ordinary users never even
notice that they're under attack unless the attacker does a really
poor job of it (e.g. bug 460374).

> In short, I don't think this is the use case to optimize for.

This is the use case that sets SSL apart from other lesser crypto schemes
that do weak/no authentication.

Gervase Markham

unread,
Dec 30, 2008, 9:55:59 PM12/30/08
to
Ian G wrote:
> Where is this documented? I do not recall a mention of this in the
> guidelines. It would seem to be a fairly important point!

As I understand it, this is a feature of our implementation of EV, not
anything to do with the guidelines. Just as we are enabling roots for EV
one at a time, we can also disable them.

Gerv

Gervase Markham

unread,
Dec 30, 2008, 9:57:40 PM12/30/08
to
Ben Bucksch wrote:
> We try to train users to check that the bar is green (on sites where it
> was green before), and not use the site when it's merely blue.
> Otherwise, EV is useless, as the scammer could get a, say, CertStar
> cert, to fake an EV site, right? Only when people start getting
> concerned and stop visiting the site when it's truning green->blue is EV
> of any use.
>
> So, that means we have the same collateral damage as now.

Well... yes and no. If we remove a root, then the user gets scary error
messages and can't easily access the site. If we remove EV status, the
CA and their customers get upset because some customers are going to get
spooked (they don't know how many - that's one of the good things). So
removing EV is, in some senses, not as big a deal as yanking a root.

Gerv

Kyle Hamilton

unread,
Dec 30, 2008, 10:28:55 PM12/30/08
to mozilla's crypto code discussion list
On Tue, Dec 30, 2008 at 1:04 PM, Florian Weimer <f...@deneb.enyo.de> wrote:
> BCP 38 requires that active MITM attacks don't work on LANs. LANs

> which violate that and are under attack are typically not very usable:
> Search engines blocks you due to automated queries, DHCP and DNS
> delivers data which is not 100% accurate (with unknown consequences),
> you receive even more web ads than usual, rogue PPPoE servers sniff
> your credentials, and so on.

I'll point out that at least one of the cases which Mozilla is using
as its standard for the security UI involved a user who was subject to
an active MITM attack while connected to a public wireless hotspot.

BCPs do NOT mandate anything. They are "best current practices", and
it's a fact that they can be ignored by anyone for any time for any
reason (they are advisory, but local policy can and will often
override them).

> In short, I don't think this is the use case to optimize for.

I think this is important to realize: security is not an
all-or-nothing thing. Anyone who puts all of their eggs in one basket
(a single thick wall, or a moat -- or, as was found in the late 90s, a
firewall) is going to be unpleasantly surprised when the security of
their supposedly-impregnable defense is breached and they have no
mitigation plan.

We NEED to optimize for this case.

(note: "unknown_issuer" without talking at all about who the issuer
claims to be -- and being able to download a certificate and then
accept it without having to see who it's issued by -- is a "WTF WAS
THE SECURITY TEAM THINK--WAIT, WAS THE SECURITY TEAM THINKING??!!!!"
situation. It failed to mitigate against the attack that Nelson
cites, bug 460374.)

-Kyle H

Florian Weimer

unread,
Jan 3, 2009, 4:41:43 PM1/3/09
to mozilla's crypto code discussion list
* Eddy Nigg:

>> just because CAs start to play games with each other. This is not


>> about "security proper". You're trying to pull us into a PR attack
>> on one of your competitors, thereby willingly reducing confidence
>> in ecommerce. (I'm exaggerating a bit, of course.)
>
> Exactly the opposite is true. If at all, I'm trying to encourage
> responsible competition on *equal* footing without compromising the
> security of the relying parties. It needs just *one* CA to devalue the
> collective work of browser vendors, certification authorities and
> cryptography specialist. Only one! Unfortunately some CAs take their
> responsibilities less serious than others - which in turn gives them a
> competitive advantage.

I can understand that point of view. But what you seem to be asking
is that browser vendors take the role of judges, regulating CA
behavior. Shouldn't that be better left to the court system, keeping
Mozilla out of the loop? What advantage does Mozilla gain by acting
as a judge on day-to-day operations of CAs?

It might make sense to demand additional elements in the CPS for
future root additions, and re-audit existing roots.

> CAs (should) have controls in place to prevent that from
> happening.

Could you explain what you're doing in this area? (A "no" is
perfectly acceptable because nothing you can do is totally secure, so
keeping the mechanisms secret actually buys you something.)

Anyway, one thing that comes to my mind is domain control verification
over multiple communication channels, perhaps by injecting multiple
email messages at different points of the Internet, to make at least
sure that any hijacking is rather close to the subject. But I don't
think you have to answer to multiple mail challenges for DV
certificates.

> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=460374

There might be a legitimate business reason to do this form of
interception (doing this to get "free AV" is quite common, I suppose).
But I agree it's interesting.

Nelson B Bolyard

unread,
Jan 3, 2009, 6:59:30 PM1/3/09
to mozilla's crypto code discussion list
Gervase Markham wrote, On 2008-12-27 05:07:
> Hi John,
>
> You raise some important questions, but it's worth having clarity on a
> few matters of fact.
>
> John Nagle wrote:
>> 1. AddTrust, a company which apparently no longer exists, has an
>> approved
>> root CA certificate. This in itself is troublesome.
>
> This is extremely common. Certificates change hands.

One must admit that there is more than a little irony at play when the
system that claims to offer sign assurances, binding keys to identity,
and which (in the case of EV) claims to offer an identity that is strong
enough that a relying party could take the party thus identified to court,
fails to similarly identify the party making the signed assertion.

Eddy Nigg

unread,
Jan 3, 2009, 7:29:32 PM1/3/09
to
On 01/03/2009 06:41 PM, Florian Weimer:

>
> I can understand that point of view. But what you seem to be asking
> is that browser vendors take the role of judges, regulating CA
> behavior. Shouldn't that be better left to the court system, keeping
> Mozilla out of the loop? What advantage does Mozilla gain by acting
> as a judge on day-to-day operations of CAs?

The same criteria should be applied to all CAs. With less definition
there is also more of room to undercut in every respect. Definitions and
agreed upon standards are nothing for the courts really, they need to be
defined first.

>> CAs (should) have controls in place to prevent that from
>> happening.
>
> Could you explain what you're doing in this area? (A "no" is
> perfectly acceptable because nothing you can do is totally secure, so
> keeping the mechanisms secret actually buys you something.)

Yes, I think I don't want to elaborate on that really. But CAs usually
have more experience and know-how to set up preventive measures than an RA.

Ian G

unread,
Jan 4, 2009, 2:48:45 AM1/4/09
to mozilla's crypto code discussion list
On 3/1/09 17:41, Florian Weimer wrote:
> I can understand that point of view. But what you seem to be asking
> is that browser vendors take the role of judges, regulating CA
> behavior. Shouldn't that be better left to the court system, keeping
> Mozilla out of the loop? What advantage does Mozilla gain by acting
> as a judge on day-to-day operations of CAs?


I think this is an extremely important point.

Security and other objective systems work on certain very basic
assumptions. One of which is (a) we put in place policies and
practices. Cool, we all agree with that.

Another very basic assumption is (b) the policies and practices are not
perfect, and will fail, from time to time, in place to place.

(Now, I recognise that this is new to some, but it is a fundamental part
of the makeup.)

What happens when it fails? If "nothing happens" then a savvy operator
realises that, all those procedures and policies and whathaveyou are
worthless. Or at least, can be ignored. Mint certs, cash in.

Now, in theory society has an answer for this: punishment for those who
fail to follow our rules. If the punishment, or "control" is designed
well, we have a balance of pre- and post- event controls, and a feedback
loop that takes events and feeds info back into policies and practices.

What is at issue here is that we don't have good post-event controls.
When something fails, we do not know what to do. We have on the table,
right now, three examples of failure. They are all radically different,
at various levels, and an objective analysis of the results will throw
up lots and lots of contrasting suggestions, and lots and lots of unfair
comparisons.

On the punishment side, about all we have is "drop the root!" which I
earlier described as a blunt weapon. Are we being sensible when we now
have to "drop the root" for the three CAs who have reported problems?

So what to do? Should "Mozilla" become "the judge" in the post-event
phase? Do we leave this job to the courts? Should we group together on
this list and pass final judgement? Should we all vote? Demand
changes? Should we implement California rules -- 3 strikes and the root
is killed?

We need something. With nothing, we have no feedback. With no
feedback, any objective system drifts to subjectivity. It is I think
the case that for the entirety of the Internet PKI system, no
participant has ever been punished; how far into insecurity are we?

iang

Eddy Nigg

unread,
Jan 4, 2009, 8:20:53 AM1/4/09
to
On 01/04/2009 04:48 AM, Ian G:

> On the punishment side, about all we have is "drop the root!" which I
> earlier described as a blunt weapon. Are we being sensible when we now
> have to "drop the root" for the three CAs who have reported problems?

Actually we've discussed this issue just recently but before you started
your valuable contributions here as well ;-)

It was suggested to work with the CAs in question to improve and solve
those outstanding issues. I think this is what Frank has always done so
far and which could be considered policy. However nothing is holding the
hands of Mozilla back from removing a root if needed. That is specially
when a risk for the users exists. Negligence by the CA and failure to
conform to the Mozilla CA Policy are certainly on the table as well I guess.


> Demand changes?

Where needed, yes.

> Should we implement California rules -- 3 strikes and the root is killed?

It very much depends on the circumstances, but yes, I think repeated
failure of a CA should have consequences. Requiring attestations about
the requested changes could be more effective before actually removing a
root.

>
> We need something. With nothing, we have no feedback. With no feedback,
> any objective system drifts to subjectivity. It is I think the case that
> for the entirety of the Internet PKI system, no participant has ever
> been punished; how far into insecurity are we?

Somewhat perhaps. On the other hand let me show you this example: When
the Debian weak keys surfaced, I wasn't overly convinced on the need for
action initially. This forum and members of it convinced me otherwise
and I worked at my organization for a resolution. It resulted (as
reported here [1]) in the following;

"I've received reports that StartSSL was one of the first CAs to
filter for bad keys, and others, such as GoDaddy and Netlock have
followed suit.)"

The work here and the influence provoked actions which were picked up by
others including the most important CAs. As I also indicated earlier, my
participation here is a give and take and can provoke changes also at
the organization I run. This shows that the right influence can have
positive results without "punishments". On other matters this might not
always be enough however, not entirely sure.

[1] http://codefromthe70s.org/sslblacklist.aspx

Eddy Nigg

unread,
Jan 4, 2009, 9:08:24 AM1/4/09
to
On 01/04/2009 10:20 AM, Eddy Nigg:

> On 01/04/2009 04:48 AM, Ian G:
>> On the punishment side, about all we have is "drop the root!" which I
>> earlier described as a blunt weapon. Are we being sensible when we now
>> have to "drop the root" for the three CAs who have reported problems?

Oh btw. where do you see three roots to drop?

Florian Weimer

unread,
Jan 4, 2009, 10:24:01 AM1/4/09
to mozilla's crypto code discussion list
* Ian G.:

> So what to do? Should "Mozilla" become "the judge" in the post-event
> phase? Do we leave this job to the courts? Should we group together
> on this list and pass final judgement? Should we all vote? Demand

> changes? Should we implement California rules -- 3 strikes and the
> root is killed?

A three strikes approach encourages confidence-reducing games, so I
don't like it.

I think that without court involvement, it's very difficult to run
proper discovery. Suppose that you have got evidence which strongly
suggests that a CA keeps an equivalent of the private key of the root
in a non-secured data center. The evidence is short of a conclusive
proof, though. You ask the CA about it, and it says, "no, that's not
true". You ask, "can you show us how you have structured control of
the private key?", and the answer is, "no, that's business
confidential information". The same dialog might happen after you
have obtained actual proof (in the form of a certificate) that
something is amiss. This time, the CA says that it has "implemented
adequate controls to prevent a recurrence of the event", and details
remain confidential.

But if all you've got is CA output due to lack of transparency, you
are in three strikes territory.

The downside of court involvement is that if all CAs are rotten and
don't want to enforce, the whole system continues to drift.

> We need something. With nothing, we have no feedback. With no
> feedback, any objective system drifts to subjectivity. It is I think
> the case that for the entirety of the Internet PKI system, no
> participant has ever been punished; how far into insecurity are we?

EV is (also) an attempt to devalue existing infrastructure, so it's
some form of group punishment.

Daniel Veditz

unread,
Jan 4, 2009, 7:27:01 PM1/4/09
to
Eddy Nigg wrote:
> On 01/04/2009 10:20 AM, Eddy Nigg:
>> On 01/04/2009 04:48 AM, Ian G:
>>> On the punishment side, about all we have is "drop the root!" which I
>>> earlier described as a blunt weapon. Are we being sensible when we now
>>> have to "drop the root" for the three CAs who have reported problems?
>
> Oh btw. where do you see three roots to drop?

The three CAs who have recently issued certs to unauthorized parties are
RapidSSL (md5 hack), Certstar/Comodo, and StartCom.

Daniel Veditz

unread,
Jan 4, 2009, 7:34:02 PM1/4/09
to
Florian Weimer wrote:
> EV is (also) an attempt to devalue existing infrastructure, so it's
> some form of group punishment.

It also provides browsers with a slightly less blunt weapon. If a CA
clearly violates EV guidelines the browser could remove the EV-ness of
the root without removing the root itself. Users could still get to the
sites so we're not punishing users and not putting sites out of
business, but the site owners are no longer getting what they paid for
and that will put pressure on the CA.

Eddy Nigg

unread,
Jan 4, 2009, 7:45:24 PM1/4/09
to
On 01/04/2009 09:34 PM, Daniel Veditz:

It has been pointed out that this scenario is less likely than having a
CA conform fully to the EV guidelines, but their other CA business might
be considered unreliable. In such a case it would be advisable for
keeping the EV-ness as you say and remove the trust bits from the root
for their other certs. Now I don't remember if that's possible.

Eddy Nigg

unread,
Jan 4, 2009, 8:03:14 PM1/4/09
to
On 01/04/2009 09:27 PM, Daniel Veditz:

There could have been many more CAs which could have fallen under this
category - RapidSSL was the unlucky one to be picked perhaps.
Potentially you could add quite a few here...not sure.

> Certstar/Comodo

Certstar is not a CA, but Comodo. As per previous suggestion, the issue
of their resellers and RAs must be looked at and if needed a solution
found in my opinion. Of course Mozilla must decide first what it exactly
expects in this respect and what is unacceptable. On the other hand, the
current Mozilla CA Policy could be applied as well, which is rather
clear on this issue.

and StartCom.

Ahhh, yes :-)

Did you or anybody else see an issue with the policies and practices of
StartCom which beyond the resolution StartCom offered and handled the
incident would warrant further discussion and actions?

Julien R Pierre - Sun Microsystems

unread,
Jan 6, 2009, 12:33:07 AM1/6/09
to
Kyle,

Kyle Hamilton wrote:
> On Wed, Dec 24, 2008 at 2:46 PM, Eddy Nigg <eddy...@startcom.org> wrote:
>> On 12/25/2008 12:36 AM, Kyle Hamilton:
>>> To be honest, Mozilla doesn't distribute keytool with Firefox, which
>>> means that I have to try to go into the (unbatchable) interface and
>>> remove the flags one. by. one. by. one. and then select the next
>>> certificate and remove those trust flags, and the next, and the next,
>>> and the next...
>> Kyle, why don't you blow that libnssckbi away from your Firefox
>> installation? Would make it easier I think. (Hope I picked the right one ;-)
>> )
>
> Primarily because I want those certs on one profile, but not another,
> and disk space is kind of at a premium right now. :)
>
> (Oh yeah, if one person who uses a computer doesn't want the built-in
> roots, but another does, they have to have separate Firefox
> installations.)

No. You can just move the libnssckbi.so to a location that won't
automatically be picked up by firefox in all profiles.

Then load libnssckbi.so explicitly only in the one profile in which you
want it, using "Manage security devices" / "Load" . It is just a PKCS#11
module, after all.

Julien R Pierre - Sun Microsystems

unread,
Jan 6, 2009, 1:28:17 AM1/6/09
to
Kyle,

Kyle Hamilton wrote:
> I am minded of the CRL entry reason "remove from CRL". Does NSS
> properly handle that reason-code?

The reason code "remove from CRL" is only applicable to delta CRLs. In
addition, this is only allowed if the certificate had the status of "on
hold" in the base CRL. You cannot otherwise unrevoke other certificates
according to RFC3280 and its replacements.

Currently, NSS does not support delta CRLs. Neither does libpkix.
So, the answer is no, this particular reason code is not handled by NSS
at this time.

But a temporary revocation can still be dealt with without the use of
delta CRLs. libpkix can fetch a full CRL where a certificate entry has
the reason code of "on hold", and will be treated as revoked. And if the
CA unrevokes it later, libpkix can pick up the next full CRL from the CA
that no longer lists that certificate.

timeless

unread,
Jan 6, 2009, 3:41:53 PM1/6/09
to time...@gmail.com
On Dec 25 2008, 12:36 am, "Kyle Hamilton" <aerow...@gmail.com> wrote:
> To be honest, Mozilla doesn't distribute keytool with Firefox, which
> means that I have to try to go into the
> (unbatchable) interface

this is false.

the ui is built as xul with js bindings to c++ objects which use idl
to expose methods. the js *script* which controls the user interface
itself is essentially batching orders.

you're free to batch this as much as you please.

> remove the flags one. by. one. by. one. and then select the next
> certificate and remove those trust flags, and the next, and the next,
> and the next...
>

> ...for all hundred or so certs that Firefox includes.

i've done this something like half a dozen times using a nokia n800
(or was it a nokia 770) with the built in certificate manager. Which
is worse by far than the one mozilla ships. You almost have my
sympathy.

Except for a few details:
1. i've been working w/ the nokia ui people + engineers to improve
their mess (i thought I had succeeded in burying their ui, but it
seems rumors of its demise were greatly exagerated).
2. i've been working to improve the mozilla ui (by writing patches)

what have you done?

> And then, once I DO manage to do that, then with the "new and
> improved" user interface updates, I then have to click at least six
> times to try to figure out what's going on, and then when I do find a
> site that's protected by an unknown CA certificate

> (OR that I've removed the trust bits on),

again, i've filed bugs and am working to improve this.
what have you done?

> I have to do the following:
>
> 1) Click 'add an exception'
> 2) click 'get certificate' (why I should have to do this is beyond me,
> since firefox obviously already has the certificate downloaded since
> it told me 'sec_error_untrusted_issuer', which it couldn't have known
> without the certificate in its possession ANYWAY)

i believe this is partly to force users (not you, real normal people)
to think before they blindly add issuers.

There's a public bug evidencing that normal users might actually add
trust to every site they encounter (because they're on an evil hot
spot which is spoofing everything).

You're a (professed) expert, our target audience is the average person
(described above), they experience for that person must be safe and
slow. thinking is good. blindly clicking through is bad.

if you're an expert, you can script pieces of this (heck, there's a
pref to speed up the steps you're describing).


> 3) click 'view'
> 4) get the name of the Issuer
> 5) hope to all the gods that there's enough information in the chain
> to figure out what root it's supposed to be going to

if there isn't, then you shouldn't be trusting it.

heck. if there isn't, go try to find a phone number and get the web
server operator to fix their server.

-- and yes, i've done this, iirc it was last month, i got sun to fix
one of their servers.

> 6) close the window
> 7) go into Preferences
> 8) click Advanced
> 9) click Encryption
> 10) click 'View Certificates'
> 11) Scroll through the list, with each click giving me approximately
> 0.6 useful results (given the preponderance of 'section headings by
> root owner', which by the way doesn't work at all with the Addtrust AB
> stuff since those are Comodo roots)

i've written a patch to improve this ui (with an eye to making the
n800 user experience better).

> 12) find the appropriate root and re-enable it for identification of websites

this seems useless. w/ my patch you could search by any criteria.

> 13) refresh the page.
>
> How 'bout this, Nelson (and I invite Frank and the entire security UI
> team to do this, as well): YOU do it. Create a new profile and
> manually remove the trust on every CA. Then, browse around, and see
> which CAs are actually used by you in your day-to-day browsing,
> reenabling them manually (since you're trying to emulate not having
> keytool around).

been there, done this.

> Furthermore, even when keytool IS available, it's entirely likely that
> its name conflicts with Java's keytool. (especially on Mac OSX.)

it's called certutil.

> This is completely unworkable, and discourages users that want to from
> taking their security into their own hands.

timeless

unread,
Jan 6, 2009, 5:08:34 PM1/6/09
to time...@gmail.com
On Dec 31 2008, 12:28 am, "Kyle Hamilton" <aerow...@gmail.com> wrote:
> (note: "unknown_issuer" without talking at all about who the issuer
> claims to be

you're missing a critical point:

the issuer is something about which we know nothing.

someone could claim "issuer: GOD" or "issuer: POTUS" or "issuer:
VeriSign". Without verifying the issuer, we can't and should neither
attest to nor show it.

And sadly, that's why it isn't shown.

Now, we could perhaps show a fingerprint (minus the fact that MD5 is
at risk), but I tried searching for some fingerprints and haven't
gotten good hits.

http://eklhad.net/linux/app/ssl-certs turns up for MD5 fingerprint
searches, but nothing shows up for the sha1 fingerprint i checked.

- the nss certutil code appears to be able to print sha1s too

> -- and being able to download a certificate and then accept it

it's true we don't do particularly well with chains, however i've
rarely seen a useful misconfigured server with a partial chain and a
missing root. if someone provides me with such a service, i'll see
about trying to improve the user experience -- note that i'd prefer to
start with an instance of a real broken server, since otherwise it's
fairly pointless, however i could do the work from an example.

> without having to see who it's issued by -- is a "WTF WAS
> THE SECURITY TEAM THINK--WAIT, WAS THE SECURITY TEAM
> THINKING??!!!!" situation.

they were thinking about it more than you were. calmer heads with more
thought prevailed. and what's important is that when our users get
angry or panic, we don't want them accidentally doing something
they'll regret later. (hopefully you regret shouting in a public
forum. personally i tend to regret each time i post anywhere.)

Jean-Marc Desperrier

unread,
Jan 8, 2009, 4:54:33 PM1/8/09
to
Eddy Nigg wrote:
> [...] We
> received already calls from people confusing us with them.
>
> - *certstar.com* as opposed to *cert.startcom*.org

Then sue them really. A concurrent that use a company name that brings
confusion for ordinary people is a typical case in which you can sue.

Call your lawyer, you'll need to build a proper juridic case, and those
calls that prove that ordinary people get confused will be very useful
for that.

Ian G

unread,
Jan 12, 2009, 1:27:03 PM1/12/09
to mozilla's crypto code discussion list, rob.st...@comodo.com
On 12/1/09 12:08, Rob Stradling wrote:
> If Mozilla want to "hold a CA to doing something",


Agreed, right now, Mozilla has no effective tools to hold anyone to
anything.

I would suggest we think in terms of "dispute resolution." This is more
general than "Mozilla holds a CA to something" but the result is more
robust, IMO.

I posted my view of what this would be 23rd december.

Question then is; do people feel comfortable about posting the message
of 23rd December as a "dispute resolution page" on the wiki under
something like /CA:disputes ?

iang

minor notes:

1. Because such guidance has legal ramifications, I'd say Mozo has a
veto on this suggestion, regardless of outsider's views and desires.

2. one practical question is #3, the precise name of the "owner".

3. I should point out that I tried to strictly reverse-engineer the
current situation, not re-engineer it. I think we should document where
we are right now; think about it, and then advance that.


On 23/12/08 14:58, Ian G edited:

> 1. How to file a dispute against Mozilla and/or a CA. This seems fairly
> easy; file a bug in bugzilla and mark it as already described here:
> https://wiki.mozilla.org/CA:How_to_apply . (With mods: Severity: as
> appropriate.)
>
>
>
> 2. What is the practice on revocation or un-trusting on roots? This is
> is perhaps a headline case of a dispute, so possible merits its own
> notes. Frank has suggested the test:
>
> a. there is a clear and present danger to Mozilla users, or
> b. to punish a CA, and/or to deter others.
>
> I would suggest that (b) be modifed slightly to give it a basis, which
> in this case might be "for breaches of policy or practices". The Policy,
> pt 4, gives the authority, as well as some examples, and adds another
> test case:
>
> c. where certificates cause technical problems with mozo software.
>
>
>
> 3. How to resolve a dispute. This is a Mozilla action & responsibility.
> Reverse-engineering and referring, I would suggest this as a teaser:
>
> a. The CA certificate "module owner" at Mozilla foundation is
> responsible. Ref, the policy, pt 15.
> b. The dispute is investigated and ruled on by module owner.
> c. The ruling is listed in the bug report above.
> d. Many disputes will be dealt with by communication, and no ruling will
> be required. This will create a default "closed, no action" ruling.
>
>
>
> 4. Finality. What happens if we disagree with the decision of the module
> owner? In the policy, it says "CAs or others objecting to a particular
> decision may appeal to mozilla.org staff, who will make a final
> decision." Ref, policy, pt 15.
>
> I would wonder about this; google suggests that "staff" is as listed here:
> http://www.mozilla.org/about/staff
> but that seems out of date. Also, due to the absence of this forum in
> the public eye, I doubt it musters the credibility we need in dispute
> review where the legal and contractual significance is high. E.g., is
> there any way we can review the decisions they made in the past?
>
> There are several possibilities:
>
> (i) Ruling is final.
> (ii) Mozilla.org staff, policy, pt 15.
> (iii) Review by board of Mozilla Foundation.
> (iv) Review by some independent party.
> (v) Review by forum at law: courts, or Arbitrator.
>
> Personally, I would plumb for (iii) and suggest the Mozo Foundation
> board as the next step. It is expensive, but available. The directors
> already have fiduciary responsibility, and can thus deal with the
> significance. It is also aligned with the review of the manager
> concerned, the policy and the general contractual issues.

Eddy Nigg

unread,
Jan 12, 2009, 1:37:53 PM1/12/09
to
On 01/12/2009 03:27 PM, Ian G:

> I would suggest we think in terms of "dispute resolution."

There is no dispute! Perhaps you can tell me what the dispute is?

> Question then is; do people feel comfortable about posting the message
> of 23rd December as a "dispute resolution page" on the wiki under
> something like /CA:disputes ?

-1

A specific failure of a CA or a request for review and changes of
practices and policy of a CA and even a removal of a root doesn't have
to be due to a dispute.

Ian G

unread,
Jan 12, 2009, 2:40:46 PM1/12/09
to mozilla's crypto code discussion list
On 12/1/09 14:37, Eddy Nigg wrote:
> On 01/12/2009 03:27 PM, Ian G:
>> I would suggest we think in terms of "dispute resolution."
>
> There is no dispute! Perhaps you can tell me what the dispute is?
>
>> Question then is; do people feel comfortable about posting the message
>> of 23rd December as a "dispute resolution page" on the wiki under
>> something like /CA:disputes ?
>
> -1
>
> A specific failure of a CA or a request for review and changes of
> practices and policy of a CA and even a removal of a root doesn't have
> to be due to a dispute.


refer to point 1 in that list:


> 1. How to file a dispute ....
> file a bug in bugzilla ...


If a dispute is not filed in bugzilla and so labelled, then there is no
dispute within the system.

For the sake of harmony and focus, maybe anything being discussed now
should be treated as "old".

iang

Eddy Nigg

unread,
Jan 12, 2009, 6:18:10 PM1/12/09
to
On 01/12/2009 04:40 PM, Ian G:

>
> refer to point 1 in that list:
>
>
> > 1. How to file a dispute ....
> > file a bug in bugzilla ...
>
>
> If a dispute is not filed in bugzilla and so labelled, then there is no
> dispute within the system.

1.) Disputes are perhaps solved by the parties directly (in whatever way
necessary). The event with Comodo is not a dispute between Mozilla and
Comodo, because Mozilla hasn't made any requests or requirements to
which Comodo hasn't complied (at least this is what we know). Hence
there hardly can be dispute at all. Would Comodo and Mozilla disagree
with each other, perhaps we could call it a dispute, however Mozilla has
other options and doesn't require the consent of Comodo in order to
solve an issue.

2.) Disputes between other parties (relying parties, certification
authorities, other third parties) is certainly not under the
jurisdiction of Mozilla. Some things may be of interest for Mozilla, but
that's about all...

3.) There is something missed???

Ian G

unread,
Jan 16, 2009, 1:40:49 PM1/16/09
to mozilla's crypto code discussion list
As threatened, I've posted my thoughts on dispute resolution here [1]:

https://wiki.mozilla.org/CA:Dispute_resolution

The first point is to document where we are now. And, hopefully agree.
And, to identify and fill some obvious holes [2].

The final point is to think about where to evolve into the future.

Which immediately raises the question: why is this important?

Next post.

iang


[1] It's important to note that Mozo people (nor Mozo) have not
commented on the essence of this one way or another [1]. So it is still
a thought experiment, albeit shared and in words. Comments seen were

* "module ownership" which has since been resolved
https://wiki.mozilla.org/Module_Owners_Activities_Modules
lists Frank as module owner now.

* the applicability of "staff" which has been explained as outdated.

* Eddy's nays which hopefully are addressed in next post.


[2] In that document, the "where to appeal / finality" section is
indeed just that: an obvious hole that should be filled.

Ian G

unread,
Jan 16, 2009, 1:53:08 PM1/16/09
to mozilla's crypto code discussion list
On 16/1/09 14:40, Ian G wrote:

> https://wiki.mozilla.org/CA:Dispute_resolution
....


> Which immediately raises the question: why is this important?

It's important to understand why a slightly more formal and documented
approach to dispute resolution is a useful and valuable direction for
all of us.

First the recent history, as a context: In the past month we saw a
series of events (4 to my counting) that brought to the forefront the
potential for trouble. During those events, some people may have
crossed the line, as far as legal and commercial and ethical behaviour went.

My point here is not to name names, or point the finger, and indeed, we
can all recognise that this will just result in counter-naming and
counter-fingering. Which is just lost efficiency.

My point is this: Crossing the line means more loss of efficiency and
more trouble. The former wheel spinning and distractions we can all
appreciate ... but let me just mention the latter. If someone crosses
the line in terms of legal behaviour, *and* a court case happens, this
will be very expensive for them, because they will have to pay the fees
as the respective attorneys battle it out [1].

So, back to efficiency. It is very useful for all here and for Mozilla
to reduce the loss of efficiency. One way we can do this is to clearly
channel disputes and set-up the rules by which the flaws can be opened
up and dealt with. That is, if you act in the way described HERE, then
you are in a dispute and you have to take it seriously. Follow the
rules HERE. You can name names, list flaws and bugs, etc, but do it HERE.

(Then, if you are not in a dispute, then you should follow the normal
rules found elsewhere in the rest of society, which deals with any
questions in its own way.)

That is, to a certain extent, we create a forum where names can be
named, flaws can be written about, and the participants will get a
certain degree of protection. Do it elsewhere, outside, and you get no
protection. When you go into court, your defence [2] is that "Mozilla
procedure for making these forms of complaints is HERE. That's what I did."

Finally, it is also worth pointing out that this applies as much to
Mozilla as anyone else. Consider the policy which states that Mozo can
e.g., drop the root. If Mozo were to go ahead and drop a root, without
a relatively clear procedure for that, then I would speculate [3] it
would have trouble surviving a challenge in court. However, if Mozo has
the right (in policy) and the procedure (that new page) and follows the
procedure (in bugzilla) then it is in a much stronger position.

iang

[1] this is generally true regardless of the merits. The love of
lawyers is the most expensive thing you will ever pay for in your life!

[2] This is not a complete defence, of course. Let's not get distracted
on binary defences, they don't exist.

[3] see thread "dropping the root is useless" and other popular hits of
2008 :)
http://groups.google.com/group/mozilla.dev.tech.crypto/browse_thread/thread/45e488d613080d71

Eddy Nigg

unread,
Jan 16, 2009, 3:42:52 PM1/16/09
to
On 01/16/2009 03:40 PM, Ian G:

> As threatened, I've posted my thoughts on dispute resolution here [1]:
>
> https://wiki.mozilla.org/CA:Dispute_resolution
>

Ian, I think you must differ between filing a complaint about a CA and
dispute resolution. I think Mozilla doesn't, can't and don't want to
solve disputes.

I'm sorry, but I'd recommend to remove this page altogether. Complaints
which effects the security of users can be posted at the mailing lists,
Bugzilla and perhaps to Frank directly. Disputes are handled elsewhere.

Ian G

unread,
Jan 16, 2009, 4:16:01 PM1/16/09
to mozilla's crypto code discussion list
On 16/1/09 16:42, Eddy Nigg wrote:
> On 01/16/2009 03:40 PM, Ian G:
>> As threatened, I've posted my thoughts on dispute resolution here [1]:
>>
>> https://wiki.mozilla.org/CA:Dispute_resolution
>>
>
> Ian, I think you must differ between filing a complaint about a CA and
> dispute resolution. I think Mozilla doesn't, can't and don't want to
> solve disputes.
>
> I'm sorry, but I'd recommend to remove this page altogether.


I recognised that Mozo might think it too adventurous, so I left off the
link from the mail CA:Overview page.


> Complaints
> which effects the security of users can be posted at the mailing lists,
> Bugzilla and perhaps to Frank directly. Disputes are handled elsewhere.


What is the difference? A complaint is a dispute as far as I can see.

Once you name a CA and put in a "complaint" about its practices, this is
a dispute.

mailing lists: these are for discussions, not complaints. Bugzilla
exists to *track* such things.

mail to Frank: we don't want to set up the back-channels. This is an
open organisation, not a soap opera. It's not polite and it isn't good
policy. Bugzilla has the ability to have "confidential" bugs if such is
needed.

iang

Eddy Nigg

unread,
Jan 17, 2009, 12:03:19 AM1/17/09
to
On 01/16/2009 06:16 PM, Ian G:

>
> What is the difference? A complaint is a dispute as far as I can see.
>

Dispute:

"A controversy or dispute occurs when parties actively disagree, argue
about, or debate, a matter of opinion. Controversies can range in size
from private disputes between two individuals to large-scale
disagreements between societies."

There are other definitions for "Disputes". A dispute happens between
different parties and "dispute resolution" is the process of resolving
disputes between parties.

> Once you name a CA and put in a "complaint" about its practices, this is
> a dispute.

Complaint:

"A complaint is an expression of displeasure, most likely accompanied
with a claim made about another party."

Now, just to make it clear, I'm all in favor to have formal rules and
procedures applied, but I don't agree on the scope and aims of your
proposal.
Mozilla might be party to a dispute, but in no way can provide dispute
resolution. Mozilla can decide to perform certain actions by being a
party to a dispute between itself and a CA (which requires active
disagreement between those two). Mozilla can not act as an instance for
dispute resolution between subscribers, relying parties, certification
authorities and itself. With Mozilla itself being a relying party, the
most, it can be party to a dispute.

Mozilla has its own interests as a relying party, whose interests might
not necessarily be the same interests as those of other relying parties.
However Mozilla might decide to perform certain actions, based upon
knowledge due to a complaint, in order to protect its own interests.
There is an inherent difference between this and your proposal I think.

Ian G

unread,
Jan 17, 2009, 9:45:01 AM1/17/09
to mozilla's crypto code discussion list
On 17/1/09 01:03, Eddy Nigg wrote:
> On 01/16/2009 06:16 PM, Ian G:
>>
>> What is the difference? A complaint is a dispute as far as I can see.
>>
>
> Dispute:
>
> "A controversy or dispute occurs when parties actively disagree, argue
> about, or debate, a matter of opinion. Controversies can range in size
> from private disputes between two individuals to large-scale
> disagreements between societies."
>
> There are other definitions for "Disputes". A dispute happens between
> different parties and "dispute resolution" is the process of resolving
> disputes between parties.
>
>> Once you name a CA and put in a "complaint" about its practices, this is
>> a dispute.
>
> Complaint:
>
> "A complaint is an expression of displeasure, most likely accompanied
> with a claim made about another party."


OK, these would be my points:

Once you name another party, then you are likely in dispute. Once you
specify a remedy, an action that should be done, then you are likely in
dispute. Once someone crosses a line, then that should be treated a
dispute.

Saying something is only a complaint is like a word game; it might
allow you to think you can say stuff without it being treated properly.
But, this is a false sense of security.

It's also irresponsible and unprofessional, and likely if we want to
make this distinction, we will have to think about a policy of no
complaints. You can't have your cake and eat it too.


> Now, just to make it clear, I'm all in favor to have formal rules and
> procedures applied, but I don't agree on the scope and aims of your
> proposal.
> Mozilla might be party to a dispute, but in no way can provide dispute
> resolution.


Mozilla can provide dispute resolution, and I would argue they are
already doing it, albeit badly.

It is an open question as to whether they should continue on that path;
it is another open question whether they can get off that path.

But they certainly can do it, I'm unsure what you mean by "in no way."


> Mozilla can decide to perform certain actions by being a
> party to a dispute between itself and a CA (which requires active
> disagreement between those two).


I doubt it can do that at the moment. If the CA doesn't want that act,
then Mozilla would have to be very careful. Very careful might be
defined as doing nothing, or following a procedure that was documented
and agreed, or doing what it was told. That's a problem with disputes,
one has to be very careful.


> Mozilla can not act as an instance for
> dispute resolution between subscribers, relying parties, certification
> authorities and itself. With Mozilla itself being a relying party, the
> most, it can be party to a dispute.


Well, that is an important point; that is a conflict in its handling of
disputes. We could argue that Mozilla is not an independent party. We
could agree to accept that, or seek an alternate.


> Mozilla has its own interests as a relying party, whose interests might
> not necessarily be the same interests as those of other relying parties.


Right, on this I agree. There is a sea of interests to be covered here,
and it is not even clear that mozo people can represent end-users [1].

As far as choosing to add a root to the list, Mozilla Foundation have
already established their place. They have already declared that they
will act according to a policy and set of interests (whatever they may be).

As far as a relying party goes, they are the owner of the list, so they
have a compelling interest in running that list, and I suggest that they
stick with the responsibility. For now, because there aren't any good
alternatives that I see.


> However Mozilla might decide to perform certain actions, based upon
> knowledge due to a complaint, in order to protect its own interests.
> There is an inherent difference between this and your proposal I think.


Indeed. In my proposal it is less arbitrary. They will be able to do
certain things, more so than before. Right now, they cannot drop anyone
from the root list. Right now, about the only thing they can do without
fear is to write and ask stuff. This proposal will make it easier to act.

iang

[1] Two recent comments, Jonathon put it like this: "Obviously [we
are] representing a browser, but Mozilla's interests tend to align with
end users most of the time." versus Julien's "Typically the NSS
priorities are dictated by the needs of the companies that employ most
of the NSS developers, that is to say, Sun, Red Hat and Google, and
these can change over time."

Eddy Nigg

unread,
Jan 17, 2009, 3:19:03 PM1/17/09
to
On 01/17/2009 11:45 AM, Ian G:

> Once you name another party, then you are likely in dispute.

Might be.

> ...then that should be treated a
> dispute.

The question is, if Mozilla is a party to that dispute. Making a
complaint about a CA might be of interest for Mozilla (in its own
right), but Mozilla still isn't part of that dispute.

> It's also irresponsible and unprofessional, and likely if we want to
> make this distinction, we will have to think about a policy of no
> complaints. You can't have your cake and eat it too.

Well, no...somebody is free to complain or inform Mozilla about
something which might affect Mozilla. If Mozilla can/should/must act
upon that information greatly depends on the nature of that information
and how this affects its users. This is the assessment Mozilla will have
to make and perform certain actions or not. A dispute still mustn't
arise, because the affected CA might not even dispute the claim, might
be cooperative, might solve the problem or whatever.

An affected party which has a dispute with a CA must solve the dispute
with the CA through the proper channels. Mozilla CAN NOT provide
resolution to the dispute - Mozilla is not a court.

> Mozilla can provide dispute resolution, and I would argue they are
> already doing it, albeit badly.

Not that I can see...at most Mozilla is party to a dispute (so far I
can't see that Mozilla has a dispute with any CA, do you?)

> But they certainly can do it, I'm unsure what you mean by "in no way."

Mozilla has no legal powers for solving disputes. Mozilla can only
protect itself and its users if such a risk exists. Mozilla might in
turn become part to a dispute, but not necessary.

>
> Right, on this I agree. There is a sea of interests to be covered here,
> and it is not even clear that mozo people can represent end-users [1].

Mozilla does not represent any end-user! Full stop! Mozilla makes on
behalf of its users a decision which CA roots to ship with its software.
Users have obligations as well and will have to solve disputes through
the proper channels. Mozilla is not such a channel.

>
> As far as choosing to add a root to the list, Mozilla Foundation have
> already established their place. They have already declared that they
> will act according to a policy and set of interests (whatever they may be).
>

Up to a certain extend. It declared under which condition it may add a
root and under which circumstances it may remove it. Mozilla hasn't
defined which action to perform under which circumstances nor promises
to do so, but clearly retains the right to do so at any time.

Now, there is of course another reality besides the legalese... Mozilla
wouldn't want to put its users to undue risks because it would be bad
for its software and standing.

> Indeed. In my proposal it is less arbitrary. They will be able to do
> certain things, more so than before. Right now, they cannot drop anyone
> from the root list.

I'm not sure from where you got this...which law are you following that
prevents Mozilla from doing so? Which legal requirements? Which
policies? Which agreements?

> Right now, about the only thing they can do without
> fear is to write and ask stuff. This proposal will make it easier to act.

Sorry Ian, I can't see how your page should change anything. Besides
that I don't agree with your statement above to start with...however, if
you feel that Mozilla should have performed certain actions and hasn't
done so, than you should speak up and complain (to Mozilla for
inaction). If you want to define certain actions Mozilla should perform
under certain circumstances, than rally Mozilla behind such a proposal
and have the foundation agree to it. It has however still nothing to do
with dispute resolution.

Ian G

unread,
Jan 17, 2009, 9:32:29 PM1/17/09
to mozilla's crypto code discussion list
On 17/1/09 16:19, Eddy Nigg wrote:
> On 01/17/2009 11:45 AM, Ian G:
>> It's also irresponsible and unprofessional, and likely if we want to
>> make this distinction, we will have to think about a policy of no
>> complaints. You can't have your cake and eat it too.
>
> Well, no...somebody is free to complain or inform Mozilla about
> something which might affect Mozilla.


Well, no. No-one is free to do that. That is, certainly anyone can say
anything but there are consequences, in business, in reputation, in law,
in relationships. These form limits and controls on behaviour. None of
us here are immune to this.


> An affected party which has a dispute with a CA must solve the dispute
> with the CA through the proper channels. Mozilla CAN NOT provide
> resolution to the dispute - Mozilla is not a court.


Um. It's not really the business of this list to talk general business,
but ok, to avoid any further mistakes here, here is the very basic
situation:

Disputes are resolved according to the agreement or contract (terms are
interchangeable) between the parties.

So, if the agreement were to say "use this page" then that's how they
are done. If on the other hand, the agreement said "disputes are solved
by 3 wise men rolling dice under a full moon" then that is how it is
done too.

Now, normally, contracts say "disputes are resolved in the Court of Law
in Mainstreet, Upper Podunk." Which is fine and dandy, but this is
simply an option.

In contract law, forums of dispute resolution is chosen by the parties.
There are all sorts of options: courts (any around the world),
mediation, arbitration, one party, both parties, choose a third party.
It's whatever you want, whatever you can write in the contract (in general).

Now, where it gets a bit sticky is that one of the options is to say
nothing. And this is entirely valid and legal, but does lead to
questions about how to resolve disputes later on. This can be resolved
by mutual agreement, or if you can't agree how to resolve the dispute,
one party can take it to a court.

Then, because the court is not NAMED IN THE AGREEMENT, it has to ask
itself "do we have jurisdiction?" And there are lots and lots of rules
and caselaw and laws about this. Assuming luck, then the court says
"yes, we assert jurisdiction" and then the case is reserved for that
court. If we have negative luck, other things can happen.

In sum, courts may resolve disputes. Others may as well. It all
depends on the agreement. Mozilla can simply state that it resolves
disputes ... by simply stating it. No problem.

(Obviously, Mozo's counsel will have something to say here too, and may
well (a ) disagree with the letter above, or (b ) the spirit, or (c )
the suitability of the idea in the first place.)


>> Indeed. In my proposal it is less arbitrary. They will be able to do
>> certain things, more so than before. Right now, they cannot drop anyone
>> from the root list.
>
> I'm not sure from where you got this...which law are you following that
> prevents Mozilla from doing so? Which legal requirements? Which
> policies? Which agreements?


OK, fair question. There is no precise law, but a procedure called an
injunction. It is fairly simple, but like most things is not understood
until you've been through it.

What happens is that party A gets wind of party B's intended action. So
party A files a dispute into court. Then, it immediately makes an
application to the court for an "injunction" which is an order from the
court to maintain things as they are, while the dispute is being heard.

OK so far? So, as we know, most court cases will take at least a year,
and as long as 4 for slightly complicated things. And, here's the clanger:

the injunction generally maintains in place for that entire time.

Now apply to say a root termination. This is a business-threatening
event. If Mozo started thinking about this, then it would very quickly
escalate to court & injunction. In my opinion (not legal advice, of
course) the business would have little option but to sue and file for
injunction.

Ergo, I conclude by the above logic that Mozo cannot drop a root. (OK,
I needed some other steps to get to that point, but they are not germane.)

[I've snipped all the stuff that is too far "out of court" to use a pun,
sorry about that.]


iang

Eddy Nigg

unread,
Jan 19, 2009, 9:07:22 AM1/19/09
to
On 01/17/2009 11:32 PM, Ian G:

> Disputes are resolved according to the agreement or contract (terms are
> interchangeable) between the parties.

So far so good...

> So, if the agreement were to say "use this page" then that's how they
> are done. If on the other hand, the agreement said "disputes are solved
> by 3 wise men rolling dice under a full moon" then that is how it is
> done too.

Nice :-)

> Now, normally, contracts say "disputes are resolved in the Court of Law
> in Mainstreet, Upper Podunk." Which is fine and dandy, but this is
> simply an option.

Sure, there are others...

> In contract law, forums of dispute resolution is chosen by the parties.

Sometimes, but parties may even disagree on this.

> There are all sorts of options: courts (any around the world),
> mediation, arbitration, one party, both parties, choose a third party.
> It's whatever you want, whatever you can write in the contract (in
> general).

Sure.

> Now, where it gets a bit sticky is that one of the options is to say
> nothing. And this is entirely valid and legal, but does lead to
> questions about how to resolve disputes later on. This can be resolved
> by mutual agreement, or if you can't agree how to resolve the dispute,
> one party can take it to a court.

Indeed.

> Then, because the court is not NAMED IN THE AGREEMENT, it has to ask
> itself "do we have jurisdiction?" And there are lots and lots of rules
> and caselaw and laws about this. Assuming luck, then the court says
> "yes, we assert jurisdiction" and then the case is reserved for that
> court. If we have negative luck, other things can happen.

Also correct.

> In sum, courts may resolve disputes. Others may as well. It all depends
> on the agreement. Mozilla can simply state that it resolves disputes ...
> by simply stating it. No problem.

But Mozilla hasn't done that so far, nor have the CAs and other parties
agreed to that. Not even Mozilla has raised such an option so far. At
the moment this is fantasy at best and not relevant. That's why I
insisted that your page does assume something which doesn't exist.

> (Obviously, Mozo's counsel will have something to say here too, and may
> well (a ) disagree with the letter above, or (b ) the spirit, or (c )
> the suitability of the idea in the first place.)

"/Something to say/" is good... :-)

>> I'm not sure from where you got this...which law are you following that
>> prevents Mozilla from doing so? Which legal requirements? Which
>> policies? Which agreements?
>

> What happens is that party A gets wind of party B's intended action. So
> party A files a dispute into court. Then, it immediately makes an
> application to the court for an "injunction" which is an order from the
> court to maintain things as they are, while the dispute is being heard.

Things like that can happen, it's just one possibility however.

> Now apply to say a root termination. This is a business-threatening
> event. If Mozo started thinking about this, then it would very quickly
> escalate to court & injunction. In my opinion (not legal advice, of
> course) the business would have little option but to sue and file for
> injunction.
>
> Ergo, I conclude by the above logic that Mozo cannot drop a root. (OK, I
> needed some other steps to get to that point, but they are not germane.)

I think that's just a wild assumption and any CA will check carefully if
it has a case at all against Mozilla. I suspect that if there are real
reasons to remove a root, the expenses for the CA will be too high in
the end - instead it will opt to fix whatever needs to be fixed in order
to get re-added to the pile (as Paul used to say). Hence removing a root
is one option, working with the CA to address the problems another. One
important thing to note is, that CAs will most likely cooperate in
removing their root in case of key compromise.

And besides that, Mozilla reserves the right to remove a root, modify
trust-bits etc. It doesn't have to get into the arbitration business for
that.

Ian G

unread,
Jan 19, 2009, 10:52:45 AM1/19/09
to mozilla's crypto code discussion list
On 19/1/09 10:07, Eddy Nigg wrote:
> On 01/17/2009 11:32 PM, Ian G:
>> In contract law, forums of dispute resolution is chosen by the parties.
>
> Sometimes, but parties may even disagree on this.


Well, at the moment of agreement, they agree. If they disagree later,
then the courts support the agreement, as a principle. They are not
likely to knock down a "reasonable agreement" whatever that means.


>> In sum, courts may resolve disputes. Others may as well. It all depends
>> on the agreement. Mozilla can simply state that it resolves disputes ...
>> by simply stating it. No problem.
>
> But Mozilla hasn't done that so far, nor have the CAs and other parties
> agreed to that. Not even Mozilla has raised such an option so far.


https://wiki.mozilla.org/CA:Dispute_resolution#Introduction

Introduction [edit]

This is a tentative step to

1. document where Mozilla's CA area is at with disputes
2. set the scene for the future.

The first step is to document where we are now, with little emphasis on
suggestions for change. Originally [on dev.tech.crypto].


> At the moment this is fantasy at best and not relevant.
> That's why I
> insisted that your page does assume something which doesn't exist.


Mozilla is resolving disputes. It just hasn't said it, nor thought
about how it is doing it.

Now, we can either go two ways here:

* document what's going on and improve it
* back out

Which do you prefer? As I believe we are responsible people here, we
take our mission seriously, and we have lots of messy stakeholder issues
to deal with, I'm not interested in the following options:

* pretending it doesn't exist
* playing word games
* having our cake and eat it too
* find ourselves in court


>> (Obviously, Mozo's counsel will have something to say here too, and may
>> well (a ) disagree with the letter above, or (b ) the spirit, or (c )
>> the suitability of the idea in the first place.)
>
> "/Something to say/" is good... :-)


Right. Let me underscore _Something To Say_ here :)


>>> I'm not sure from where you got this...which law are you following that
>>> prevents Mozilla from doing so? Which legal requirements? Which
>>> policies? Which agreements?
>>
>> What happens is that party A gets wind of party B's intended action. So
>> party A files a dispute into court. Then, it immediately makes an
>> application to the court for an "injunction" which is an order from the
>> court to maintain things as they are, while the dispute is being heard.
>
> Things like that can happen, it's just one possibility however.


The question is, what matters? In business, this matters because it
drives behaviour. Other "possibilities" don't matter because they don't
drive behaviour.


>> Now apply to say a root termination. This is a business-threatening
>> event. If Mozo started thinking about this, then it would very quickly
>> escalate to court & injunction. In my opinion (not legal advice, of
>> course) the business would have little option but to sue and file for
>> injunction.
>>
>> Ergo, I conclude by the above logic that Mozo cannot drop a root. (OK, I
>> needed some other steps to get to that point, but they are not germane.)
>
> I think that's just a wild assumption and any CA will check carefully if
> it has a case at all against Mozilla. I suspect that if there are real
> reasons to remove a root, the expenses for the CA will be too high in
> the end - instead it will opt to fix whatever needs to be fixed in order
> to get re-added to the pile (as Paul used to say). Hence removing a root
> is one option, working with the CA to address the problems another. One
> important thing to note is, that CAs will most likely cooperate in
> removing their root in case of key compromise.


Of course. All those things will happen, as long as the root isn't removed.


> And besides that, Mozilla reserves the right to remove a root, modify
> trust-bits etc. It doesn't have to get into the arbitration business for
> that.


It has to get into the business in order to use that right. If it just
reserves the right, and doesn't use it, no problem.

The way to think of this as a security person is to think like an
attacker. What happens in an aggressive situation? Well, the sides
fight. What's the easiest way to fight? That depends on who has how
many lawyers...

iang

Eddy Nigg

unread,
Jan 20, 2009, 12:22:05 AM1/20/09
to
On 01/19/2009 12:52 PM, Ian G:

> Mozilla is resolving disputes. It just hasn't said it, nor thought about
> how it is doing it.

Well, it's my point that I think that Mozilla doesn't, hasn't and
shouldn't resolve disputes. However, continue below....

> * document what's going on and improve it
> * back out

Neither. I'd rather would suggest procedures and rules how certain
issues should to be dealt with. If this is the improvement you are
seeking, than I'm with you. However I believe that it must be
Mozilla/User centric to the benefit of both. This is what I believe
Mozilla cares. Mozilla doesn't need to resolve disputes, it must know
what to do under certain circumstances in order to protect itself and
its users. Those are two different kind of things.

> * having our cake and eat it too

Yes :-)

Besides, why should Mozilla care beyond that?

> * find ourselves in court

Well, Mozilla has a legal department and lawyers taking care of it
should need arise.

>
> Of course. All those things will happen, as long as the root isn't removed.

No, I think that's not correct. This comment is perhaps most authentic
in this respect: https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c24

> It has to get into the business in order to use that right. If it just
> reserves the right, and doesn't use it, no problem.

I simply don't get it...why the h*** does Mozilla have to put itself
into such a position between two different parties with arbitration? All
it cares is itself and its users!?

Ian G

unread,
Jan 20, 2009, 4:50:07 PM1/20/09
to mozilla's crypto code discussion list
On 20/1/09 01:22, Eddy Nigg wrote:
> On 01/19/2009 12:52 PM, Ian G:
>> Mozilla is resolving disputes. It just hasn't said it, nor thought about
>> how it is doing it.
>
> Well, it's my point that I think that Mozilla doesn't, hasn't and
> shouldn't resolve disputes. However, continue below....
>
>> * document what's going on and improve it
>> * back out
>
> Neither. I'd rather would suggest procedures and rules how certain
> issues should to be dealt with. If this is the improvement you are
> seeking, than I'm with you. However I believe that it must be
> Mozilla/User centric to the benefit of both.


So far, no dispute :)


> This is what I believe Mozilla cares.
> Mozilla doesn't need to resolve disputes, it must know
> what to do under certain circumstances in order to protect itself and
> its users. Those are two different kind of things.


OK, so let's say you are Mozo, coz you know what they care about, and it
is YOUR DAY IN COURT! Because you care.

Imagine a CA has sued you, and a bunch of users are lining up a
class-action. The rest of the CAs are up in arms about the favouritism,
and the media smells blood. The lobbying and public opinion thing is in
full swing [1].

Now, explain to the judge what the difference between a dispute and
protecting self and protecting users is?


>> * having our cake and eat it too
>
> Yes :-)
>
> Besides, why should Mozilla care beyond that?


Oh, it doesn't need to. Unless it has a mission about protecting
end-users. Unless there's business involved. Unless it ends up in
court. Unless there is a claim of security, and a promise of
protection. Unless there is doco saying this is important. Unless
there is a duty, or an asymmetry, or a process. Unless they have
imposed something.

I mean, it is really easy to create a strategy where we don't need to
care, we ... just don't care about things.


>> * find ourselves in court
>
> Well, Mozilla has a legal department and lawyers taking care of it
> should need arise.


Ah. Add another one to the list!

* pass the buck, they got the money.

Actually, I disagree. They don't have the lawyers, and the first one
who tries this *will* have the lawyers.

Who really has the lawyers?


>> Of course. All those things will happen, as long as the root isn't
>> removed.
>
> No, I think that's not correct. This comment is perhaps most authentic
> in this respect: https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c24


Authentic it a good word!

I see some techies discussing a dispute without being aware of what they
are doing. This is normal when there is no policy or business angle.
It's cool and fun in an open source context, because business doesn't
matter; the wonderful open source invention separates the business out
of the code perfectly.

However, we aren't talking about *code* but about *business*. They are
unaware that they've drifted out of their familiar territory.

So, what we got? A bunch of slashdotters, itching to wipe out a
business, they know where the kiloton-revoke button is, and they want to
push it? But they haven't a clue about the megaton-response button that
their competitor has.

An authentic disaster in the making, is that your point?


>> It has to get into the business in order to use that right. If it just
>> reserves the right, and doesn't use it, no problem.
>
> I simply don't get it...why the h*** does Mozilla have to put itself
> into such a position between two different parties with arbitration? All
> it cares is itself and its users!?


Well, you've laid it out better than I could :)

The point here is not how it is done, but something is done; which was
why I started documenting it.

Either Mozo does something, or it loses control. If for example, the
process ends up in courts [2] then the courts will throw out Mozilla's
interests, the end-users' interests, likely every other CA's interests
as well. I guarantee it, or sue me :)

Security is a non-starter. After a court has finished listening to the
security experts of both sides, you'll wish you never uttered the word.

One advantage of having Mozilla resolve most disputes is that it
actually understands [3] the business. And the end-users. And the CAs.
And the certs. And the politics. And the conspiracies. And the
cartels. And the mistakes.

Good luck on trying to bring a judge up to speed on all that. I'm glad
you know about that *lots of money* thing, it's going to be needed.

iang

[1] Interesting article... may be germane or may be not:
http://www.wired.com/techbiz/it/magazine/17-02/ff_killgoogle?currentPage=all

[2] another little question, which court?

[3] at least, it has a good chance of doing this, arguably better than
others.

Eddy Nigg

unread,
Jan 21, 2009, 10:16:22 AM1/21/09
to
On 01/20/2009 06:50 PM, Ian G:

>> This is what I believe Mozilla cares.
>> Mozilla doesn't need to resolve disputes, it must know
>> what to do under certain circumstances in order to protect itself and
>> its users. Those are two different kind of things.
>
>
> OK, so let's say you are Mozo, coz you know what they care about, and it
> is YOUR DAY IN COURT! Because you care.

Well, that day happens way before you get to court. I've been suggesting
a while ago that CAs sign on the dotted line of the Mozilla CA policy
(at least, if not an outright agreement stating a few things). Mainly
CAs should know, accept and agree to section 4 of the Mozilla CA policy.

> Imagine a CA has sued you, and a bunch of users are lining up a
> class-action. The rest of the CAs are up in arms about the favouritism,
> and the media smells blood. The lobbying and public opinion thing is in
> full swing [1].

Yeah...and ACTION! :-)

Besides that, I simply believe that it never will happen this way
because CAs have an easier and cheaper way fixing whatever needs to get
fixed. Assuming that all CAs will be and are treated equally, I don't
have any reason to believe something else. I've been in contact with
many CAs and this is the general tone. A policy or required practice is
acceptable as long as it applies to all CAs.

> Who really has the lawyers?

Right! Like the water flows the easiest way it can, CAs will do the
same. Paying lawyers and getting into court with Mozilla is perhaps
harder than to apply a change to a certain practice or whatever.


> I see some techies discussing a dispute without being aware of what they
> are doing. This is normal when there is no policy or business angle.
> It's cool and fun in an open source context, because business doesn't
> matter; the wonderful open source invention separates the business out
> of the code perfectly.

Perhaps you haven't read the comment and what I found interesting:

"If we had a "rogue" CA cert then we may well have no objection to you
issuing something that looked like it to assist in the removal of trust
from it. Heck, we'd help you do it."

For their own protection, CAs would like to have the root removed and/or
marked in case of key compromise.

> The point here is not how it is done, but something is done; which was
> why I started documenting it.

OK, great!

> Either Mozo does something, or it loses control.

I agree!

> One advantage of having Mozilla resolve most disputes is that it
> actually understands [3] the business. And the end-users. And the CAs.
> And the certs. And the politics. And the conspiracies. And the cartels.
> And the mistakes.

I think we must strictly limit what matters to Mozilla, define it and
provide procedures and guidelines for different cases.

Lets start with having CAs sign on the dotted line of the Mozilla CA
policy. Actually by applying they already agree to it somehow, but it's
not clear. I'd like to have that improved.

I prefer to call it differently, not dispute resolution. There are many,
many cases of potential disputes in relation to CAs, none of which is
Mozilla's business.


(BTW, it somewhat bothers me....why are you calling Mozilla "Mozo"?
Which abbreviation is it? I know about MoFo (Mozilla Foundation), about
MoCo (Mozilla Corporation), about MoMe (Mozilla Messaging), but what the
heck is Mozo? Mozilla Zoo??? :-) )

0 new messages