....and we are waiting patiently for a posting by a Mozilla
representative here.
--
Regards
Signer: Eddy Nigg, StartCom Ltd.
XMPP: star...@startcom.org
Blog: http://blog.startcom.org/
Twitter: http://twitter.com/eddy_nigg
I appreciate your patience...
(I've been ill, so not online much the past couple of days.)
A good description about what happened has been posted here:
https://www.eff.org/deeplinks/2011/03/iranian-hackers-obtain-fraudulent-https
Comodo has provided their summary of the events here:
http://blogs.comodo.com/category/it-security/
http://www.comodo.com/Comodo-Fraud-Incident-2011-03-23.html
We were informed by Comodo promptly, and they followed the Security Bugs
process: http://www.mozilla.org/projects/security/security-bugs-policy.html
The Security Bug is here:
https://bugzilla.mozilla.org/show_bug.cgi?id=642395
The bad certs were all revoked promptly, but blacklisting allows us to
cope with the possibility that an attacker might block OCSP if they
control the victim's network. OCSP failures aren't fatal by default
(though they do downgrade EV to DV) so we wanted to make sure these
certs never worked.
Mozilla requested that Comodo take the following actions.
1) Revoke the RA rights of the RA concerned. (Done)
2) Monitor their OCSP logs for evidence of use of these certificates, or
evidence that access to their OCSP responders is being blocked from
certain geographical locations
3) Publish a full account of exactly what happened. (See comodo.com
links above)
4) Provide compelling public evidence that:
- the root private key was not compromised
- the channel of mis-issuance is now shut
- all mis-issued certs have been found and listed
- the list of other certs to come through that channel recently has been
carefully audited, and representatives of each organization are
being/have been contacted to confirm correct issuance.
(partially complete, Comodo plans to engage WebTrust auditors)
5) Move, in a short time-frame (small number of months) towards a model
where each RA issues from a separate subordinate CA certificate.
(started)
We are still evaluating the situation, and may request further action
from Comodo.
Additionally, this issue has brought several concerns about
implementation of CRL and OCSP to the forefront, and some new bugs have
been filed. Also, it turned out that an NSS-based fix for this instance
was not as simple as we would like. A new bug has been created for
improving NSS to allow us to add certs to the store which are
specifically untrusted:
https://bugzilla.mozilla.org/show_bug.cgi?id=642503
Comodo responded appropriately to this incident by revoking the
certificates immediately, and notifying the major browser vendors so
that more proactive actions could be taken.
This incident alone is concerning enough, but as we are all aware, there
are previous incidents that have been logged in Bugzilla regarding certs
issued within Comodo’s hierarchy, for example:
https://bugzilla.mozilla.org/show_bug.cgi?id=470897
https://bugzilla.mozilla.org/show_bug.cgi?id=526560
https://bugzilla.mozilla.org/show_bug.cgi?id=599856
Of particular note is the comment in
https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c68
"… we have retrofitted our own DV process into the RA's ordering process
in the vast majority of cases.
By 'our own DV process', I mean that Comodo performs an automated check
of domain control by sending (and confirming receipt of) an email to an
address which is either on the domain to be validated or is explicitly
mentioned in the WHOIS entry for the domain to be validated."
We have asked Comodo to explain why this mechanism was not active (or
did not work) for this RA.
I am extremely concerned about the history of questionable certs being
issued within the Comodo hierarchy. One possible result is for Mozilla
to turn off all of the trust bits for the Comodo root certificates that
are included by default in NSS. However, I think that the CAs actions
in response to the most recent situation should also be taken under
consideration. It would have been much easier for the CA to not say
anything to browser vendors about their mistake – they had caught it
quickly and revoked the certs already. Comodo was very much aware of the
previous issues that had been logged in Bugzilla, and of the possible
outcome of making this mistake public. Regardless, Comodo chose to do
the right thing and disclose this situation to the browser vendors so
that they could also take action.
We could focus on punishment, but I think it would be more beneficial
for us to focus on actions that will prevent such mistakes in the
future. As well as sharing the lessons-learned, to help prevent other
CAs from making the same mistakes.
I ask the representatives of Comodo to contribute actively to this
discussion group. In particular, please described the actions Comodo is
taking now (e.g. the list of requested actions above), as well as the
actions that Comodo plans to take to prevent such a mistake from
happening again.
I have added a few items to
https://wiki.mozilla.org/CA:CertPolicyUpdates#Second_Phase
Under the heading “RAs and Subordinate CAs”
There are some practices, such as some roots signing end-entity certs
directly, that this incident has demonstrated should no longer be
allowed, and need to be corrected soon.
I am also following the CAB Forum and EFF SSL Observatory discussions on
this topic. So no need to copy info from them, unless there are specific
action items that pertain to Mozilla.
Kathleen
What is the definition of "revoke"? Does it have a cryptographic or
operational implication?
> 3) Publish a full account of exactly what happened. (See comodo.com
> links above)
I do not believe that the account is yet "full," as Jake observed:
https://blog.torproject.org/blog/detecting-certificate-authority-compromises-and-web-browser-collusion#Update
As Andy noted, this point...
> 5) Move, in a short time-frame (small number of months) towards a model
> where each RA issues from a separate subordinate CA certificate.
> (started)
....seems mutually exclusive with this point...
> Of particular note is the comment in
> https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c68
> "… we have retrofitted our own DV process into the RA's ordering process
> in the vast majority of cases.
> By 'our own DV process', I mean that Comodo performs an automated check
> of domain control by sending (and confirming receipt of) an email to an
> address which is either on the domain to be validated or is explicitly
> mentioned in the WHOIS entry for the domain to be validated."
>
> We have asked Comodo to explain why this mechanism was not active (or
> did not work) for this RA.
....and the latter seems preferable. The combination, in which the
"parent" does validation and issuance, but uses a different sub-CA for
each RA seems ideal.
> I am extremely concerned about the history of questionable certs being
> issued within the Comodo hierarchy. One possible result is for Mozilla
> to turn off all of the trust bits for the Comodo root certificates that
> are included by default in NSS.
This episode raises concerns not just about Comodo's practices, but
about Mozilla's. How is it that a well-understood problem from 3 years
ago (Comodo's unsafe RA practices) was not confirmed to be resolved by
now? I'll ask again...
....will Mozilla mandate public disclosure of all RAs, and the CA's
precise practices with respect to delegating authority
(cryptographically or operationally) to them?
> However, I think that the CAs actions in
> response to the most recent situation should also be taken under
> consideration. It would have been much easier for the CA to not say
> anything to browser vendors about their mistake – they had caught it
> quickly and revoked the certs already. Comodo was very much aware of the
> previous issues that had been logged in Bugzilla, and of the possible
> outcome of making this mistake public. Regardless, Comodo chose to do
> the right thing and disclose this situation to the browser vendors so
> that they could also take action.
This logic only incentivizes CAs to not perform due diligence under the
assumption that as long as they admit their mistakes publicly they will
not be held accountable. You're right, Comodo was very much aware of
the previous issues that had been logged in Bugzilla... and they even
knew what they had to do to mitigate their harm, and claimed that they
were doing so, but evidently did not.
> We could focus on punishment, but I think it would be more beneficial
> for us to focus on actions that will prevent such mistakes in the
> future. As well as sharing the lessons-learned, to help prevent other
> CAs from making the same mistakes.
How is this time different from last time?
Hmmm. I'm not seeing this posting in Thunderbird, only in Google
Groups.
Yes, for the case where an intermediate CA corresponds to an RA, I was
assuming that the "parent" CA would be in full control of the
intermediate CAs.
I add this clarification to the related bullet point in the wiki page:
https://wiki.mozilla.org/CA:CertPolicyUpdates#RAs_and_Subordinate_CAs
Thanks,
Kathleen
Operational.
>> 3) Publish a full account of exactly what happened. (See comodo.com
>> links above)
>
> I do not believe that the account is yet "full," as Jake observed:
> https://blog.torproject.org/blog/detecting-certificate-authority-compromises-and-web-browser-collusion#Update
>
Fair enough.
>
> As Andy noted, this point...
>
>> 5) Move, in a short time-frame (small number of months) towards a model
>> where each RA issues from a separate subordinate CA certificate.
>> (started)
>
> ....seems mutually exclusive with this point...
>
The intent is that Comodo operates these intermediate CAs and maintains
full control/auditing of them. The point is that if an RA does make a
mistake, Comodo can revoke that particular intermediate CA.
>> Of particular note is the comment in
>> https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c68
>> "… we have retrofitted our own DV process into the RA's ordering process
>> in the vast majority of cases.
>> By 'our own DV process', I mean that Comodo performs an automated check
>> of domain control by sending (and confirming receipt of) an email to an
>> address which is either on the domain to be validated or is explicitly
>> mentioned in the WHOIS entry for the domain to be validated."
>>
>> We have asked Comodo to explain why this mechanism was not active (or
>> did not work) for this RA.
>
> ....and the latter seems preferable. The combination, in which the
> "parent" does validation and issuance, but uses a different sub-CA for
> each RA seems ideal.
>
I think these practices should be incorporated by any CA who uses
external RAs. I've added points about both of these to
https://wiki.mozilla.org/CA:CertPolicyUpdates#RAs_and_Subordinate_CAs
>> I am extremely concerned about the history of questionable certs being
>> issued within the Comodo hierarchy. One possible result is for Mozilla
>> to turn off all of the trust bits for the Comodo root certificates that
>> are included by default in NSS.
>
> This episode raises concerns not just about Comodo's practices, but
> about Mozilla's. How is it that a well-understood problem from 3 years
> ago (Comodo's unsafe RA practices) was not confirmed to be resolved by
> now? I'll ask again...
>
> http://groups.google.com/group/mozilla.dev.security.policy/browse_thread/thread/381acf2eca0c011f/f57b56dd111ba48b?q=comodo#f57b56dd111ba48b
>
Fair enough. At this point I think we should ask Comodo to do a full
audit of all of their RAs. It appears that their updated practices
haven't been adopted by all of their RAs.
>
> ....will Mozilla mandate public disclosure of all RAs, and the CA's
> precise practices with respect to delegating authority
> (cryptographically or operationally) to them?
>
I don't know about publicly disclosing the list of their RAs.
But I do agree that Comodo should publicly disclose their practices with
respect to delegating authority to their RAs. They should also provide a
description of the changes that they are going to make, have new
training for all of their RAs, and audit all of their RAs.
>> However, I think that the CAs actions in
>> response to the most recent situation should also be taken under
>> consideration. It would have been much easier for the CA to not say
>> anything to browser vendors about their mistake – they had caught it
>> quickly and revoked the certs already. Comodo was very much aware of the
>> previous issues that had been logged in Bugzilla, and of the possible
>> outcome of making this mistake public. Regardless, Comodo chose to do
>> the right thing and disclose this situation to the browser vendors so
>> that they could also take action.
>
> This logic only incentivizes CAs to not perform due diligence under the
> assumption that as long as they admit their mistakes publicly they will
> not be held accountable. You're right, Comodo was very much aware of the
> previous issues that had been logged in Bugzilla... and they even knew
> what they had to do to mitigate their harm, and claimed that they were
> doing so, but evidently did not.
>
It's a shame if CAs view it that way.
I'll wait for Comodo's answers to pass judgment, but I agree that it's
concerning that practices that were said to be put into place don't
appear to have been enforced with all of their RAs
>> We could focus on punishment, but I think it would be more beneficial
>> for us to focus on actions that will prevent such mistakes in the
>> future. As well as sharing the lessons-learned, to help prevent other
>> CAs from making the same mistakes.
>
> How is this time different from last time?
Well, the way it happened is different. Also, Comodo caught it
immediately, and took action immediately -- it didn't take someone else
to notice. So apparently they do have a practice of monitoring their RAs
in place that did work. Some of that monitoring should be automated, and
should happen before the cert gets issued...
Kathleen
How do you plan to ensure that they are doing what they are claiming to
do without requiring disclosure? Disclosure and the resulting
accountability is a fundamental part of Mozilla's cert management
process. *Public* disclosure is essential because Mozilla relies on
*this* public community for due diligence.
> I'll wait for Comodo's answers to pass judgment, but I agree that it's
> concerning that practices that were said to be put into place don't
> appear to have been enforced with all of their RAs
>
>>> We could focus on punishment, but I think it would be more beneficial
>>> for us to focus on actions that will prevent such mistakes in the
>>> future. As well as sharing the lessons-learned, to help prevent other
>>> CAs from making the same mistakes.
>>
>> How is this time different from last time?
>
> Well, the way it happened is different. Also, Comodo caught it
> immediately, and took action immediately -- it didn't take someone else
> to notice. So apparently they do have a practice of monitoring their RAs
> in place that did work. Some of that monitoring should be automated, and
> should happen before the cert gets issued...
How is the accountability for their claims or Mozilla's wishes different
from last time? Mozilla was evidently operating under the assumption
that Comodo was doing something that they were not doing. You do not
propose any more disclosure than last time, so in that respect I don't
see how it is different this time.
Even if they do give us the full list, how are we to know that they've
updated the processes and monitoring for all of their RAs?
I think the only feasible way is to rely on an audit. Perhaps we should
require Comodo to go through a comprehensive independent audit of all of
their RAs to prove that the updated practices and monitoring is in place
across all of their RAs.
>> I'll wait for Comodo's answers to pass judgment, but I agree that it's
>> concerning that practices that were said to be put into place don't
>> appear to have been enforced with all of their RAs
>>
>>>> We could focus on punishment, but I think it would be more beneficial
>>>> for us to focus on actions that will prevent such mistakes in the
>>>> future. As well as sharing the lessons-learned, to help prevent other
>>>> CAs from making the same mistakes.
>>>
>>> How is this time different from last time?
>>
>> Well, the way it happened is different. Also, Comodo caught it
>> immediately, and took action immediately -- it didn't take someone else
>> to notice. So apparently they do have a practice of monitoring their RAs
>> in place that did work. Some of that monitoring should be automated, and
>> should happen before the cert gets issued...
>
> How is the accountability for their claims or Mozilla's wishes different
> from last time? Mozilla was evidently operating under the assumption
> that Comodo was doing something that they were not doing. You do not
> propose any more disclosure than last time, so in that respect I don't
> see how it is different this time.
Yes, I agree. What do you think of the idea of requiring an independent
audit of all of their RAs?
Kathleen
Well, you'd have to require them to disclose the audit... thereby
disclosing the list. Or did you envision an audit where the result was
just some auditor's attestation that all of the RAs were complying?
But what possible legitimate reason, from the perspective of users that
rely on these entities, could there be for not requiring disclosure? At
times, there seems to be a posture on the part of the Moz cert team
against things that are thought to inconvenience the CAs.
I can imagine a variety of ways that public disclosure would help with
policing. It would mean that people could conduct tests against the
full list to verify actual validation. People could report RAs that do
not have disclosed audits. Etc...
>>> I'll wait for Comodo's answers to pass judgment, but I agree that it's
>>> concerning that practices that were said to be put into place don't
>>> appear to have been enforced with all of their RAs
>>>
>>>>> We could focus on punishment, but I think it would be more beneficial
>>>>> for us to focus on actions that will prevent such mistakes in the
>>>>> future. As well as sharing the lessons-learned, to help prevent other
>>>>> CAs from making the same mistakes.
>>>>
>>>> How is this time different from last time?
>>>
>>> Well, the way it happened is different. Also, Comodo caught it
>>> immediately, and took action immediately -- it didn't take someone else
>>> to notice. So apparently they do have a practice of monitoring their RAs
>>> in place that did work. Some of that monitoring should be automated, and
>>> should happen before the cert gets issued...
>>
>> How is the accountability for their claims or Mozilla's wishes different
>> from last time? Mozilla was evidently operating under the assumption
>> that Comodo was doing something that they were not doing. You do not
>> propose any more disclosure than last time, so in that respect I don't
>> see how it is different this time.
>
> Yes, I agree. What do you think of the idea of requiring an independent
> audit of all of their RAs?
Obviously from the above, I think that disclosure is necessary. Audits
are probably also necessary. How often would you propose that they
happen? Must an audit be done of every RA before they begin operations?
However, I'm not sure that applying these policies against "RAs" is the
right approach. Or maybe it is, depending on what the definition of
"RA" is. According to WebTrust (http://www.webtrust.org/item27804.pdf):
"The CA or RA verifies the identity of the subscriber in accordance with
the CA’s established business practices (that may be contained in a
certification practice statement), and then issues a digital certificate."
and
"An RA is an entity that is responsible for the identification and
authentication of subscribers, but does not sign or issue certificates."
So perhaps even WebTrust isn't consistent... does an RA issue certs?
Does it verify identity? I think that any entity that has the
operational and/or cryptographic ability to issue certs (directly, or by
asserting identity to an entity that issues the certs) should be audited
and publicly disclosed. Is that unreasonable?
>>> We could focus on punishment, but I think it would be more beneficial
>>> for us to focus on actions that will prevent such mistakes in the
>>> future. As well as sharing the lessons-learned, to help prevent other
>>> CAs from making the same mistakes.
>>
>> How is this time different from last time?
>
>Well, the way it happened is different. Also, Comodo caught it immediately,
>and took action immediately -- it didn't take someone else to notice.
How do we know that there haven't been dozens of similar compromises that
weren't caught? This seems to be saying "lax practice are OK, we'll pretend
that they're not happening until forced by public disclosure to do something,
and even then it'll be nothing more than a slap on the wrist with a wet bus
ticket". Even before the various public failures, Comodo had a bit of a
reputation in the PKI community for lax practices, to the extent that I'd
heard it referred to privately as "Commode-o" by other PKI people (a name that
I may have accidentally let slip into a public post in the past :-). So there
appears to have been some unease about them for a long time, predating any of
the public disclosures of problems, but it was just ignored.
What we have is an industry that claims to be selling trust, but that has
repeatedly demonstrated its untrustworthiness in a very public manner. And
while other trust-based industries rely on "trust but verify", the browsers
just give us "trust and hope". When the PKI fails, as it has repeatedly,
there's nothing else there to fall back on. If you read the discussion on
various web forums (admittedly not a representative survey, but the best we
have to go on at the moment), most people have pretty much zero confidence in
browser PKI, and very low confidence in browser vendors' seriousness about
protecting users from harm. This is the real problem that browsers should be
looking at, not "how wet shall we make the bus ticket before we slap the CA's
wrist with it" but "how do we deal with the fact that our chosen (and only)
web security mechanism is repeatedly, and publicly, failing, and our only
response to date has been PKI-me-harder".
Peter.
Luckily they detected it before more damage could be done. Unfortunately
disclosure to the public has not been prompt by any party involved and
this should not have happened in this way. The reasons are obvious.
> We have asked Comodo to explain why this mechanism was not active (or
> did not work) for this RA.
I believe that this is a major issue which clearly shows a failure
across the band. If Comodo would have implemented their domain control
validation process as claimed in that bug, this attempt would have been
most likely successfully foiled.
It gets me that this wasn't the case, that this hasn't been made the
declared policy and that this hasn't been audited accordingly! More than
that, if we can't rely on what a CA claims, we have a far bigger issue
than nine fraudulent certs, seriously.
It is very clear to me that no system is 100% secure all the time -
obviously an RA somewhere out there is more likely to fail than the CA
with its resources. It's however not understandable to me that no
sincere best effort has been made since the many issues preceding this one.
It's not understandable that the API for RAs hasn't been requiring the
DV process at the CA side as promised.
It's not understandable that no better measures for authentication
against the API have been implemented.
It's not understandable that Comodo wasn't able to implement a lousy
flagging system for the top 100 targets which would be most likely
requested in such an event.
There are CAs that by choice do NOT work with such RAs - with the
expected result of lesser revenues. There are CAs that have strict
access controls to the CA systems, to those parts that can issue a
certificate - with the expected inconvenience and added costs. There are
CAs that have multiple controls in place in order to detect fraudulent
attempts should a particular lawyer fail in the chain of events and
infrastructure - with the expected result of higher development costs
and operating resources and man-power required.
But what is it worth all that, if there are those that simple don't care
or are incapable to care?!?! And Comodo might be not alone with that.
>
> I am extremely concerned about the history of questionable certs being
> issued within the Comodo hierarchy. One possible result is for Mozilla
> to turn off all of the trust bits for the Comodo root certificates
> that are included by default in NSS.
Even though this should be really the last defense of Mozilla, some
action must happen that clearly makes a point. It will be probably not
an easy decision on your part and most likely you won't make everybody
happy. But something got to happen.
> Regardless, Comodo chose to do the right thing and disclose this
> situation to the browser vendors so that they could also take action.
It might be that those certificates would have surfaced at some point
anyway, so better this.
>
> We could focus on punishment, but I think it would be more beneficial
> for us to focus on actions that will prevent such mistakes in the
> future. As well as sharing the lessons-learned, to help prevent other
> CAs from making the same mistakes.
Nope, this works for some CAs, for others it simply doesn't, period. The
screws must be tightened here and now.
> There are some practices, such as some roots signing end-entity certs
> directly, that this incident has demonstrated should no longer be
> allowed, and need to be corrected soon.
I have been saying this for years, back at the time preceding EV already
(probably arguing with Frank).
The same goes for outsourcing domain control validation.
The same goes for unrestricted externally operated intermediate CAs.
Requiring RAs to be channeled through dedicated intermediate CAs is in
my opinion only partly helpful in case of a compromise - the issuance
processes must be improved, strictly, without mercy and conflicting
business interest.
I agree that measures MUST be taken against recidivist liars, instead
of continuing to trust them.
Comodo clearly lied when they wrote that technical countermeasures had
been installed to prevent such problems.
I personally won't trust such a lier anymore.
On the other hand, I've heard arguments against the trust revocation
for some browsers (but not all), in the sense that such a situation
could cause loss of market share for those browsers. I can understand
this. But this means that the power relationship changes completely as
soon as a CA is included in a browser; before acceptance, a CA needs
its root to be deployed in order to sell certificates; after
acceptance, the browser needs to keep the biggest number of roots to
keep their users happy. That's insane.
Could it be possible to temporarily disable all trust in Comodo
certificates, wait for them to be positively audited on the technical
modifications they might implement to prevent other human failures to
happen so easily (with a public disclosure of the audit report, of
course), and re-enable them after a public discussion?
About the "anti-X.509" buzz, I've also read a lot of such things,
comparing "evil money-based CA (X.509)" and "good PGP-like users", but
honestly (and sadly, yes), they haven't understood what the real
problem was. This has nothing to do with the chosen trust model, it
could have happened with a web-of-trust (and it still does happen,
today).
--
Erwann.
>
> On the other hand, I've heard arguments against the trust revocation for
> some browsers (but not all), in the sense that such a situation could cause
> loss of market share for those browsers.
Maybe we just need to think of it differently from a UI perspective. When you visit a suspected malware site, Firefox, Chrome, etc. popup up a nice nasty warning saying that the site probably has something awfully wrong with it, and you shouldn't visit it.
Maybe in the cases of a site that has a certificate issued by a CA that is being de-listed, or has been de-listed, the browser shouldn't show normal certificate warnings, it should advertise something more about *why* it has made this decision. In this case it is quite straightforward and a lot easier to explain than signature expiry, mismatch of CN/Hostname, etc. It is that we don't trust a given CA that has been de-certified.
BTW, this isn't a statement saying what ought to be done with the current CA under discussion, just suggesting an approach UI-wise that might make browses more nimble.
Just a thought.
- Andy
Yes! All RAs are effectively a CA, because they can issue certs, and
must be subject to the same scrutiny as CAs.
It's not acceptable that we all trust on a bunch of companies, and
Comodo refuses to even say who they are. (Even in an event like this,
they still refuse to say who is the offender.)
I understand their business model relies on hundreds of resellers.
There's no problem with resellers. There's a problem with RAs, third
party entities which perform verification, i.e. a core function of the
CA. That must stop. IMHO, the whole concept of RAs is directly against
our policy, because they are notpart of the audit (as you say), neither
are they are not approved by us (which is what would still be missing).
A policy which says "I can let anyone in the world do the verification
for me" is obviously not very useful, but that's preciselywhat their
policy is.
Again, I have no problem with resellers, as long as Comodo does all the
CA functions, including all verifications, itself.
Ben
Exactly.
No, it's not unreasonable, but the idea of the Mozilla CA policy. It was
just circumvented here. I don't think we should retrospectively
legitimize it.
Ben
I would like to replace this entire section with:
"RAs and sub-CAs must not exist. All verifications and issuance happen
by the CA itself and cannot be delegated or outsourced. All persons and
machinery that is able to issue certs in any way must be subject to and
part of the audit, physically."
In view of the current section, this change may sound extreme, but I
believe it is justified for the reasons above, and anything else would
simply circumvent the whole policy.
Otherwise I can make a Über-CA, write in my policy that I delegate RAs
as I wish, and then every new CA doesn't have to go through our policy
and approval, but just pays me $$.
Exactly that seems to be the business model of Comodo, and I claim it's
in direct contradiction of the idea of our policy.
Ben
Thanks for bringing this up in this way. I believe too that it's
unacceptable that we have CAs of new root inclusion requests jump
through extensive scrutiny, modifications of policies, re-auditing and
so forth - whereas there are some CA roots included that have repeating
failures without any review of the relevant policies, public discussions
and attestations of requested corrective measures.
> Could it be possible to temporarily disable all trust in Comodo
> certificates, wait for them to be positively audited on the technical
> modifications they might implement to prevent other human failures to
> happen so easily (with a public disclosure of the audit report, of
> course), and re-enable them after a public discussion?
If the same equation for existing roots is applied as for new ones, this
should be the correct action at this time. But Mozilla would not be able
to do it on its own without risking a degradation of its own product.
Best regards
Robin Alden
Comodo
> -----Original Message-----
> From: dev-security-policy-
> bounces+robin=comod...@lists.mozilla.org [mailto:dev-
> security-policy-bounces+robin=comod...@lists.mozilla.org] On
> Behalf Of Kathleen Wilson
> Sent: 24 March 2011 21:24
> To: mozilla-dev-s...@lists.mozilla.org
> Subject: Re: Web Browsers and Comodo Announce A Successful
> Certificate Authority Attack, Perhaps From Iran
>
> On 3/23/11 6:07 PM, Eddy Nigg wrote:
> > On 03/23/2011 09:16 PM, From Steve Schultze:
> >> Many of you are probably already following this story.
> >
> > ....and we are waiting patiently for a posting by a Mozilla
> > representative here.
> >
>
> I appreciate your patience...
> (I've been ill, so not online much the past couple of days.)
>
> A good description about what happened has been posted here:
> https://www.eff.org/deeplinks/2011/03/iranian-hackers-obtain-
> fraudulent-https
>
> Comodo has provided their summary of the events here:
> http://blogs.comodo.com/category/it-security/
> http://www.comodo.com/Comodo-Fraud-Incident-2011-03-
> 23.html
>
> We were informed by Comodo promptly, and they followed the
> Security Bugs
> process: http://www.mozilla.org/projects/security/security-bugs-
> policy.html
>
> The Security Bug is here:
> https://bugzilla.mozilla.org/show_bug.cgi?id=642395
>
> The bad certs were all revoked promptly, but blacklisting allows us
> to cope with the possibility that an attacker might block OCSP if
> they control the victim's network. OCSP failures aren't fatal by
> default (though they do downgrade EV to DV) so we wanted to
> make sure these certs never worked.
>
> Mozilla requested that Comodo take the following actions.
>
> 1) Revoke the RA rights of the RA concerned. (Done)
>
> 2) Monitor their OCSP logs for evidence of use of these
> certificates, or evidence that access to their OCSP responders is
> being blocked from certain geographical locations
>
> 3) Publish a full account of exactly what happened. (See
> comodo.com links above)
>
> 4) Provide compelling public evidence that:
> - the root private key was not compromised
> - the channel of mis-issuance is now shut
> - all mis-issued certs have been found and listed
> - the list of other certs to come through that channel recently has
> been carefully audited, and representatives of each organization
> are being/have been contacted to confirm correct issuance.
> (partially complete, Comodo plans to engage WebTrust auditors)
>
> 5) Move, in a short time-frame (small number of months) towards
> a model where each RA issues from a separate subordinate CA
> certificate.
> (started)
>
> We are still evaluating the situation, and may request further
> action from Comodo.
>
> Additionally, this issue has brought several concerns about
> implementation of CRL and OCSP to the forefront, and some new
> bugs have been filed. Also, it turned out that an NSS-based fix for
> this instance was not as simple as we would like. A new bug has
> been created for improving NSS to allow us to add certs to the
> store which are specifically untrusted:
> https://bugzilla.mozilla.org/show_bug.cgi?id=642503
>
> Comodo responded appropriately to this incident by revoking the
> certificates immediately, and notifying the major browser vendors
> so that more proactive actions could be taken.
>
> This incident alone is concerning enough, but as we are all aware,
> there are previous incidents that have been logged in Bugzilla
> regarding certs issued within Comodo’s hierarchy, for example:
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=470897
> https://bugzilla.mozilla.org/show_bug.cgi?id=526560
> https://bugzilla.mozilla.org/show_bug.cgi?id=599856
>
> Of particular note is the comment in
> https://bugzilla.mozilla.org/show_bug.cgi?id=470897#c68
> "… we have retrofitted our own DV process into the RA's ordering
> process in the vast majority of cases.
> By 'our own DV process', I mean that Comodo performs an
> automated check of domain control by sending (and confirming
> receipt of) an email to an address which is either on the domain to
> be validated or is explicitly mentioned in the WHOIS entry for the
> domain to be validated."
>
> We have asked Comodo to explain why this mechanism was not
> active (or did not work) for this RA.
>
> I am extremely concerned about the history of questionable certs
> being issued within the Comodo hierarchy. One possible result is
> for Mozilla to turn off all of the trust bits for the Comodo root
> certificates that are included by default in NSS. However, I think
> that the CAs actions in response to the most recent situation
> should also be taken under consideration. It would have been
> much easier for the CA to not say anything to browser vendors
> about their mistake – they had caught it quickly and revoked the
> certs already. Comodo was very much aware of the previous
> issues that had been logged in Bugzilla, and of the possible
> outcome of making this mistake public. Regardless, Comodo chose
> to do the right thing and disclose this situation to the browser
> vendors so that they could also take action.
>
> We could focus on punishment, but I think it would be more
> beneficial for us to focus on actions that will prevent such mistakes
> in the future. As well as sharing the lessons-learned, to help
> prevent other CAs from making the same mistakes.
>
> I ask the representatives of Comodo to contribute actively to this
> discussion group. In particular, please described the actions
> Comodo is taking now (e.g. the list of requested actions above), as
> well as the actions that Comodo plans to take to prevent such a
> mistake from happening again.
>
> I have added a few items to
> https://wiki.mozilla.org/CA:CertPolicyUpdates#Second_Phase
> Under the heading “RAs and Subordinate CAs”
> There are some practices, such as some roots signing end-entity
> certs directly, that this incident has demonstrated should no longer
> be allowed, and need to be corrected soon.
>
> I am also following the CAB Forum and EFF SSL Observatory
> discussions on this topic. So no need to copy info from them,
> unless there are specific action items that pertain to Mozilla.
>
> Kathleen
>
>
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
However, the Mozilla policy has been updated to require periodic audits.
I assume someone in Mozilla will be looking at the audit reports.
--
David E. Ross
<http://www.rossde.com/>
On occasion, I might filter and ignore all newsgroup messages
posted through GoogleGroups via Google's G2/1.0 user agent
because of spam from that source.
It doesn't appear that this is done comprehensively, or that not
providing an updated audit is grounds for removal. For example, the
spreadsheet shows the most recent CNNIC audit as being August 2009:
http://spreadsheets.google.com/pub?key=ttwCVzDVuWzZYaDosdU6e3w&single=true&gid=0&output=html
https://cert.webtrust.org/SealFile?seal=935&file=pdf
The Moz Cert Policy 2.0 says that "CAs are expected to maintain the
level of service that was established in the Inclusion Section of the
Mozilla CA Certificate Policy."
http://www.mozilla.org/projects/security/certs/policy/MaintenancePolicy.html
However, there doesn't seem to be any mechanism for active policing of
this expectation, other than yearly audit updates. Of course, the only
output of the audit updates is the two-page attestation from an auditor
that a new audit was done... none of the details from the actual report.
And of course, the audit requirement is only one of many requirements
for acceptance.
http://www.mozilla.org/projects/security/certs/policy/InclusionPolicy.html
In addition, Kathleen has said:
"We don't usually go back and retroactively apply root inclusion process
changes to roots. The exception being that we have started to require
regularly updated audit documents for all included root certificate
authorities."
http://groups.google.com/group/mozilla.dev.security.policy/browse_thread/thread/eea04805fbd98045#c9c1c678497fae86
So I believe that the original critique stands -- there is a de-facto
difference between the requirements (and the incentives on CAs) for
acceptance vs maintenance. There is even more difference between
recently approved roots and less recently approved roots.
Steve
That would be fine if we'd have the ability to monitor the accompanying
CA policies (and changes thereof). I believe it would be impossible for
this group here to maintain that - however it will become easier for us
(and Kathleen) to handle once the Basic Guidelines have been adopted and
incorporated into the Mozilla CA Policy requirements.
This will require to look at two place - compliance to the basic
guidelines and attestation by audit. Possible improvements to the
guidelines will be maintained by the CAB Forum, a process that has
already kind of started. I believe this tool will become very important
for all of us.
So in summary, "we can't actually enforce the expectations we just wrote
into the new guidelines."
It's also not just a matter of CA policies... it's a matter of
practices. However, without disclosure of things like SubCA's and RA's,
it's impossible for anyone to even try in a comprehensive way.
> This will require to look at two place - compliance to the basic
> guidelines and attestation by audit. Possible improvements to the
> guidelines will be maintained by the CAB Forum, a process that has
> already kind of started. I believe this tool will become very important
> for all of us.
The CAB Forum doesn't own the requirements on CA's. The browsers and OS
vendors do. If they wish to adopt the CAB Forum guidelines, they can.
If the CAB Forum would like to begin conducting its business in a more
public way, this outcome might be more likely.
The legitimate certs of two of the targeted webapps are apparently EV certs:
<https://login.live.com/> and <https://addons.mozilla.org/>.
The generated impostor certs were not EV -- but /could/ they have been? I.e. is
Comodo (and other CAs) facilitating their RAs to issue EV certs? Could the
compromised RA user account have actually issued an EV certs for login.live.com
and addons.mozilla.org ?
If Comodo is facilitating EV issuance by any of their RAs, are they following
(where their RAs are concerned) the procedures in Sections 12.2 (and section 12
in general) and section 10.12 of <http://cabforum.org/Guidelines_v1_3.pdf> ?
Perhaps these are questions to add to the list for them to answer.
thanks,
=JeffH
Of course this auditors have to confirm - failing that, they fail too.
> However, without disclosure of things like SubCA's and RA's, it's
> impossible for anyone to even try in a comprehensive way.
One doesn't exclude the other obviously. Luckily we are free to add our
own requirements :-)
> The CAB Forum doesn't own the requirements on CA's.
No, it's a group that can (and does) propose the guidelines.
> The browsers and OS vendors do. If they wish to adopt the CAB Forum
> guidelines, they can.
Obviously. And of course I'd urge exactly that to do once it becomes
relevant.
Actually, CNNIC did provide the updated audit in Bug #607208. I just
hadn't added that update to the spreadsheet yet. It's there now.
> The Moz Cert Policy 2.0 says that "CAs are expected to maintain the
> level of service that was established in the Inclusion Section of the
> Mozilla CA Certificate Policy."
>
> http://www.mozilla.org/projects/security/certs/policy/MaintenancePolicy.html
>
>
> However, there doesn't seem to be any mechanism for active policing of
> this expectation, other than yearly audit updates. Of course, the only
> output of the audit updates is the two-page attestation from an auditor
> that a new audit was done... none of the details from the actual report.
> And of course, the audit requirement is only one of many requirements
> for acceptance.
> http://www.mozilla.org/projects/security/certs/policy/InclusionPolicy.html
>
Yes, I rely on the audits. About every 6 months I go through my
spreadsheet to find audits that are out of date, and contact the
relevant CAs.
The nice thing (for me) is that since we updated the Mozilla CA Cert
Policy, CAs have started to proactively send me their updated audits
before I ask.
> In addition, Kathleen has said:
> "We don't usually go back and retroactively apply root inclusion process
> changes to roots. The exception being that we have started to require
> regularly updated audit documents for all included root certificate
> authorities."
>
> http://groups.google.com/group/mozilla.dev.security.policy/browse_thread/thread/eea04805fbd98045#c9c1c678497fae86
>
>
> So I believe that the original critique stands -- there is a de-facto
> difference between the requirements (and the incentives on CAs) for
> acceptance vs maintenance. There is even more difference between
> recently approved roots and less recently approved roots.
>
However, we do expect all CAs to comply with Version 2.0 of the Mozilla
CA Cert Policy by August 8, 2011. This was communicated to CAs via
email, and I have been keeping an internal spreadsheet of the responses
and progress.
https://wiki.mozilla.org/CA:CertPolicyUpdates#Transitioning_to_the_Updated_Policy
Likewise, we will give all CAs a certain amount of time to comply with
the next update of the Mozilla CA Cert Policy.
Compliance means that the CA policy documentation has been updated as
needed, and the new practices put into place. Then the next regular
audit should included these updates, since they are part of the policy
documentation.
Kathleen
The auditors do not confirm everything in the Mozilla requirements...
including some of the required practices. That is the point.
A column in the spreadsheet listing the date of the last audits might
make it easier for you and for outsiders to do this.
At what point does an overdue audit update trigger removal?
> Compliance means that the CA policy documentation has been updated as
> needed, and the new practices put into place. Then the next regular
> audit should included these updates, since they are part of the policy
> documentation.
Ah, I see how this works with InclusionPolicy #14.
It seems that a great deal rises and falls on the auditors. It seems
like anything that can be done to allow the public to more easily verify
compliance would be good.
Maybe you got me wrong - it's the basic guidelines to which the CA will
have to commit in the policy. Whatever is beyond is obviously not
covered, but that already should be a significant improvement.
This is a new policy. Thus, it has not been required in the past.
>From the wording of the policy -- "This section of the Mozilla CA
Certificate Policy describes the obligations of Certification
Authorities for maintaining confidence in their root certificates that
are included in Mozilla Products." and "This is the official Mozilla
policy for Certification Authorities to maintain their CA Certificates
that are distributed in Mozilla products" -- I would assume that a
failure to provide an annual auditor's attestation would result in the
removal of a root certificate.
Kathleen: Would a root certificate indeed be removed for any
certificate authority for which Mozilla does not receive an annual
auditor's attestation? Or would this merely be grounds for removal? I
hope it is the former. We have seen repeated grounds for removing
Comodo roots, yet none have been actually removed.
Some more possible information here:
Just got it from https://bugzilla.mozilla.org/show_bug.cgi?id=642395
>...so this person, if I understand it correctly, is claiming that Comodo is
>using a username/password authentication system for its RAs.
Which I find at least as amusing as the fact that, every year, the PKI experts
who attend the PKI workshop authenticate themselves with username and password
(and the fact that browser vendors dealt with the Comodo fiasco using updated
binaries, not CRLs and OCSP, and ...). Does anyone involved in PKI actually
believe in it sufficiently to secure their infrastructure with it?
Peter.
I would like to continue on the question from Peter Gutmann, *"How do we know that there haven't
been dozens of similar compromises that weren't caught?"*
The last years there have been various incident's with Comodo that all came back on the fact that
there partners are able to tick the box. (*“I have sufficiently validated this application”*)
A Comodo representative gave me a few key points last month to "convince" me that we now could trust
there system. They would have learned from there mistakes :-).
COMODO: *"All new partners from a few years ago cannot validate certificate orders."*
... what about the 'old' partners?
COMODO: *"Mandatory that for the first 5-10 orders they send in documentation (Training)"*
... wow that is a huge training! (so you can issue a false certificate after I issued the first 10)
COMODO: *"Upon approval and only a case by case basis do e approve a partner to be an RA."*
... but it's still offered to me :-)
COMODO: *"Even as an RA we still enforce that All partners use DCV on all orders. (OV, DV, EV), See
attached Domain Control Integration guide."*
... yes this would give me a lot of trust in the quality of there OV and EV validations!
And finally some information from there Web Hosts Partner Network program, that is enough
information for me that this CA has no right to issue trust at all!
"
*Appendix 1: Validation Guidelines for Web Hosts*
In order to ensure the integrity of the trust infrastructure provided by Comodo through InstantSSL
it is essential that members of the Web Host Programme operate under the following guidelines when
issuing Certificates.
*You must address two core areas prior to issuing a Certificate to a customer:*
1. The customer has the right to use the domain name to be included in the Certificate
2. The customer exists as a responsible legal entity or individual
In most cases you will have a pre-established relationship with the applicant. This may include
existing business relations established through the sale of products / services such as web hosting
services or domain name registration. In this event, the right of a customer to the domain name
contained within the application will have been established you will be hosting the domain name on
behalf of the customer. It is therefore only necessary for to validate the existence of the customer
as a responsible commercial entity or individual.
If you have an existing business relationship with customer, this requirement may already have been
fulfilled in the guise of payments / invoices paid by the end entity. In the event that an existing
business relationship does not exist, or you are not fully confident that the customer is
sufficiently known to you, it is your responsibility to sufficiently determine their organisational
/ individual status. To complete this validation, we recommend you request any of the following
documentation from the customer:
*If the Certificate is for a commercial entity:*
• Articles of Incorporation
• Business License
• DUNS details (e.g. your Dun & Bradstreet company number)
• Trading License
*If the Certificate is for a non-commercial entity:*
• Copy of your drivers license or passport
Upon receipt of the relevant documentation, you should verify the information received correctly
matches the information provided during application.
It is deemed that the above validation will have taken place when you check the *“I have
sufficiently validated this application”* checkbox when applying for a Certificate.
"
Kind regards,
Paul van Brouwershaven
Networking4all B.V.
____________________________________________
Phone: (31) 20 7881042 Fax: (31) 20 7881040
Email: p.vanbrou...@networking4all.com
Internet: https://www.networking4all.com
LinkedIn: https://www.linkedin.com/in/pvanbrouwershaven
Facebook: https://www.facebook.com/p.vanbrouwershaven
Twitter: https://www.twitter.com/vanbroup
>Some more possible information here:
He's now also posted an update containing the complete decompiled Comodo RA
DLL, you can find it at http://pastebin.com/DBDqm6Km. Incidentally, here's a
message I posted to the cypherpunks list a few days ago:
-- Snip --
Subject: Iranian state-sponsored cyberwarfare is indistinguishable from script kiddies
Pretty much every news report I've seen so far that mentions any kind of
Iranian connection is claiming that it's Iranian state-sponsored hacking. If
what's happened with the certs so far (someone grabbed a few sample certs for
high-profile domains, and there was a report of one of them briefly appearing
on a test server in Iran) is an indication of their competence then we really
have nothing to fear from them.
Let's look at what would have happened if *I'd* figured out a way to
compromise a CA. First, I'd get a few test certs issued for high-profile
domains, Microsoft, Google, Yahoo, and perhaps a CA cert just for giggles.
Then I'd set up a server somewhere and install one of the sample certs to see
whether any web browser noticed a problem.
Gosh, this sounds awfully like what actually happened. New Zealand must have
a state-sponsored cyberwar program! The only difference in my case is that
after a day or so of inviting security people to have a giggle at the test
server with my "genuine" cert, I'd notify the CA about the problem. If I was
an Iranian script kiddie I probably wouldn't have much motivation to do that.
So what we have here is either (a) the world's most incompetent state-
sponsored cyberwar program, who get the keys to the kingdom and then have no
idea what to do with them, or (b) a bunch of script kiddies having fun.
What do you reckon the odds are?
(And in all this I haven't seen any mention of the Al Kai-yee-da angle. What
happened, is everyone asleep?).
-- Snip --
(OK, I get to do the I-toldja-so dance now).
Peter.
Translation: "I have a reseller account with Comodo, I can get certs issued
for any domain as long as I use their API and not the web GUI".
I've frequently referred to CAs as "certificate vending machines" to get
across to people how they work, but it looks like Comodo really *is* just a
certificate vending machine.
Peter.
Mozilla has now published more details about what happened:
http://blog.mozilla.com/security/2011/03/25/comodo-certificate-issue-follow-up/
I am ready to answer any other questions I can about the incident.
> ....will Mozilla mandate public disclosure of all RAs, and the CA's
> precise practices with respect to delegating authority
> (cryptographically or operationally) to them?
I was about to add this to
https://wiki.mozilla.org/CA:Comodo_Misissuance_Response , but it seems
you beat me to it :-)
Gerv
Noted.
Gerv
"The perpetrator can only make use of these certificates if it had control of the DNS infrastructure."
It's ideal for government. :) They control the infrastructure, and now they have the certficates too. :)
I think the CA should be dropped.
Üdvözlettel/Regards,
Varga Viktor
Üzemeltetési és Vevőszolgálati Vezető
IT Service and Customer Service Executive
Netlock Kft.
_______________________________________________________________________
Ezt az e-mailt virus- es SPAM-szuresnek vetettuk ala a filter:mail MessageLabs rendszerrel. Tovabbi informacio: http://www.filtermax.hu
This email has been scanned for viruses and SPAM by the filter:mail MessageLabs System. More information: http://www.filtermax.hu ________________________________________________________________________________________
A few days and a few thoughts later, I thought it would be beneficial if
I'd share what my expectations would be in resolving this issue. In case
Mozilla doesn't already plans to remove or disable the Comodo roots,
then the following minimum set of actions must be taken in order to
ensure some reasonable reliance and trust into the certificates issued
by Comodo from now on.
Hopefully this gives some ideas and some of the basics in order to
continue with a resolution. I'd propose the following requirements and
commitments:
Comodo disables immediately all RAs and Sub CAs (the later share the
same problem as RAs according to the information I have).
Comodo implements a domain control validation procedures into their API
and web based control panels for RAs with the following sequences:
* A query option for RAs to obtain a list of authoritative email
addresses according to the Mozilla CA Policy.
* A request for domain control validation by the RA by submitting
which domain and email address from the above list shall be used
for domain control. Comodo must check if the email address is
permitted for this purpose. If correct, Comodo sends a
verification code to that email address (the email may be branded
to reflect the RA).
* A post response of the verification code by the RA to Comodo which
includes the domain, the email address used and the verification
code. This response should be within a time frame sufficient to
receive an email and reply the code (this may be as low as 15
minutes, but should not exceed more than a couple of hours the most).
* If the response was successful, the domain or domain space may be
used for a certificate for the specific applicant.
* Comodo implements a flagging system for the top target domains and
high profile brands that might be targeted including well known
banks, so-called social sites etc.
* Comodo implements better authentication mechanisms for their API
and web based control panels of their RAs, probably based on
client certificates or similar solutions.
Further Comodo modifies its policies to reflect the above changes and
commitment for validating all domains through their own processes.
Comodo engages with its auditor and performs an extra-ordinary audit
confirming above implementations.
After publishing of the audit report and acceptance by Mozilla, Comodo
may enable the API for RAs and Sub CAs. I expect the action items to be
resolved within 30 - 90 days. During this time Comodo will accept and
process certificate requests only through its own web sites and systems.
Mozilla confirms at the policy newsgroup that it received the audit
report and accepted its wording and confirms the changes to the policy.
No further discussion would be necessary, provided that this would be
the accepted resolution.
Some more: http://pastebin.com/X8znzPWH