ssl.com: Certificate with Debian weak key

[Matt Palmer]

(Pre) Certificate https://crt.sh/?id=2531502044 has been issued with a known
weak key, specifically Debian weak key 2048/i386/rnd/pid17691. I believe
this issuance to be in contravention of SSL.com's CPS, version 1.8, section
6.1.1.2, which states "SSL.com shall reject a certificate request if the
request has a known weak Private Key".

- Matt

[Ryan Sleevi]

Thanks. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1620772

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>

[Matt Palmer]

On Sat, Mar 07, 2020 at 09:07:11AM -0500, Ryan Sleevi wrote:
> Thanks. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1620772

I'll give points to SSL.com for a speedy initial response, but I'm a bit
disconcerted about this:

> The fingerpint of the claimed Debian weak key was not included in our database.

I think it's worth determining exactly where SSL.com obtained their
fingerprint database of weak keys. The private key in my possession, which
I generated for inclusion in the pwnedkeys.com database, was obtained by
using the script provided in the `openssl-blacklist` source package, with no
special options or modifications.

The key used in this certificate is not one for a niche architecture or
unusual configuration -- i386 was the dominant architecture at the time of
the flaw, and 2048 bits is a standard key size. It would be somewhat more
understandable if the key was, say, a 4096 bit key generated on a MIPS
machine (it took *aaaaaaaaaaaages* to generate all of those), although it
would still be a Debian weak key and thus still be a BR violation to issue a
certificate for it.

On the off-chance that there *was* a mistake in my key generation procedure,
*and* a one-in-a-trillion collision of private keys, I've confirmed that the
public key of the certificate in question is in the `openssl-blacklist`
Debian package (https://packages.debian.org/jessie/openssl-blacklist,
uploaded in 2011) with the following command:

grep $(wget -O - -q https://crt.sh/?d=2531502044 \
| openssl x509 -noout -pubkey \
| openssl rsa -pubin -noout -modulus \
| sha1sum | cut -d ' ' -f 1 | cut -c 21-) \
/usr/share/openssl-blacklist/blacklist.RSA-2048

As further independent confirmation, the crt.sh page for the certificate
shows that crt.sh *also* identifies the certificate as having a Debian weak
key. My understanding is that crt.sh uses a database of keys that was
independently generated by the operator of the crt.sh service.

- Matt

[Rob Stradling]

On 07/03/2020 23:57, Matt Palmer via dev-security-policy wrote:
<snip>

> As further independent confirmation, the crt.sh page for the certificate
> shows that crt.sh *also* identifies the certificate as having a Debian weak
> key. My understanding is that crt.sh uses a database of keys that was
> independently generated by the operator of the crt.sh service.

For the crt.sh check, I augmented Debian's original blacklists with some
other blacklists I generated ~12yrs ago for a few less common key sizes
[1]. See also [2].


[1] https://secure.sectigo.com/debian_weak_keys/

[2]
https://www.mail-archive.com/dev-secur...@lists.mozilla.org/msg10060.html

> - Matt

--
Rob Stradling
Senior Research & Development Scientist
Email: r...@sectigo.com

[Nick Lamb]

On Sun, 8 Mar 2020 10:57:49 +1100

Matt Palmer via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:

> > The fingerpint of the claimed Debian weak key was not included in
> > our database.
>
> I think it's worth determining exactly where SSL.com obtained their
> fingerprint database of weak keys. The private key in my possession,
> which I generated for inclusion in the pwnedkeys.com database, was
> obtained by using the script provided in the `openssl-blacklist`
> source package, with no special options or modifications.

Yes, I would certainly want SSL.com's report to give me confidence
that

#1 they've identified why they didn't spot this key, were there (many?)
other keys which would also have been missed?

#2 they now have a complete and accurate list of such keys

#3 they went back and did the work to re-check other certificates
they've issued for this (these?) extra weak keys and any matches were
revoked and the subscriber contacted


Depending on the circumstances in #1 there may well be a lesson for
other CAs, especially if using a setup which is similar in some way to
SSL.com and so this point is very important. There might also be
further questions about SSL.com's processes which failed to detect this
mistake.

This sort of incident is also important because of the impact on the
Subscriber. Had this subscriber used a different CA with a complete
list they'd have been informed immediately that their chosen key was a
problem. Because SSL.com didn't do that in fact this subscriber was
potentially vulnerable to active, and in some cases even passive
attacks on their TLS services for the period between issuance and
discovery.


Nick.

[Chris Kemmerer]

We have updated https://bugzilla.mozilla.org/show_bug.cgi?id=1620772 with the findings of our current investigation.

We believe all issues raised in this thread are addressed in this update. Our investigation is ongoing and we welcome any positive input by the community as an opportunity to improve our practices and communal PKI operations generally. Our team is evaluating possible improvements for this process, such as acquiring actual public keys created by the weak Debian RNG and using those keys to produce fingerprints compatible with our CA software. We have also contacted our CA software vendor regarding an improved blacklist.

csk
Chris Kemmerer
SSL.com

[Matt Palmer]

On Tue, Mar 10, 2020 at 01:48:49PM -0700, Chris Kemmerer via dev-security-policy wrote:
> We have updated https://bugzilla.mozilla.org/show_bug.cgi?id=1620772 with
> the findings of our current investigation.

Thanks for this update. I have... comments.

Before I get into the nitty-gritty, though, I'd just like to say that I
found your response to be unnecessarily defensive. So much of it reads --
to me, at least -- as "please don't blame us, it wasn't our fault!". Fault
isn't really the issue at hand. Discovering what went wrong, and making
sure that the issues are fixed, both for SSL.com and, if appropriate, other
CAs, is what I, at least, am trying to achieve.

Therefore, while a lot of my responses below address specific points that
you've made in your defence, please understand that I'm not trying to rebut
them in an attempt to say "nee nah nee nah, it *is* your fault!", but rather
to try and provide further information that SSL.com and other CAs could
benefit from considering for improvement in the future, given my experience
dealing with Debian weak keys.

> For the purpose of identifying whether a Private Key is weak, SSL.com uses
> a set of Debian weak keys that was provided by our CA software vendor as
> the basis for our blacklist.

I think it's worth getting additional, *very* detailed, information from
your CA software vendor as to where *they* got their Debian weak key list
from. That appears to be the fundamental breakdown here -- you relied on a
third-party to give you good service, and they didn't. So I think that
digging into your vendor's practices is an important line of enquiry to go
down.

Also, given that there is a non-zero chance that other CAs trusted by
Mozilla may be using the same software, with the same incomplete list of
weak keys, I think it's important that the fact that the vendor is using a
demonstrably incomplete list be circulated to the other customers of this
vendor. How would you suggest this is best accomplished?

> This information was disclosed on 2020-03-07 to the person that submitted
> the Certificate Problem Report.

I don't see anything from SSL.com that looks relevant in my inbox, but it's
not an important point, just thought I'd mention it in passing.

> The above practices comply with our CP/CPS, version 1.8, section 6.1.1.2,

> which states:
>
> "SSL.com shall reject a certificate request if the request has a known
> weak Private Key".

Hmm, "known" is doing a lot of work in that sentence. Known to *whom* is
the important, but unanswered, question. It may be a question which should
be answered explicitly in SSL.com's CPS, as well as the CPS of all other CAs
(and even, potentially, in the BRs -- although my understanding is that that
little bit of the BRs may at some point in the near future get a tidy-up,
per https://github.com/cabforum/documents/issues/164).

If the appropriate answer is "known to SSL.com", then you could run your CA
software with an empty blacklist, issue certs for all manner of known-weak
keys, and still be compliant with your CPS. As such, that's probably not an
acceptable answer to Mozilla. "Known to SSL.com's CA vendor" is similarly
problematic.

If, on the other hand, the appropriate interpretation is "known to anyone",
then because the key was known to me, and I think I count as "anyone", your
CPS was not followed.

"Known to the vendor of the software which generated the known-bad key" is
also an answer that results in your violating your CPS and the BRs, because
the key you issued a certificate for was, as previously mentioned, included
in the `openssl-blacklist` Debian package.

Do you have another answer to the question "known to whom?" which SSL.com is
using in that sentence?

> Our understanding is that there is no single or complete list with all
> known Debian weak keys, either one that is normative for use by the CAs
> included in the Mozilla Root Program, nor one specified in the Baseline
> Requirements.

That is, I believe, correct, however (at the risk of tooting my own horn)
there is quite a comprehensive collection of Debian weak keys in the
pwnedkeys.com database. You are welcome to encourage your CA software
vendor to perform lookups against the public API if you wish. I don't claim
that any possible key you could generate from a buggy Debian system is
already in pwnedkeys, but I've accumulated a fair collection of likely
candidates, at great cost of (mostly emulated) CPU cycles.

> This can be demonstrated by submitting the following CSR

That CSR uses a 2048 bit key generated on an i386 system, using OpenSSH with
an exponent of 3, with PID 23747. It might not be in certwatch, but it's in
pwnedkeys. :grin:

> We have strong indications that there are several different lists of
> precalculated vulnerable keys whose precise populations depend on
> combinations of:
>
> architecture

Yes, different CPU word sizes and endianness produce different keys.
That is covered in https://wiki.debian.org/SSLkeys#How_weak.3F.

> key size

That different key sizes would produce different keys should not be
surprising.

> process ID

Yes, that is the fundamental nature of the bug, that the random seed is
taken entirely from the ID of the process that generates the key.

> (under certain conditions) application (openssh, openvpn, openssl)

Yes, insofar as OpenSSL and OpenSSH use slightly different key generation
parameters (for example, older versions of OpenSSH used e=3, which produce a
different set of public keys than when e=65537 or otherwise). OpenVPN
itself only generates static (pre-shared) keys, not RSA keys, so is not
relevant for TLS.

> openssl environment (for example, the use of .rnd file in the user's home
> directory and whether openssl can read this file).

Yes, whether `~/.rnd` exists and/or is readable is one of the parameters
that is varied during generation of a complete set of weak private keys
using OpenSSL, for a given architecture/keysize pair. This is probably the
hardest aspect of the weak key generation process to know about, because it
does require examination of the method by which `openssl-blacklist` was
generated, but if your CA software vendor only included keys generated by
OpenSSH, it's somewhat of a moot point.

> We were not aware that the application and variables produce different keys.

If you're relying on your CA software vendor for your key blacklist, then I
wouldn't say that you would *need* to be particularly aware of the
permutations for generating a list of known-weak keys. You merely have to
have a CA software vendor that *does* have that awareness.

> The lists provided by our CA software vendor and
> https://github.com/g0tmi1k/debian-ssh were probably produced using
> openssh.

Quite possibly; once again, you really should ask your CA software vendor
for full and complete details of how the key blacklist they gave you came to
be. If they only used a key list that was generated by OpenSSH, I'd be
concerned on your behalf, because by far the more common source of keys used
in SSL certificates would be generated by OpenSSL, since that is, I would
expect, what most people use to generate CSRs.

> While openssh uses libcrypto/ssl for the key generation, it seems to
> generate different keys than openssl itself.

Yes, it does, by using different parameters (such as e=3 rather than
e=65537) in older versions of OpenSSH.

> In our understanding, the events detailed in this bug do not constitute a
> compliance violation that rises to the level of an incident.

My understanding is that *all* compliance violations are an incident, in
Mozilla's eyes. However if my understanding is incorrect, I'm sure a module
peer or owner will be willing to correct me.

- Matt

[Chris Kemmerer]

On Tuesday, March 10, 2020 at 8:44:49 PM UTC-5, Matt Palmer wrote:
> On Tue, Mar 10, 2020 at 01:48:49PM -0700, Chris Kemmerer via dev-security-policy wrote:
> > We have updated https://bugzilla.mozilla.org/show_bug.cgi?id=1620772 with
> > the findings of our current investigation.
>
> Thanks for this update. I have... comments.
>
> Before I get into the nitty-gritty, though, I'd just like to say that I
> found your response to be unnecessarily defensive. So much of it reads --
> to me, at least -- as "please don't blame us, it wasn't our fault!". Fault
> isn't really the issue at hand. Discovering what went wrong, and making
> sure that the issues are fixed, both for SSL.com and, if appropriate, other
> CAs, is what I, at least, am trying to achieve.
>
> Therefore, while a lot of my responses below address specific points that
> you've made in your defence, please understand that I'm not trying to rebut
> them in an attempt to say "nee nah nee nah, it *is* your fault!", but rather
> to try and provide further information that SSL.com and other CAs could
> benefit from considering for improvement in the future, given my experience
> dealing with Debian weak keys.

We share the same view. Our analysis so far confirms that this is a tricky issue, and it gets trickier if one considers compatibility issues between CA software and lists of weak keys which are publicly available.
This is exactly why it would be a useful contribution to the ecosystem, if anyone in possession of such weak key databases publishes them along with the keys.
We have included more details for this below.

> > For the purpose of identifying whether a Private Key is weak, SSL.com uses
> > a set of Debian weak keys that was provided by our CA software vendor as
> > the basis for our blacklist.
>
> I think it's worth getting additional, *very* detailed, information from
> your CA software vendor as to where *they* got their Debian weak key list
> from. That appears to be the fundamental breakdown here -- you relied on a
> third-party to give you good service, and they didn't. So I think that
> digging into your vendor's practices is an important line of enquiry to go
> down.

As mentioned on our report, we used that list as a basis, and paid attention to augment it with other weak keys from available blacklists, even for the ROCA vulnerability.
So, instead of just relying on the CA software vendor, we definitely did - and do, on a regular basis - our own homework.

> Also, given that there is a non-zero chance that other CAs trusted by
> Mozilla may be using the same software, with the same incomplete list of
> weak keys, I think it's important that the fact that the vendor is using a
> demonstrably incomplete list be circulated to the other customers of this
> vendor. How would you suggest this is best accomplished?

Our CA software vendor is monitoring this list so they should be able to address this issue independently.
For what it's worth, we believe that the current language in the BRs could be less ambiguous as far as the Debian weak keys are concerned. For example, it seems that the community's expectations are for CAs to detect and block weak Debian keys generated by vulnerable RNG using OpenSSL in popular architectures. That could be added in the BRs:

Change:
"The CA SHALL reject a certificate request if the requested Public Key does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if it has a known weak Private Key (such as a Debian weak key, see http://wiki.debian.org/SSLkeys)"

to something like:
"The CA SHALL reject a certificate request if the requested Public Key does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if it has a known weak Private Key using, as a minimum set the Debian weak keys produced by OpenSSL in i386 and x64 architectures (see http://wiki.debian.org/SSLkeys)"

Then, it would be clear that all CAs would need to block "at least" the vulnerable keys produced by OpenSSL and could add other keys produced by OpenSSH or other applications if they wanted a "more complete" list.

> > This information was disclosed on 2020-03-07 to the person that submitted
> > the Certificate Problem Report.
>
> I don't see anything from SSL.com that looks relevant in my inbox, but it's
> not an important point, just thought I'd mention it in passing.

For the record, we replied to mpa...@hezmatt.org at 2020-03-07 8:41 PM (UTC)
Email subject: "Problem Report for certificate(s) with compromised private key"
Can you please confirm?
We did disclose the source of our weak keys in that email.

> > The above practices comply with our CP/CPS, version 1.8, section 6.1.1.2,
> > which states:
> >
> > "SSL.com shall reject a certificate request if the request has a known
> > weak Private Key".
>
> Hmm, "known" is doing a lot of work in that sentence. Known to *whom* is
> the important, but unanswered, question. It may be a question which should
> be answered explicitly in SSL.com's CPS, as well as the CPS of all other CAs
> (and even, potentially, in the BRs -- although my understanding is that that
> little bit of the BRs may at some point in the near future get a tidy-up,
> per https://github.com/cabforum/documents/issues/164).
>
> If the appropriate answer is "known to SSL.com", then you could run your CA
> software with an empty blacklist, issue certs for all manner of known-weak
> keys, and still be compliant with your CPS. As such, that's probably not an
> acceptable answer to Mozilla. "Known to SSL.com's CA vendor" is similarly
> problematic.
>
> If, on the other hand, the appropriate interpretation is "known to anyone",
> then because the key was known to me, and I think I count as "anyone", your
> CPS was not followed.
>
> "Known to the vendor of the software which generated the known-bad key" is
> also an answer that results in your violating your CPS and the BRs, because
> the key you issued a certificate for was, as previously mentioned, included
> in the `openssl-blacklist` Debian package.
>
> Do you have another answer to the question "known to whom?" which SSL.com is
> using in that sentence?

This language was mainly taken from the Baseline Requirements.
We will update this section in our next CP/CPS version to make it clearer, but we agree that Baseline Requirements should be updated to express the exact requirement more clearly.
We elaborate further on that below.

> > Our understanding is that there is no single or complete list with all
> > known Debian weak keys, either one that is normative for use by the CAs
> > included in the Mozilla Root Program, nor one specified in the Baseline
> > Requirements.
>
> That is, I believe, correct, however (at the risk of tooting my own horn)
> there is quite a comprehensive collection of Debian weak keys in the
> pwnedkeys.com database. You are welcome to encourage your CA software
> vendor to perform lookups against the public API if you wish. I don't claim
> that any possible key you could generate from a buggy Debian system is
> already in pwnedkeys, but I've accumulated a fair collection of likely
> candidates, at great cost of (mostly emulated) CPU cycles.

Unfortunately the format of the blacklists in openssl-blacklist is incompatible with our CA software which expects the SHA256 hash of the modulus, that's why we mention about acquiring the actual public keys. While we could use the "openssl-blacklist" package to conduct a short-term scan, it would be inefficient compared to the final solution.
We already scanned our entire certificate database to detect weak Debian keys included in "openssl-blacklist" and found only 1 entry (the certificate that was reported and is now revoked).
In our understanding, many individuals are in possession of such lists and it would certainly expedite things, along with reducing waste from duplicate effort, if you or others with similar lists, decided to share the actual keys with the broader community.
The same applies for private keys that have been publicly disclosed. Maintaining such a list and justifying each entry is a great contribution to the community. Even getting the modulus of those keys would solve many compatibility issues with various "key detection" solutions.

Regarding the use of external third-party systems, such as the pwnedkeys.com database, we are of the opinion that no CA's certificate issuance process and/or their compliance should rely on an external system which has not been evaluated and without any SLA guarantees.

This is useful. It confirms some of our investigation findings. We first wanted to understand the underlying technology before developing an improved Debian weak key detection mechanism.

> > We were not aware that the application and variables produce different keys.
>
> If you're relying on your CA software vendor for your key blacklist, then I
> wouldn't say that you would *need* to be particularly aware of the
> permutations for generating a list of known-weak keys. You merely have to
> have a CA software vendor that *does* have that awareness.
>
> > The lists provided by our CA software vendor and
> > https://github.com/g0tmi1k/debian-ssh were probably produced using
> > openssh.
>
> Quite possibly; once again, you really should ask your CA software vendor
> for full and complete details of how the key blacklist they gave you came to
> be. If they only used a key list that was generated by OpenSSH, I'd be
> concerned on your behalf, because by far the more common source of keys used
> in SSL certificates would be generated by OpenSSL, since that is, I would
> expect, what most people use to generate CSRs.

This was not required in the Baseline Requirements. We agree that this SHOULD be the common source of keys for TLS Certificates and we will work to include this list in our weak key detection engine.

> > While openssh uses libcrypto/ssl for the key generation, it seems to
> > generate different keys than openssl itself.
>
> Yes, it does, by using different parameters (such as e=3 rather than
> e=65537) in older versions of OpenSSH.
>
> > In our understanding, the events detailed in this bug do not constitute a
> > compliance violation that rises to the level of an incident.
>
> My understanding is that *all* compliance violations are an incident, in
> Mozilla's eyes. However if my understanding is incorrect, I'm sure a module
> peer or owner will be willing to correct me.

You are correct, each compliance violation is considered an incident. However in our opinion we have not violated our CP/CPS or the current Baseline Requirements. Although this is a complex issue with no definite consensus on which authoritative list to use (only suggestions), we do have a weak keys detection mechanism in place, it does detect Debian weak keys (although it's not perfect) and it also detects ROCA vulnerable keys.

Of course, we continue to use this discussion as an opportunity for improvement, for our own operations and for the community at large.


csk

> - Matt

[Ryan Sleevi]

On Wed, Mar 11, 2020 at 1:46 PM Chris Kemmerer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> You are correct, each compliance violation is considered an incident.
> However in our opinion we have not violated our CP/CPS or the current
> Baseline Requirements. Although this is a complex issue with no definite
> consensus on which authoritative list to use (only suggestions), we do have
> a weak keys detection mechanism in place, it does detect Debian weak keys
> (although it's not perfect) and it also detects ROCA vulnerable keys.

I've commented on the bug as much, but I find this response deeply
disappointing and disconcerting.

This CA ignored a widely known, explicitly circulated list of
known-compromised keys, and is now doubling down that there's nothing wrong
with this. The justification is "This key was not known to be compromised
/by us/", with their rationale of "The BRs explicitly tell us where we
could find a list of known weak/compromised keys, but doesn't say we have
to look at it, and so our elective ignorance is a virtue, not a vice".

Whatever your view of the correctness [1] of this argument, as a systemic
response from a CA, the entrenchedness here suggests that unless the CA can
be hand-held into being trustworthy, they will do the minimum possible
thing.

I appreciate the suggestions for improvement, and that's at least slightly
positive, but if the answer is "You have to tell us to read that page or we
won't, even if you tell us /about/ that page", then... meh, that's not a CA
that inspires confidence.

[1] https://www.youtube.com/watch?v=hou0lU8WMgo

[Chris Kemmerer]

We regret your impression that we take this issue with anything less than the utmost seriousness.

We have opened a ticket and are actively working with our CA software vendor to address the underlying issue.

Rather than stopping there, we have been working concurrently to put into place the necessary checks against openssl-blacklist independently of the CA software vendor.

Whether through our CA software vendor or independently, we are committed to finding a long-term solution that is effective and efficient.

We will provide regular updates of our progress to the bug (to which this message has also been posted).

[Matt Palmer]

On Wed, Mar 11, 2020 at 10:46:05AM -0700, Chris Kemmerer via dev-security-policy wrote:
> On Tuesday, March 10, 2020 at 8:44:49 PM UTC-5, Matt Palmer wrote:

> > On Tue, Mar 10, 2020 at 01:48:49PM -0700, Chris Kemmerer via dev-security-policy wrote:
> > > For the purpose of identifying whether a Private Key is weak, SSL.com uses
> > > a set of Debian weak keys that was provided by our CA software vendor as
> > > the basis for our blacklist.
> >
> > I think it's worth getting additional, *very* detailed, information from
> > your CA software vendor as to where *they* got their Debian weak key list
> > from. That appears to be the fundamental breakdown here -- you relied on a
> > third-party to give you good service, and they didn't. So I think that
> > digging into your vendor's practices is an important line of enquiry to go
> > down.
>

> As mentioned on our report, we used that list as a basis, and paid
> attention to augment it with other weak keys from available blacklists,

So presumably if there are other Mozilla-trusted CAs using the same CA
vendor, who *are* doing the bare minimum and just using the CA vendor's key
list, they're even more vulnerable to a potential misissuance. As you
mentioned that your CA software vendor does read this list, I *really* hope
they speak up soon so we can figure out how they got their key list.

> weak keys from available blacklists, even for the ROCA vulnerability.

Sidenote: my understanding of ROCA is that it is of a different form to the
Debian weak key problem, in that you can't a priori enumerate ROCA-impacted
keys, but can only identify them as you find them. As such, my
understanding is that there isn't, and in fact *cannot*, be a comprehensive
"blacklist", as such, of keys affected by ROCA.

Is your understanding of the ROCA vulnerability different to my description
above, and if not, can you explain how a "blacklist"-based approach is a
suitable mitigation for avoiding issuance of certificates using
ROCA-impacted private keys?

(Conversely, if it *is* possible get a comprehensive list of ROCA-impacted
keys, I know what I'm doing this weekend...)

> For what it's worth, we believe that the current language in the BRs could
> be less ambiguous as far as the Debian weak keys are concerned. For
> example, it seems that the community's expectations are for CAs to detect
> and block weak Debian keys generated by vulnerable RNG using OpenSSL in
> popular architectures.

The problem with using the argument that "the BRs are ambiguous" to try and
defend a breach of them is that there are always potential ambiguities in
all language -- in many ways, "ambiguity is in the eye of the beholder"
("ambiguer"?). My understanding of the consensus from past discussions on
this list is that if a CA believes there is an ambiguity in the BRs, the
correct action is to raise that in the CA/B Forum *before* they fall foul of
it.

CAs should be reading the BRs, as I understand it, in a "defensive" mode,
looking for requirements that could be read multiple ways, and when they are
found, the CA needs to ensure either that they are complying with the
strictest possible reading, or else bringing the ambiguity to the attention
of the CA/B Forum and suggesting less ambiguous wording.

At any rate, it would be helpful to know what, precisely, SSL.com's
understanding of this requirement of the BRs prior to the commencement of
this incident. Can you share this with us? Presumably SSL.com did a
careful analysis of all aspects of the BRs, and came to a conclusion as to
precisely what was considered "compliant". With regards to this
requirement, what was SSL.com's position as to what was necessary to be
compliant with this aspect of the BRs?

> That could be added in the BRs:
>
> Change:
> "The CA SHALL reject a certificate request if the requested Public Key
> does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if
> it has a known weak Private Key (such as a Debian weak key, see
> http://wiki.debian.org/SSLkeys)"
>
> to something like:
> "The CA SHALL reject a certificate request if the requested Public Key
> does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if

> it has a known weak Private Key using, as a minimum set the Debian weak
> keys produced by OpenSSL in i386 and x64 architectures (see
> http://wiki.debian.org/SSLkeys)"

It would appear that SSL.com is a member in good standing of the CA/B Forum.
Is there any intention on the part of SSL.com to propose this change as a
ballot? While you're at it, if you could include a fix for the issue
described in https://github.com/cabforum/documents/issues/164, that would be
appreciated, since it is the same sentences that need modification, and for
much the same reasons.

> Then, it would be clear that all CAs would need to block "at least" the
> vulnerable keys produced by OpenSSL and could add other keys produced by
> OpenSSH or other applications if they wanted a "more complete" list.

Well, you're still missing the rnd/nornd/noreadrnd variations, and there's
no specification as to what key sizes are considered the bare minimum.

> > > This information was disclosed on 2020-03-07 to the person that submitted
> > > the Certificate Problem Report.
> >
> > I don't see anything from SSL.com that looks relevant in my inbox, but it's
> > not an important point, just thought I'd mention it in passing.
>

> For the record, we replied to mpa...@hezmatt.org at 2020-03-07 8:41 PM (UTC)
> Email subject: "Problem Report for certificate(s) with compromised private key"
> Can you please confirm?
> We did disclose the source of our weak keys in that email.

Unfortunately that's the default subject line I use on my problem reports,
so it's not as helpful as it might otherwise be. Looking through recent
correspondence, though, I'm certainly not finding anything. If you've got a
message ID, or even better the queue ID that my MTA would have responded
with when it accepted the message (which could be obtained from your MTA's
logs), that'd help me track down what happened to it. At any rate, there's
no delivery attempts at *all* in my mail logs between 20:37:19 and 21:03:30
on 2020-03-07, so it looks like it got in the post to at least some degree.

> > > The above practices comply with our CP/CPS, version 1.8, section 6.1.1.2,
> > > which states:
> > >
> > > "SSL.com shall reject a certificate request if the request has a known
> > > weak Private Key".
> >
> > Hmm, "known" is doing a lot of work in that sentence. Known to *whom* is
> > the important, but unanswered, question.

[snip]

> This language was mainly taken from the Baseline Requirements.

Sure, but SSL.com presumably came to some understanding as to what that
sentence meant at the time it was included in SSL.com's CPS, given that the
organisation was committing to do it. I think it would be valuable to know
what that understanding was, if for no other reason than so that Mozilla can
say, in the next CA communication, "if you think that "known weak Private
Key" means "known to <X>, be advised that that that is not an appropriate
interpretation, and you'll want to get onto that."

> Unfortunately the format of the blacklists in openssl-blacklist is
> incompatible with our CA software which expects the SHA256 hash of the
> modulus, that's why we mention about acquiring the actual public keys.

That is an... interesting approach. How does your CA software check for
known-bad EC keys? Yes, OpenSSL 0.9.8 didn't support EC keys, but Debian
weak keys aren't the only known-bad keys in existence.

> In our understanding, many individuals are in possession of such lists and
> it would certainly expedite things, along with reducing waste from
> duplicate effort, if you or others with similar lists, decided to share
> the actual keys with the broader community.

Given the number of active, unrevoked certificates using private keys that
I've already found laying around (see, for example,
https://misissued.com/batch/64/), dumping all the known-compromised private
keys in existence in one place *might* not be considered a winning idea by
all and sundry. I do appreciate your desire to watch the world burn,
though. It's very Cyberpunk.

Further, a point-in-time publication of any list of keys would not be
particularly useful, as new private keys are *regularly* being disclosed,
and thus any list would very quickly be out-of-date. Even if the published
list were regularly augmented with new entries, history has shown that
people are far too keen on downloading a list once, stuffing into their (in
this case) CA software, and saying "job done!", and thus they lull
themselves into a false sense of security.

I've seen this happen in many contexts over the years, such as in network
operations -- bogon lists get loaded into routers and firewalls, they don't
get updated, and it causes problems everywhere and forever. I'm not going
to be party to creating yet another instance of the same problem with
private keys.

> The same applies for private keys that have been publicly disclosed.
> Maintaining such a list and justifying each entry is a great contribution
> to the community. Even getting the modulus of those keys would solve many

> compatibility issues with various "key detection" solutions.

I am not averse to the provision of alternate formats of the pwnedkeys.com
dataset, however I don't have enough spare time to generate every possible
permutation of key representation, especially when they're as odd as "SHA256
of the modulus" (even assuming there were a single unambiguous
representation of an RSA modulus, which there most certainly isn't).

> Regarding the use of external third-party systems, such as the
> pwnedkeys.com database, we are of the opinion that no CA's certificate
> issuance process and/or their compliance should rely on an external system
> which has not been evaluated and without any SLA guarantees.

There are commercial solutions to the problem of a lack of SLA guarantees,
if that is the only roadblock.

- Matt

[Chris Kemmerer]

ROCA vulnerability detection is part of our “weak keys detection mechanism”, not part of a blacklist. Our original language “we do have a weak keys detection mechanism in place, it does detect Debian weak keys (although it's not perfect) and it also detects ROCA vulnerable keys” makes that clear.

>
> > For what it's worth, we believe that the current language in the BRs could
> > be less ambiguous as far as the Debian weak keys are concerned. For
> > example, it seems that the community's expectations are for CAs to detect
> > and block weak Debian keys generated by vulnerable RNG using OpenSSL in
> > popular architectures.
>
> The problem with using the argument that "the BRs are ambiguous" to try and
> defend a breach of them is that there are always potential ambiguities in
> all language -- in many ways, "ambiguity is in the eye of the beholder"
> ("ambiguer"?). My understanding of the consensus from past discussions on
> this list is that if a CA believes there is an ambiguity in the BRs, the
> correct action is to raise that in the CA/B Forum *before* they fall foul of
> it.
>
> CAs should be reading the BRs, as I understand it, in a "defensive" mode,
> looking for requirements that could be read multiple ways, and when they are
> found, the CA needs to ensure either that they are complying with the
> strictest possible reading, or else bringing the ambiguity to the attention
> of the CA/B Forum and suggesting less ambiguous wording.
>
> At any rate, it would be helpful to know what, precisely, SSL.com's
> understanding of this requirement of the BRs prior to the commencement of
> this incident. Can you share this with us? Presumably SSL.com did a
> careful analysis of all aspects of the BRs, and came to a conclusion as to
> precisely what was considered "compliant". With regards to this
> requirement, what was SSL.com's position as to what was necessary to be
> compliant with this aspect of the BRs?

We have already described our understanding of the expectations expressed in BR 6.1.1.3 and the steps we took to comply with it. We provide details in bug 1620772, which remains the primary channel for this issue. As always, we attempt to be as transparent as possible, because we strongly feel that this is exactly the approach which better serves the ecosystem.
Our implementation did not meet these expectations, as it was missing direct checks of keys matching the "openssl-blacklist" package. Immediately upon our coming to this understanding, we initiated development of a fix to meet this expectation. This fix was tested and pushed to production last Friday. In parallel, we are conducting an analysis of which control(s) failed. Details on our findings thus far and steps we are taking in remediation are also included in our incident report.

>
> > That could be added in the BRs:
> >
> > Change:
> > "The CA SHALL reject a certificate request if the requested Public Key
> > does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if
> > it has a known weak Private Key (such as a Debian weak key, see
> > http://wiki.debian.org/SSLkeys)"
> >
> > to something like:
> > "The CA SHALL reject a certificate request if the requested Public Key
> > does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if
> > it has a known weak Private Key using, as a minimum set the Debian weak
> > keys produced by OpenSSL in i386 and x64 architectures (see
> > http://wiki.debian.org/SSLkeys)"
>
> It would appear that SSL.com is a member in good standing of the CA/B Forum.
> Is there any intention on the part of SSL.com to propose this change as a
> ballot? While you're at it, if you could include a fix for the issue
> described in https://github.com/cabforum/documents/issues/164, that would be
> appreciated, since it is the same sentences that need modification, and for
> much the same reasons.

Yes, this is reasonable, and we treated such key as compromised, revoking it within 24 hours.

We would support a ballot that makes this clear. We also monitor the discussion in https://github.com/cabforum/documents/issues/164.

>
> > Then, it would be clear that all CAs would need to block "at least" the
> > vulnerable keys produced by OpenSSL and could add other keys produced by
> > OpenSSH or other applications if they wanted a "more complete" list.
>
> Well, you're still missing the rnd/nornd/noreadrnd variations, and there's
> no specification as to what key sizes are considered the bare minimum.

We mention these variations in bug 1620772, but thank you for repeating it here for completeness.
For the record, this fact (“there's no specification as to what key sizes are considered the bare minimum”) is exactly our point too.

The BRs allow specific RSA key sizes. If we want to be unambiguous about this requirement, we need to include our minimum (baseline) expectations.

>
> > > > This information was disclosed on 2020-03-07 to the person that submitted
> > > > the Certificate Problem Report.
> > >
> > > I don't see anything from SSL.com that looks relevant in my inbox, but it's
> > > not an important point, just thought I'd mention it in passing.
> >
> > For the record, we replied to mpa...@hezmatt.org at 2020-03-07 8:41 PM (UTC)
> > Email subject: "Problem Report for certificate(s) with compromised private key"
> > Can you please confirm?
> > We did disclose the source of our weak keys in that email.
>
> Unfortunately that's the default subject line I use on my problem reports,
> so it's not as helpful as it might otherwise be. Looking through recent
> correspondence, though, I'm certainly not finding anything. If you've got a
> message ID, or even better the queue ID that my MTA would have responded
> with when it accepted the message (which could be obtained from your MTA's
> logs), that'd help me track down what happened to it. At any rate, there's
> no delivery attempts at *all* in my mail logs between 20:37:19 and 21:03:30
> on 2020-03-07, so it looks like it got in the post to at least some degree.

The reply was submitted to the MTA at 2020-03-07 04:04 UTC. Apologies that the timezone conversion was incorrect. The ticket number was QGI-680-16751.

>
> > > > The above practices comply with our CP/CPS, version 1.8, section 6.1.1.2,
> > > > which states:
> > > >
> > > > "SSL.com shall reject a certificate request if the request has a known
> > > > weak Private Key".
> > >
> > > Hmm, "known" is doing a lot of work in that sentence. Known to *whom* is
> > > the important, but unanswered, question.
>
> [snip]
>
> > This language was mainly taken from the Baseline Requirements.
>
> Sure, but SSL.com presumably came to some understanding as to what that
> sentence meant at the time it was included in SSL.com's CPS, given that the
> organisation was committing to do it. I think it would be valuable to know
> what that understanding was, if for no other reason than so that Mozilla can
> say, in the next CA communication, "if you think that "known weak Private
> Key" means "known to <X>, be advised that that that is not an appropriate
> interpretation, and you'll want to get onto that."

We have responded as to how we interpreted and implemented this requirement. Our blacklist did not include the entire set of Debian weak keys. We have been completely transparent about this issue and we have improved our weak key detection mechanism to include the openssl-blacklist package.

We examined similar failures/incidents, such as:

- https://bugzilla.mozilla.org/show_bug.cgi?id=1472052
- https://bugzilla.mozilla.org/show_bug.cgi?id=1435770
- https://community.letsencrypt.org/t/2017-09-09-late-weak-key-revocation/42519

The last one shows that at least one more CA had a similar interpretation of the BRs.

For us, this is an indication that this issue is not one particular interpretation of a single CA. Updating this section in the BRs to reduce misinterpretation would be beneficial for the greater PKI ecosystem.

>
> > Unfortunately the format of the blacklists in openssl-blacklist is
> > incompatible with our CA software which expects the SHA256 hash of the
> > modulus, that's why we mention about acquiring the actual public keys.
>
> That is an... interesting approach. How does your CA software check for
> known-bad EC keys? Yes, OpenSSL 0.9.8 didn't support EC keys, but Debian
> weak keys aren't the only known-bad keys in existence.

For EC keys, the CA software uses a different method to calculate the fingerprints.

>
> > In our understanding, many individuals are in possession of such lists and
> > it would certainly expedite things, along with reducing waste from
> > duplicate effort, if you or others with similar lists, decided to share
> > the actual keys with the broader community.
>
> Given the number of active, unrevoked certificates using private keys that
> I've already found laying around (see, for example,
> https://misissued.com/batch/64/), dumping all the known-compromised private
> keys in existence in one place *might* not be considered a winning idea by
> all and sundry. I do appreciate your desire to watch the world burn,
> though. It's very Cyberpunk.

We only expected a list of weak or compromised public keys, not private keys, to be disclosed.

>
> Further, a point-in-time publication of any list of keys would not be
> particularly useful, as new private keys are *regularly* being disclosed,
> and thus any list would very quickly be out-of-date. Even if the published
> list were regularly augmented with new entries, history has shown that
> people are far too keen on downloading a list once, stuffing into their (in
> this case) CA software, and saying "job done!", and thus they lull
> themselves into a false sense of security.
>
> I've seen this happen in many contexts over the years, such as in network
> operations -- bogon lists get loaded into routers and firewalls, they don't
> get updated, and it causes problems everywhere and forever. I'm not going
> to be party to creating yet another instance of the same problem with
> private keys.

Once a private key has been compromised, it's compromised. It's an "add-only" database. Therefore, we believe it would be a large contribution to this community if the *public keys* of compromised private keys (and reason of compromise for verification) were available for download. CAs could periodically synchronize their local blacklists with new entries of this blacklist. This is one way we see to deal with the interoperability issue of different implementations for weak/compromised key detection.

We would welcome discussion with security researchers regarding collecting, documenting and disclosing such weak/compromised public keys as a means of improving weak/compromised key detection by CAs and other participants in the PKI community.

>
> > The same applies for private keys that have been publicly disclosed.
> > Maintaining such a list and justifying each entry is a great contribution
> > to the community. Even getting the modulus of those keys would solve many
> > compatibility issues with various "key detection" solutions.
>
> I am not averse to the provision of alternate formats of the pwnedkeys.com
> dataset, however I don't have enough spare time to generate every possible
> permutation of key representation, especially when they're as odd as "SHA256
> of the modulus" (even assuming there were a single unambiguous
> representation of an RSA modulus, which there most certainly isn't).

This was not our suggestion. Providing the public key associated with each compromised private key would allow any implementation to do the necessary conversions independently.

>
> > Regarding the use of external third-party systems, such as the
> > pwnedkeys.com database, we are of the opinion that no CA's certificate
> > issuance process and/or their compliance should rely on an external system
> > which has not been evaluated and without any SLA guarantees.
>
> There are commercial solutions to the problem of a lack of SLA guarantees,
> if that is the only roadblock.

SSL.com has been and continues to be transparent, by disclosing practices and sources of information along with its understanding of the situation. Overall, although we regret the circumstances, we view this issue as an opportunity for improvement of both our own and the community’s understanding and operations. We attempt to capture this, and to provide concrete steps for this improvement, in our full incident report as posted in the corresponding bug.

[Ryan Sleevi]

On Mon, Mar 16, 2020 at 3:12 PM Chris Kemmerer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> > It would appear that SSL.com is a member in good standing of the CA/B
> Forum.
> > Is there any intention on the part of SSL.com to propose this change as a
> > ballot? While you're at it, if you could include a fix for the issue
> > described in https://github.com/cabforum/documents/issues/164, that
> would be
> > appreciated, since it is the same sentences that need modification, and
> for
> > much the same reasons.
>

> Yes, this is reasonable, and we treated such key as compromised, revoking
> it within 24 hours.
>
> We would support a ballot that makes this clear. We also monitor the
> discussion in https://github.com/cabforum/documents/issues/164.
>

While you answered "Yes", you followed with a different clarification. "We
would support" seems different from "Is there any intention on SSL.com to
propose"

> We have responded as to how we interpreted and implemented this
> requirement. Our blacklist did not include the entire set of Debian weak
> keys. We have been completely transparent about this issue and we have
> improved our weak key detection mechanism to include the openssl-blacklist
> package.
>
> We examined similar failures/incidents, such as:
>
> - https://bugzilla.mozilla.org/show_bug.cgi?id=1472052
> - https://bugzilla.mozilla.org/show_bug.cgi?id=1435770
> -
> https://community.letsencrypt.org/t/2017-09-09-late-weak-key-revocation/42519
>
> The last one shows that at least one more CA had a similar interpretation
> of the BRs.
>

Having read these, while I appreciate SSL.com highlighting them, I fail to
see them supporting the claim being made here. Could you please elaborate
why/how you believe this statement to be true? They seem rather remarkably
different, and certainly, the last one highlights a CA actively working to
be comprehensive, while much of SSL.com's reply seems to be "We shouldn't
have to be comprehensive"

[Chris Kemmerer]

On Monday, March 16, 2020 at 2:46:46 PM UTC-5, Ryan Sleevi wrote:
> On Mon, Mar 16, 2020 at 3:12 PM Chris Kemmerer via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > > It would appear that SSL.com is a member in good standing of the CA/B
> > Forum.
> > > Is there any intention on the part of SSL.com to propose this change as a
> > > ballot? While you're at it, if you could include a fix for the issue
> > > described in https://github.com/cabforum/documents/issues/164, that
> > would be
> > > appreciated, since it is the same sentences that need modification, and
> > for
> > > much the same reasons.
> >
> > Yes, this is reasonable, and we treated such key as compromised, revoking
> > it within 24 hours.
> >
> > We would support a ballot that makes this clear. We also monitor the
> > discussion in https://github.com/cabforum/documents/issues/164.
> >
>
> While you answered "Yes", you followed with a different clarification. "We
> would support" seems different from "Is there any intention on SSL.com to
> propose"

Yes, we are happy to propose a ballot change to the BR language pertaining to this issue.

>
> > We have responded as to how we interpreted and implemented this
> > requirement. Our blacklist did not include the entire set of Debian weak
> > keys. We have been completely transparent about this issue and we have
> > improved our weak key detection mechanism to include the openssl-blacklist
> > package.
> >
> > We examined similar failures/incidents, such as:
> >
> > - https://bugzilla.mozilla.org/show_bug.cgi?id=1472052
> > - https://bugzilla.mozilla.org/show_bug.cgi?id=1435770
> > -
> > https://community.letsencrypt.org/t/2017-09-09-late-weak-key-revocation/42519
> >
> > The last one shows that at least one more CA had a similar interpretation
> > of the BRs.
> >
>
> Having read these, while I appreciate SSL.com highlighting them, I fail to
> see them supporting the claim being made here. Could you please elaborate
> why/how you believe this statement to be true? They seem rather remarkably
> different, and certainly, the last one highlights a CA actively working to
> be comprehensive, while much of SSL.com's reply seems to be "We shouldn't
> have to be comprehensive"

We were merely pointing out previous discussions on this topic. I can see why you may have such an impression, but "we shouldn't have to be comprehensive" is not what we are thinking or believe. On the contrary, we've increased our efforts to be as comprehensive as possible, and we will continue to expand our weak keys list even after this issue is closed.

We believe in a robustly secure ecosystem, not in half measures.

[Matt Palmer]

On Mon, Mar 16, 2020 at 12:11:57PM -0700, Chris Kemmerer via dev-security-policy wrote:

> We have already described our understanding of the expectations expressed
> in BR 6.1.1.3 and the steps we took to comply with it.

Sorry, I must have missed the description of SSL.com's understanding. Could
you quote or reference it here, for clarity?

> Our implementation did not meet these expectations, as it was missing
> direct checks of keys matching the "openssl-blacklist" package.
> Immediately upon our coming to this understanding,

This is SSL.com's post-incident understanding; what I believe is important
to also know is SSL.com's *pre*-incident understanding of the BR
requirements.

> > > That could be added in the BRs:
> > >
> > > Change:
> > > "The CA SHALL reject a certificate request if the requested Public Key
> > > does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if
> > > it has a known weak Private Key (such as a Debian weak key, see
> > > http://wiki.debian.org/SSLkeys)"
> > >
> > > to something like:
> > > "The CA SHALL reject a certificate request if the requested Public Key
> > > does not meet the requirements set forth in Sections 6.1.5 and 6.1.6 or if
> > > it has a known weak Private Key using, as a minimum set the Debian weak
> > > keys produced by OpenSSL in i386 and x64 architectures (see
> > > http://wiki.debian.org/SSLkeys)"
> >
> > It would appear that SSL.com is a member in good standing of the CA/B Forum.
> > Is there any intention on the part of SSL.com to propose this change as a
> > ballot? While you're at it, if you could include a fix for the issue
> > described in https://github.com/cabforum/documents/issues/164, that would be
> > appreciated, since it is the same sentences that need modification, and for
> > much the same reasons.
>

> Yes, this is reasonable, and we treated such key as compromised, revoking
> it within 24 hours.
>
> We would support a ballot that makes this clear. We also monitor the
> discussion in https://github.com/cabforum/documents/issues/164.

As Ryan mentioned, "we would support a ballot" is not the same as "we intend
to propose a ballot". Thank you for clarifying that SSL.com intends to
propse a ballot in your reply to Ryan.

> > > Then, it would be clear that all CAs would need to block "at least" the
> > > vulnerable keys produced by OpenSSL and could add other keys produced by
> > > OpenSSH or other applications if they wanted a "more complete" list.
> >
> > Well, you're still missing the rnd/nornd/noreadrnd variations, and there's
> > no specification as to what key sizes are considered the bare minimum.
>

> We mention these variations in bug 1620772, but thank you for repeating it
> here for completeness.
>
> For the record, this fact (“there's no specification as to what key sizes
> are considered the bare minimum”) is exactly our point too.

I think you misunderstood my point here. I was making an observation that
your proposed improved language was not comprehensive, and would allow CAs
in the future to continue to argue that the BRs were ambiguous when *they*
issued a certificate with a known-to-the-openssl-blacklist-package weak key.
It was an attempt to show that drafting ambiguity-free requirements is
*hard*, to emphasise that reading the BRs in a "what's the bare minimum we
can possibly get away with" mindset is a dangerous path to tread.

> The BRs allow specific RSA key sizes.

I can't find anything in the BRs that mandates specific RSA public modulus
lengths. It defines a *minimum* length, to provide a reasonable security
level baseline, and most CAs impose maximum length (because, as I understand
it, 1,000,000 bit modulii tend to give HSMs indigestion), but I can't find
an explicit list of specific key sizes anywhere. Could you provide the
section number that describes which specific RSA key sizes are allowed?

> > > > > This information was disclosed on 2020-03-07 to the person that submitted
> > > > > the Certificate Problem Report.
> > > >
> > > > I don't see anything from SSL.com that looks relevant in my inbox, but it's
> > > > not an important point, just thought I'd mention it in passing.
> > >
> > > For the record, we replied to mpa...@hezmatt.org at 2020-03-07 8:41 PM (UTC)
> > > Email subject: "Problem Report for certificate(s) with compromised private key"
> > > Can you please confirm?
> > > We did disclose the source of our weak keys in that email.
> >
> > Unfortunately that's the default subject line I use on my problem reports,
> > so it's not as helpful as it might otherwise be. Looking through recent
> > correspondence, though, I'm certainly not finding anything. If you've got a
> > message ID, or even better the queue ID that my MTA would have responded
> > with when it accepted the message (which could be obtained from your MTA's
> > logs), that'd help me track down what happened to it. At any rate, there's
> > no delivery attempts at *all* in my mail logs between 20:37:19 and 21:03:30
> > on 2020-03-07, so it looks like it got in the post to at least some degree.
>

> The reply was submitted to the MTA at 2020-03-07 04:04 UTC. Apologies
> that the timezone conversion was incorrect. The ticket number was
> QGI-680-16751.

Still nothing in my MTA's logs for anywhere around that time. Would that
ticket number have been in the message subject? I log all subject lines,
and there's no incidence of that string in my logs for that week. As I said
before, either the message ID, or the queue ID that my MTA includes in the
`250` response to the DATA command, would be the best information to help me
track down the message.

> > > > > The above practices comply with our CP/CPS, version 1.8, section 6.1.1.2,
> > > > > which states:
> > > > >
> > > > > "SSL.com shall reject a certificate request if the request has a known
> > > > > weak Private Key".
> > > >
> > > > Hmm, "known" is doing a lot of work in that sentence. Known to *whom* is
> > > > the important, but unanswered, question.
> >
> > [snip]
> >
> > > This language was mainly taken from the Baseline Requirements.
> >
> > Sure, but SSL.com presumably came to some understanding as to what that
> > sentence meant at the time it was included in SSL.com's CPS, given that the
> > organisation was committing to do it. I think it would be valuable to know
> > what that understanding was, if for no other reason than so that Mozilla can
> > say, in the next CA communication, "if you think that "known weak Private
> > Key" means "known to <X>, be advised that that that is not an appropriate
> > interpretation, and you'll want to get onto that."
>

> We have responded as to how we interpreted and implemented this requirement.

Again, whilst you have explained SSL.com's implementation of this part of
its CPS, I don't feel that SSL.com's interpretation has been explained.
Specifically, the intended meaning of the word "known" in the phrase "known
weak Private Key" in the SSL.com CPS still, as far as I can see, has not
been explained.

> > Further, a point-in-time publication of any list of keys would not be
> > particularly useful, as new private keys are *regularly* being disclosed,
> > and thus any list would very quickly be out-of-date. Even if the published
> > list were regularly augmented with new entries, history has shown that
> > people are far too keen on downloading a list once, stuffing into their (in
> > this case) CA software, and saying "job done!", and thus they lull
> > themselves into a false sense of security.
> >
> > I've seen this happen in many contexts over the years, such as in network
> > operations -- bogon lists get loaded into routers and firewalls, they don't
> > get updated, and it causes problems everywhere and forever. I'm not going
> > to be party to creating yet another instance of the same problem with
> > private keys.
>

> Once a private key has been compromised, it's compromised. It's an
> "add-only" database. Therefore, we believe it would be a large
> contribution to this community if the *public keys* of compromised private
> keys (and reason of compromise for verification) were available for
> download. CAs could periodically synchronize their local blacklists with
> new entries of this blacklist. This is one way we see to deal with the
> interoperability issue of different implementations for weak/compromised
> key detection.

Is SSL.com willing to come to the party on this "large contribution", or are
they expecting volunteers to do all the work for them for free?

> We would welcome discussion with security researchers regarding
> collecting, documenting and disclosing such weak/compromised public keys
> as a means of improving weak/compromised key detection by CAs and other
> participants in the PKI community.

I think I compromise a representative portion of the set of "security
researchers" working on disclosed private keys. Here I am. Discuss away.

- Matt