Compliance with 7.1.4.2.1 (internal names revocation)

[Nick Lamb]

As mentioned previously I have been working on verifying that CAs complied with BR 7.1.4.2.1 which requires

"Effective 1 October 2016, CAs SHALL revoke all unexpired Certificates whose subjectAlternativeName extension or Subject commonName field contains a Reserved IP Address or Internal Name."

This work is ongoing, but it seemed worth a brief note to say that so far things look good, I have detected complete compliance across ~1M certificates examined so far, with one small exception. My own monitor doesn't (yet?) have a public web site, so here's a crt.sh link

https://crt.sh/?spkisha256=7c0edb0ed3a86eae5f16bb9d8bb39532542cde75955bdda913fd75fb952ec803

This certificate is from a hierarchy that isn't trusted by Mozilla today, but is trusted by Microsoft (according to Rob's tools anyway). It is quite a mess, but would probably validate in most or all browsers as an SSL server certificate, and was presumably in active use on such a web server in the past. It has not been revoked, although the revocation infrastructure for the CA is still in place.

The end entity certificate which listed a non-Internet DNS name does not seem to still be in use on the web. The root CA is still issuing (as of December) but only for Slovenian domains and either largely or entirely for Pošta Slovenije itself, an ordinary government-owned postal monopoly of the sort which was usual across all of Europe in the twentieth century.

Do we care in m.d.s.policy about such deviations? Or only those which affect CAs trusted, recently trusted or soon to be trusted by Mozilla / NSS?

[Gervase Markham]

On 03/01/17 18:11, Nick Lamb wrote:
> As mentioned previously I have been working on verifying that CAs
> complied with BR 7.1.4.2.1

Thank you for your vigilance :-)

> Do we care in m.d.s.policy about such deviations? Or only those which
> affect CAs trusted, recently trusted or soon to be trusted by Mozilla
> / NSS?

Mozilla does not care about deviations in hierarchies it does not trust.
Other root store operators may care; if you don't have a contact for
Microsoft, email me and I can provide you with one. The community as a
whole may also be interested, so please feel free to continue posting
findings like this here, as this group's area of interest is larger than
just Mozilla's root policy.

Gerv

[Nick Lamb]

Work continues. After the initial good news, to my surprise the second million or so certificates processed threw up some deviations from major public CAs

Comodo
https://crt.sh/?id=1246507
https://crt.sh/?id=1825806

Verisign / Symantec
https://crt.sh/?id=1450883

I would appreciate feedback, generally from m.d.s.policy participants about whether they believe that for some reason these certificates did not need to be revoked to achieve compliance with 7.1.4.2.1 and specifically from Comodo and Symantec on why the certificates weren't in fact revoked.

I would also be interested in learning whether auditors would be expected to identify and report this deviation.

[Gervase Markham]

On 06/01/17 09:52, Nick Lamb wrote:
> Comodo https://crt.sh/?id=1246507
> https://crt.sh/?id=1825806
>
> Verisign / Symantec https://crt.sh/?id=1450883
>
> I would appreciate feedback, generally from m.d.s.policy participants
> about whether they believe that for some reason these certificates
> did not need to be revoked to achieve compliance with 7.1.4.2.1 and
> specifically from Comodo and Symantec on why the certificates weren't
> in fact revoked.

One possibility for the latter two is that Comodo and/or Symantec used
an algorithm for detecting certs with internal names which was "no
dots", which wouldn't have turned these up. .local is clearly a local
domain - RFC 6762. .corp was originally just another TLD, but it was
controversial due to widespread internal use, and I was arguing for it
to be reserved for special use, but I don't know if it ever was. Does
anyone know the current status?

However, the first one of the three is a clear internal name, and it
would be good to hear from Comodo as to how it missed their revocation
sweep.

> I would also be interested in learning whether auditors would be
> expected to identify and report this deviation.

Given that you had to process 2 million certs to find them, and given
that auditors currently check on a "sampling" basis, I wouldn't
necessarily expect auditors to find these.

Gerv

[Nick Lamb]

On Saturday, 7 January 2017 09:08:21 UTC, Gervase Markham wrote:
> One possibility for the latter two is that Comodo and/or Symantec used
> an algorithm for detecting certs with internal names which was "no
> dots", which wouldn't have turned these up. .local is clearly a local
> domain - RFC 6762. .corp was originally just another TLD, but it was
> controversial due to widespread internal use, and I was arguing for it
> to be reserved for special use, but I don't know if it ever was. Does
> anyone know the current status?

Presumably Mozilla would agree that checking only for dotless names was not adequate to meet the text or the intention of 7.1.4.2.1 here? Although .corp isn't called out as reserved, it has also never been delegated for use. Nothing ever legitimately owned these names in the Internet DNS hierarchy.

https://icannwiki.com/.corp says that ICANN labelled .corp and .home as "high risk" after reviewing research into how bad the collisions would be if they were delegated, with the implication that they'll remain high risk for the foreseeable future and so these TLDs mustn't be delegated.

Actually reserving these names would reward earlier misbehaviour, such as Microsoft and its certified professionals advising clients to use names in .corp for internal systems (Microsoft's documentation and training now says not to do this any more).

Even if ICANN relents tomorrow, and gives an outlier like Donuts control over .corp the certificates I'm examining significantly pre-date such a decision and if anything it would become even more urgent to revoke them.

> However, the first one of the three is a clear internal name, and it
> would be good to hear from Comodo as to how it missed their revocation
> sweep.

> Given that you had to process 2 million certs to find them, and given
> that auditors currently check on a "sampling" basis, I wouldn't
> necessarily expect auditors to find these.

I see. I guess I was mostly thinking about the aspects auditors are expected to be strong on, such as identifying whether a CA has a documented process to obey this aspect of 7.1.4.2.1 and ensured that process happened in a timely fashion.

CT grants us good (and eventually superb) visibility into whether they actually do obey 7.1.4.2.1 but the auditors are better placed to understand why they didn't, for example because of inadequate process. You will know from our interactions outside m.d.s.policy that I'm very sceptical of the value of audit in the Web PKI today, and one reason is that even when audit is the appropriate mechanism to detect a problem it doesn't seem to have worked. This was true (and Mozilla took appropriate action) for WoSign/ StartCom and it's been true in other problem cases too.

Anyway, I have another million certificates to examine now.

[Lewis Resmond]

May I ask a small offtopic question? How did you examine the certificates? Is there a mechanism where you can get all the *.pem files so you can check them with a self developed script?

[Nick Lamb]

On Saturday, 7 January 2017 23:22:08 UTC, Lewis Resmond wrote:
> May I ask a small offtopic question? How did you examine the certificates? Is there a mechanism where you can get all the *.pem files so you can check them with a self developed script?

Certificate Transparency logs keep copies of every certificate they issue an SCT for, hence the name "logs". You can request these copies (DER encoded sorry, not PEM) from a CT log using the get-entries call described in RFC 6962.

Currently I have some very crude code that does this, primarily it is currently harvesting certificates from Google's "pilot" log, and then I am running a variety of experiments, some of which I would think of as "public interest" and others not so much. The 7.1.4.2.1 checks are in that "public interest" category. I was already pretty sure I wanted to build a CT log monitor, so this is just paddling in the shallow end before I dive in for real.

Depending on the volumes involved public interest stuff will probably either appear here, or on a web pages linked from here.

I commend the approach of examining a big pile of real world Web PKI certificates for yourself if you have an interest and the ability to write code. However, if you have statistical questions about the certificates rather than an interest in what's inside each individual certificate - I'd suggest looking at censys.io rather than building a CT log monitor.

[Robin Alden]

Nick,
Thanks for the heads-up.
We agree that the certificates you found should have been revoked.

We revoked a body of certificates on 1st October 2016 in accordance with
7.1.4.2.1.
Regrettably a mistake was made when we created the list of certificates to
be revoked.

As a word of background we track replacement certificates within the
lifetime of a particular certificate order, in part to avoid unnecessary
certificate revocations and associated CRL bloat.
E.g. If a customer requests (and we issue) a certificate C1 for key k1 with
domains d1.dom, d2.com, and subsequently requests (and has issued) a
replacement certificate for C2 {k1, d1.com, d2.com, d3.com} we do not
automatically revoke the prior certificate - although we reserve the right
to do so.
Our error in creating the list of certificates to be revoked was in not
including in that list certificates for which a replacement certificate had
been requested.

We had two people independently, using different methods, produce the list
of certificates to be revoked today to increase confidence in the result.

This has led us to revoke 28 further certificates, including the 2 that you
brought to our attention.

Here are links to the certificates we have revoked today. All but 3 are
newly published today to CT.

https://crt.sh/?id=1246507
https://crt.sh/?id=1825806
https://crt.sh/?id=39425459
https://crt.sh/?id=74893260
https://crt.sh/?id=74893262
https://crt.sh/?id=74893264
https://crt.sh/?id=74893267
https://crt.sh/?id=74893270
https://crt.sh/?id=74893273
https://crt.sh/?id=74893275
https://crt.sh/?id=74893278
https://crt.sh/?id=74893281
https://crt.sh/?id=74893284
https://crt.sh/?id=74893286
https://crt.sh/?id=74893289
https://crt.sh/?id=74893292
https://crt.sh/?id=74893295
https://crt.sh/?id=74893297
https://crt.sh/?id=74893299
https://crt.sh/?id=74893301
https://crt.sh/?id=74893303
https://crt.sh/?id=74893305
https://crt.sh/?id=74893307
https://crt.sh/?id=74893308
https://crt.sh/?id=74893310
https://crt.sh/?id=74893312
https://crt.sh/?id=74893314
https://crt.sh/?id=74893317

Thank you for your diligence.

Regards
Robin Alden
Comodo

> -----Original Message-----
> From: dev-security-policy On Behalf Of Nick Lamb
> Sent: 06 January 2017 09:52
> To: mozilla-dev-s...@lists.mozilla.org
> Subject: Re: Compliance with 7.1.4.2.1 (internal names revocation)
>
> Work continues. After the initial good news, to my surprise the second
> million or so certificates processed threw up some deviations from major
> public CAs
>

> Comodo
> https://crt.sh/?id=1246507
> https://crt.sh/?id=1825806
>
> Verisign / Symantec
> https://crt.sh/?id=1450883
>
> I would appreciate feedback, generally from m.d.s.policy participants
about
> whether they believe that for some reason these certificates did not need
to
> be revoked to achieve compliance with 7.1.4.2.1 and specifically from
> Comodo and Symantec on why the certificates weren't in fact revoked.
>

> I would also be interested in learning whether auditors would be expected
to
> identify and report this deviation.

> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy

[Nick Lamb]

On Monday, 9 January 2017 14:05:25 UTC, Robin Alden wrote:
> Nick,
> Thanks for the heads-up.
> We agree that the certificates you found should have been revoked.

Thank you Robin for investigating this, for your explanation of what happened and for the sensible response of CT logging and revoking the affected certificates. Please pass on my thanks to any additional people at Comodo who made that happen.

It would also be good to know (if you have relevant insight) whether you would expect your auditors to

a) Notice and report if Comodo had not even tried to comply with this element of 7.1.4.2.1
OR
b) Notice and report the type of mistake made here, in which a process was followed to attempt compliance but it missed a proportion of affected certificates.

[Robin Alden]

Hi Nick,

I expect that our auditors would have noticed and reported if we had not
tried to comply with 7.1.4.2.1.
Our next WebTrust audit starts shortly and I anticipate that the criteria
used will be
"WebTrust Principles and Criteria for Certification Authorities - SSL
Baseline with Network Security - Version 2.1"
http://www.webtrust.org/principles-and-criteria/item83666.pdf
Those criteria specifically call out 7.1.4.2.1 and the 1 October 2016 date.

Regards
Robin Alden
Comodo

> -----Original Message-----
> From: dev-security-policy [mailto:dev-security-policy-
> bounces+robin=comod...@lists.mozilla.org] On Behalf Of Nick Lamb
> Sent: 09 January 2017 16:41
> To: mozilla-dev-s...@lists.mozilla.org
> Subject: Re: Compliance with 7.1.4.2.1 (internal names revocation)
>

> On Monday, 9 January 2017 14:05:25 UTC, Robin Alden wrote:

> > Nick,
> > Thanks for the heads-up.
> > We agree that the certificates you found should have been revoked.
>

> Thank you Robin for investigating this, for your explanation of what
> happened and for the sensible response of CT logging and revoking the
> affected certificates. Please pass on my thanks to any additional people
at
> Comodo who made that happen.
>
> It would also be good to know (if you have relevant insight) whether you
> would expect your auditors to
>
> a) Notice and report if Comodo had not even tried to comply with this
> element of 7.1.4.2.1
> OR
> b) Notice and report the type of mistake made here, in which a process was
> followed to attempt compliance but it missed a proportion of affected
> certificates.

[Rick Andrews]

Thanks for finding this, Nick. We're in the process of revoking the cert you found, and searching for any others. We'll get back to you when we're done.

[Rick Andrews]

We conducted a search of our databases in September 2016, in which we examined every CN and SAN in every certificate still valid at the time. Each CN and SAN was examined to see if it contained no dot or an invalid DNS suffix; if so, the certificate was classified as an internal server cert and revoked. For all remaining CNs and SANs, those were checked against our internal list of TLDs built from information provided by ICANN and IANA. That list had a status value associated with each TLD, and our mistake was in excluding TLDs with certain status values.

Our scans conducted this week discovered three additional certificates that had not been revoked as of October 2016. These, and the certificate discovered by Nick, have now been revoked. Here are the links to those certificates:

https://crt.sh/?sha256=A642406A2BDF92DF8C9FB9322A81736506DDED79A20A7CD33CBEFD2AD2581167
https://crt.sh/?sha256=12B3CCC45D66B9CB2206DEF1C5A24B062CCC938694C92A0806D1D34845C0FC19
https://crt.sh/?sha256=E90AFAE4998D2B8103058ADF35810D87CCE5E98A0E1D691D2A558A6A4E115BAC

Thanks again to Nick for discovering this and pointing it out.

[Steve Medin]

Symantec has an additional disclosure regarding internal name certificates
valid after October 1. First, we disclose 3 certificates that remained valid
after October 1 but expired prior to our previous report. Second, we
disclose 3 certificates that were revoked as a result of our analysis but
not included in our initial report.

The cause of both issues is the execution of a query to inform us what
action needed to be taken within 24 hours. That result excluded revoked and
expired certificates. This led to our initial report of additional
certificates revoked along with the one reported to us by Nick Lamb.

The specific cause of the additional revoked but not disclosed certificates
is proactive effort by a team member to consult with two customers with
relationship/enterprise accounts concurrent with other efforts to work with
individual certificate owners. The revoked relationship/enterprise account
certificates we disclose today were revoked prior to execution of the report
and the report was used as the basis for our prior disclosure.

Disclosure:
https://crt.sh/?q=3518624
https://crt.sh/?q=78728901
https://crt.sh/?q=78728902
https://crt.sh/?q=78728903
https://crt.sh/?q=78728904
https://crt.sh/?q=78728905

> -----Original Message-----
> From: dev-security-policy [mailto:dev-security-policy-
> bounces+steve_medin=symant...@lists.mozilla.org] On Behalf Of Rick
> Andrews
> Sent: Wednesday, January 11, 2017 10:26 AM
> To: mozilla-dev-s...@lists.mozilla.org
> Subject: Re: Compliance with 7.1.4.2.1 (internal names revocation)
>

> We conducted a search of our databases in September 2016, in which we
> examined every CN and SAN in every certificate still valid at the time.
Each
> CN and SAN was examined to see if it contained no dot or an invalid DNS
> suffix; if so, the certificate was classified as an internal server cert
and
> revoked. For all remaining CNs and SANs, those were checked against our
> internal list of TLDs built from information provided by ICANN and IANA.
That
> list had a status value associated with each TLD, and our mistake was in
> excluding TLDs with certain status values.
>
> Our scans conducted this week discovered three additional certificates
that
> had not been revoked as of October 2016. These, and the certificate
> discovered by Nick, have now been revoked. Here are the links to those
> certificates:
>

> https://clicktime.symantec.com/a/1/zaK1Ry0U7rpBU7N7oUg8VKvELOYaomC
> 6td_b_grLhtQ=?d=1Tjdh1nkBUvl3Ieoed4QOfdma64XoBtRI7P4FrBClOZzIPZC6
> gloJVNfUNg7YuoczOU1s5h2FQEikj_V4Ek5gom-
> nUsaD5z1M_mr1BK_8M5KQx5C4M6oPnnIGHObc6tL3ilL07CqP7riK7XrmNexc
> _jukzroGa-ablqJpuYEfAsJXEkYRZLKsjUdW5nvTQ8rdmamWA6T_-
> 7CR8rpZFMtJ3OUHyIBvnFwqBIeteRjXzTHckwBBi3RZ8XQIlN8WokwyTFhO9otr
> lKAPBNSs9Y_kKCnwrJ7cl_y7enkSqc8A4Fmu57zdPIvh1c4sxaFQEBSyPTztGqi1L
> ai72GG1ArkQrZrGwBYvLscIjca4dTCi6JyGANQtcoumZ5Dzk6G4WK2SkVtDPMT
> pZ9YT1Hr16bXatTxRll3mWVHnROQDbDnmyzKOC_1uYVyyZTfj_HYA90Z4htBg
> MyBCz_rhfAbwqHhXd6ijIZdKd_pHhu_WA%3D%3D&u=https%3A%2F%2Fcrt.
> sh%2F%3Fsha256%3DA642406A2BDF92DF8C9FB9322A81736506DDED79A20A
> 7CD33CBEFD2AD2581167
> https://clicktime.symantec.com/a/1/0-
> oGgxxfVZ5MoF1oKVElUpBOfhFQcamcIpg21Ex6nNI=?d=1Tjdh1nkBUvl3Ieoed
> 4QOfdma64XoBtRI7P4FrBClOZzIPZC6gloJVNfUNg7YuoczOU1s5h2FQEikj_V4E
> k5gom-
> nUsaD5z1M_mr1BK_8M5KQx5C4M6oPnnIGHObc6tL3ilL07CqP7riK7XrmNexc
> _jukzroGa-ablqJpuYEfAsJXEkYRZLKsjUdW5nvTQ8rdmamWA6T_-
> 7CR8rpZFMtJ3OUHyIBvnFwqBIeteRjXzTHckwBBi3RZ8XQIlN8WokwyTFhO9otr
> lKAPBNSs9Y_kKCnwrJ7cl_y7enkSqc8A4Fmu57zdPIvh1c4sxaFQEBSyPTztGqi1L
> ai72GG1ArkQrZrGwBYvLscIjca4dTCi6JyGANQtcoumZ5Dzk6G4WK2SkVtDPMT
> pZ9YT1Hr16bXatTxRll3mWVHnROQDbDnmyzKOC_1uYVyyZTfj_HYA90Z4htBg
> MyBCz_rhfAbwqHhXd6ijIZdKd_pHhu_WA%3D%3D&u=https%3A%2F%2Fcrt.
> sh%2F%3Fsha256%3D12B3CCC45D66B9CB2206DEF1C5A24B062CCC938694C92
> A0806D1D34845C0FC19
> https://clicktime.symantec.com/a/1/UzPJvyQ4_OFDb-
> clEVONu_2vV6i20nAXDeD9Ur9jZvw=?d=1Tjdh1nkBUvl3Ieoed4QOfdma64Xo
> BtRI7P4FrBClOZzIPZC6gloJVNfUNg7YuoczOU1s5h2FQEikj_V4Ek5gom-
> nUsaD5z1M_mr1BK_8M5KQx5C4M6oPnnIGHObc6tL3ilL07CqP7riK7XrmNexc
> _jukzroGa-ablqJpuYEfAsJXEkYRZLKsjUdW5nvTQ8rdmamWA6T_-
> 7CR8rpZFMtJ3OUHyIBvnFwqBIeteRjXzTHckwBBi3RZ8XQIlN8WokwyTFhO9otr
> lKAPBNSs9Y_kKCnwrJ7cl_y7enkSqc8A4Fmu57zdPIvh1c4sxaFQEBSyPTztGqi1L
> ai72GG1ArkQrZrGwBYvLscIjca4dTCi6JyGANQtcoumZ5Dzk6G4WK2SkVtDPMT
> pZ9YT1Hr16bXatTxRll3mWVHnROQDbDnmyzKOC_1uYVyyZTfj_HYA90Z4htBg
> MyBCz_rhfAbwqHhXd6ijIZdKd_pHhu_WA%3D%3D&u=https%3A%2F%2Fcrt.
> sh%2F%3Fsha256%3DE90AFAE4998D2B8103058ADF35810D87CCE5E98A0E1D6

> 91D2A558A6A4E115BAC
>
> Thanks again to Nick for discovering this and pointing it out.
>