GoDaddy Revocations Due to a Variety of Issues

518 views
Skip to first unread message

Daymion Reynolds

unread,
Jul 20, 2018, 9:39:17 PM7/20/18
to mozilla-dev-s...@lists.mozilla.org
Revoke Notification

GoDaddy has been proactively auditing certificates under management. We have identified 1000 certificates having one or more of the 6 issues defined below. The majority of these certs are 3yrs old or older. Most are from 2013 or before.

The certificates were identified by analyzing results from both zlint and certlint. We also verified all lint findings against current and past BRs. We discovered multiple defects with the linters, and submitted pull requests to correct them. See below.

Zlint PR to correct these issues:
https://github.com/zmap/zlint/pull/232
https://github.com/zmap/zlint/commit/12b8dc0338e6261fb4ad6a623c0a4c1bc99b3dfe

CertLint PRs to correct issues:

In Progress, will publish if requested.

Once we had confirmation of the lint issues we then proceeded to incrementally notify and revoke. See timeline is below.

Timeline of Events for Revocation:
6/26/2018 – 7/10/2018 – Test runs and bug fixes for Zlint/CertLint
7/11/2018 9:45am – First round list of certs identified to revoke.
------------------------------------------------------------------------------------------------------------------------------------------------
| Error | Quantity of Certs | Last Occurrence |
------------------------------------------------------------------------------------------------------------------------------------------------
|BR certificates must be 39 months in validity or less |27 |4/1/15 13:07 |
-------------------------------------------------------------------------------------------------------------------------------------------------
|BR certificates must be 60 months in validity or less |84 |8/7/13 16:48 |
-------------------------------------------------------------------------------------------------------------------------------------------------
|BR certificates with organizationName must include countryName |6 |1/17/13 14:51 |
-------------------------------------------------------------------------------------------------------------------------------------------------
| e_dnsname_not_valid_tld, | |
|e_subject_common_name_not_from_san, | |
|e_dnsname_bad_character_in_label |4 |*7/5/18 11:48 |
------------------------------------------------------------------------------------------------------------------------------------------------
| e_subject_common_name_not_from_san, | | |
|e_dnsname_bad_character_in_label |28 |*7/9/18 21:12 |
------------------------------------------------------------------------------------------------------------------------------------------------
| RSA subject key modulus must be at least 2048 bits |638 |10/5/09 8:25 |
------------------------------------------------------------------------------------------------------------------------------------------------
*Total of 17 certificates issued in 2018 were revoked due to invalid extended ascii characters. CertLint was not catching these issues, which would have prevented issuance. We have since remediated these problems, and are adding zLint to our certificate issuance process as a second check.
Issued in 2018 certificate serial numbers 4329668077199547083, 8815069853166416488, 8835430332440327484, 13229652153750393997, 12375089233389451640, 11484792606267277228, 11919098489171585007, 9486648889515633287, 14583473664717830410, 7612308405142602244, 4011153125742917275, 6919066797946454186, 15449193186990222652, 14380872970193550115, 1792501994142248245, 12601193235728728125, 10465762057746987360
Cert.sh was unavailable when this was crafted else I would provide links to the 4 certs which were CT logged.
7/11/2018 10:30am – Certificate revocation process started, with emails sent to certificate owners.
7/12/2018 9am – All first-round certificates revoked.
7/13/2018 11:30am - Second round of certs identified.
------------------------------------------------------------------------------------------------------------------------------------------------
| Error | Quantity of Certs | Last Occurrence |
------------------------------------------------------------------------------------------------------------------------------------------------
| RSA subject key modulus must be at least 2048 bits |213 |7/30/09 13:52 |
-------------------------------------------------------------------------------------------------------------------------------------------------
7/13/2018 1:30pm- Final round of cert revocations completed.
Please let us know of any questions or concerns.
Daymion Reynolds


Peter Bowen

unread,
Jul 21, 2018, 12:39:04 AM7/21/18
to drey...@godaddy.com, mozilla-dev-s...@lists.mozilla.org
On Fri, Jul 20, 2018 at 6:39 PM Daymion Reynolds via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> The certificates were identified by analyzing results from both zlint and
> certlint. We also verified all lint findings against current and past BRs.
> We discovered multiple defects with the linters, and submitted pull
> requests to correct them. See below.
>
> CertLint PRs to correct issues:
>
> In Progress, will publish if requested.
>

Yes, I would very much like to have either PRs or just a list of issues.


> | e_dnsname_not_valid_tld, |
> |
> |e_subject_common_name_not_from_san, |
> |
> |e_dnsname_bad_character_in_label |4
> |*7/5/18 11:48 |
>
> ------------------------------------------------------------------------------------------------------------------------------------------------
> | e_subject_common_name_not_from_san, | |
> |
> |e_dnsname_bad_character_in_label |28
> |*7/9/18 21:12 |
>
> ------------------------------------------------------------------------------------------------------------------------------------------------
> *Total of 17 certificates issued in 2018 were revoked due to invalid
> extended ascii characters. CertLint was not catching these issues, which
> would have prevented issuance. We have since remediated these problems, and
> are adding zLint to our certificate issuance process as a second check.
> Issued in 2018 certificate serial numbers 4329668077199547083,
> 8815069853166416488, 8835430332440327484, 13229652153750393997,
> 12375089233389451640, 11484792606267277228, 11919098489171585007,
> 9486648889515633287, 14583473664717830410, 7612308405142602244,
> 4011153125742917275, 6919066797946454186, 15449193186990222652,
> 14380872970193550115, 1792501994142248245, 12601193235728728125,
> 10465762057746987360
> Cert.sh was unavailable when this was crafted else I would provide links
> to the 4 certs which were CT logged.


https://crt.sh/?id=294808610&opt=zlint,cablint is one of the
certificates. It is not clear to me that there is an error here. The DNS
names in the SAN are correctly encoded and the Common Name in the subject
has one of the names found in the SAN. The Common Name contains a DNS name
that is the U-label form of one of the SAN entries.

It is currently undefined if this is acceptable or unacceptable for
certificates covered by the BRs. I put a CA/Browser Forum ballot forward a
while ago to try to clarify it was not acceptable, but it did not pass as
several CAs felt it was not only acceptable but is needed and desirable.

If Mozilla (or another browser) puts forward a policy on this, I'm happy to
update certlint to reflect the poicy.

Thanks,
Peter

Daymion Reynolds

unread,
Jul 23, 2018, 12:11:17 PM7/23/18
to mozilla-dev-s...@lists.mozilla.org
Once we have the CertLint change pull requests done we will submit them to this thread. They were much more involved, and many times more numerous. It is worth a write up on where they overlap and diverge.

Daymion

Joanna Fox

unread,
Jul 25, 2018, 5:07:50 PM7/25/18
to mozilla-dev-s...@lists.mozilla.org
On Friday, July 20, 2018 at 9:39:04 PM UTC-7, Peter Bowen wrote:
Using the example provided of https://crt.sh/?id=294808610&opt=zlint,cablint, the error to which we were addressing is, “ERROR: Characters in labels of DNSNames MUST be alphanumeric, - , _ or *”. RFC 5280 states that the SAN field can contain a dnsName but it must be in the IA5String format. IA5String is defined as the first 128 characters in the ASCII alphabet. Right now as this is defined, it does not include international variance of ISO 646. Should we revisit this issue to clarify if international characters should be included? GoDaddy would be in support of adding this clarification.

Thanks, Joanna

Peter Bowen

unread,
Jul 26, 2018, 1:24:36 PM7/26/18
to jwe...@godaddy.com, mozilla-dev-s...@lists.mozilla.org
On Wed, Jul 25, 2018 at 2:08 PM Joanna Fox via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Friday, July 20, 2018 at 9:39:04 PM UTC-7, Peter Bowen wrote:
> Using the example provided of
> https://crt.sh/?id=294808610&opt=zlint,cablint, the error to which we
> were addressing is, “ERROR: Characters in labels of DNSNames MUST be
> alphanumeric, - , _ or *”. RFC 5280 states that the SAN field can contain a
> dnsName but it must be in the IA5String format. IA5String is defined as
> the first 128 characters in the ASCII alphabet. Right now as this is
> defined, it does not include international variance of ISO 646. Should we
> revisit this issue to clarify if international characters should be
> included? GoDaddy would be in support of adding this clarification.


That error is coming from zlint and appears, from my reading, to be a bug
in zlint. The DNSName entries in the SAN in that certificate only contain
allowable characters. The commonName attribute value in the Subject does
have characters that are not allowed in the DNSName entry, but commonName
is allowed to be a utf8string, which allows these characters. Further the
commonName contains a fully qualified domain name that also appears in the
SAN, so that BR requirement is met.

The challenge in this case is that the BRs do not specify the required
encoding of domain names. This doesn't really matter when handling pre-IDN
domain names, but with Internationalized Domain Names (IDNs), encoding does
need to be specified. Until Mozilla or the CA/Browser Forum clarifies, I
do not think this certificate has any errors.

Thanks,
Peter

Wayne Thayer

unread,
Aug 1, 2018, 12:16:54 PM8/1/18
to mozilla-dev-security-policy
Thank you for this report Daymion. 3 of the issues were recent:

| e_dnsname_not_valid_tld, |
|
|e_subject_common_name_not_from_san, |
|
|e_dnsname_bad_character_in_label |4
|*7/5/18 11:48 |

I think the "e_dnsname_not_valid_tld" errors are false positives. 5 of the
6 certificates [1] are for the new .llc TLD and according to WHOIS were
issued after the FQDN was registered. The TLD hasn't yet been added to
zlint [2], triggering the error. The last one looks like the punycode
encoding in the SAN is triggering the error for the .рф TLD.

The other two errors are the result of SAN encoding that is, as Peter
described, not strictly a policy violation.

For the older issues, it's nice to see these older (most likely SHA-1 and
thus untrusted) certificates being cleaned up, but again I don't see any
policies being violated:

|BR certificates must be 39 months in validity or less |27
|4/1/15 13:07 |

The BR language is: "Except as provided for below, Subscriber Certificates
issued after 1 April 2015 MUST have a Validity Period no greater than 39
months"

The other 3 issues stopped occurring before Mozilla required BR compliance.

My conclusion is that there is no action required on Mozilla's behalf.

- Wayne

[1] https://crt.sh/?zlint=798&iCAID=904&minNotBefore=2018-01-01
[2] https://github.com/zmap/zlint/blob/master/data/newgtlds.txt

On Thu, Jul 26, 2018 at 10:26 AM Peter Bowen via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> On Wed, Jul 25, 2018 at 2:08 PM Joanna Fox via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > On Friday, July 20, 2018 at 9:39:04 PM UTC-7, Peter Bowen wrote:
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>

Nick Lamb

unread,
Aug 9, 2018, 10:31:42 AM8/9/18
to dev-secur...@lists.mozilla.org
On Fri, 20 Jul 2018 21:38:45 -0700
Peter Bowen via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:

> https://crt.sh/?id=294808610&opt=zlint,cablint is one of the
> certificates. It is not clear to me that there is an error here.
> The DNS names in the SAN are correctly encoded and the Common Name in
> the subject has one of the names found in the SAN. The Common Name
> contains a DNS name that is the U-label form of one of the SAN
> entries.
>
> It is currently undefined if this is acceptable or unacceptable for
> certificates covered by the BRs. I put a CA/Browser Forum ballot
> forward a while ago to try to clarify it was not acceptable, but it
> did not pass as several CAs felt it was not only acceptable but is
> needed and desirable.

It would be helpful if any such CAs can tell us why this was "needed and
desirable" with actual examples.

Since the CN field in Web PKI certs always contains information
duplicated from a field that has been better defined for decades I'm
guessing in most cases the cause is crappy software. But if we know
which software is crappy we can help get that fixed rather than
muddling along forever.

Ryan Sleevi

unread,
Aug 9, 2018, 11:33:14 AM8/9/18
to Nick Lamb, MDSP
This information is readily available in the discussions for CA/Browser
Forum Ballot 202 -
https://cabforum.org/2017/07/26/ballot-202-underscore-wildcard-characters/
- which would have unambiguously specified and clarified this.

The following CAs voted against: Buypass, CFCA, DocuSign France, Entrust,
GDCA, GlobalSign, SHECA

Buypass - https://cabforum.org/pipermail/public/2017-July/011744.html
CFCA - https://cabforum.org/pipermail/public/2017-July/011733.html
Docusign - https://cabforum.org/pipermail/public/2017-July/011708.html
Entrust - https://cabforum.org/pipermail/public/2017-July/011747.html
GlobalSign - https://cabforum.org/pipermail/public/2017-July/011692.html
GDCA - https://cabforum.org/pipermail/public/2017-July/011736.html
SHECA - https://cabforum.org/pipermail/public/2017-July/011737.html

You can see not all objections are strictly related to that matter at hand,
but hopefully it provides you further information.
Reply all
Reply to author
Forward
0 new messages