CA Validation quality is failing

[Mike Pasarella]

I found out that often the OV or EV validation of CA's is lacking and concerning the baseline requirements data submitted for a TLS certificate should be valid and thus validated. So when a country is Amsterdam, that should fail or a city Utrecht is placed in the province Zuid-Holland, that should fail to. And in my believe these checks are not difficult, just implement the Google Maps API and you would probably fix 60% of the bad certificate data. But The thing I do not understand is when validating a company, which will use the certificate for whatever website request a TLS certificate. Mostly the company registration office (for example KVK in the Netherlands) will supply the CA with correct data. If the data submitted by the certificate requester is incorrect, the certificate should never be issued. Period.

Here is a public link of a screenshot from the data found in the certificate:
https://dl.dropboxusercontent.com/u/2676712/digicert.png

Lately I discover those issues with several DigiCert certificates, but they are not the only CA making those mistakes. And to be honest those mistake really downgrade the OV and or EV value of the certificates to nothing more than a domain validated, encryption only TLS connection. As part of this bad validation and in my opinion failing to comply to the baseline requirements. Which could initiative encourage phishing and the de-trust in TLS in general.

Kind Regards,

Mike

[Mike Pasarella]

To add some more concerning this issue:

https://xenapp.alpinvest.com/
https://adoftheyear.com
https://secure.mobihealth.com
https://portal.mobilitymixx.nl
https://mijn.nfu.nl
https://portal.payplaza.com

I also believe that this happens often with the re-use of once (wrong) data for issue-ing new certificates.
I think it is a bad idea that SSL certificates for OV and EV do not get a max 3 months re-issue to be sure all data is correct or the company did not went bankrupt. But that might be something for another topic.

[Ryan Sleevi]

On Wednesday, April 19, 2017 at 3:13:36 PM UTC-4, Mike Pasarella wrote:
> To add some more concerning this issue:
>
> https://xenapp.alpinvest.com/

https://crt.sh/?id=42227446

localityName of Amsterdam
stateOrProvinceName of 19
countryName of NL

Problem has existed since 2013 - https://crt.sh/?id=6164627

> https://adoftheyear.com

https://crt.sh/?id=55977126

localityName of Rotterdam
stateOrProvinceName of California
countryName of NL

https://crt.sh/?id=5178826 goes back to at least 2014

Previous (good) cert from Comodo at https://crt.sh/?id=36863825

> https://secure.mobihealth.com

https://crt.sh/?id=38952224

localityName of Enschede
stateOrProvinceName of 15
countryName of NL

Strangely, this cert had issues from 2013 - 2014 (see https://crt.sh/?id=734399 https://crt.sh/?id=3495854 https://crt.sh/?id=5271322 ), briefly fixed the issue (see https://crt.sh/?id=9342945 https://crt.sh/?id=10983769 https://crt.sh/?id=12915701 https://crt.sh/?id=36254431 ) then went back to the issue.

It appears the distinction was DigiCert SHA2 Secure Server CA (which does the right thing) and DigiCert High Assurance CA-3 (which does the wrong thing)

> https://portal.mobilitymixx.nl

I'm not sure I understand enough to know what the issues are here. Could you explain?

> https://mijn.nfu.nl

https://crt.sh/?id=41866884

localityName of Utrecht
stateOrProvinceName of 03
countryName of NL

> https://portal.payplaza.com

https://crt.sh/?id=106229165

I'm not sure I understand the issues enough to know what's wrong here?

[Kurt Roeckx]

On Wed, Apr 19, 2017 at 12:28:16PM -0700, Ryan Sleevi via dev-security-policy wrote:
> > https://portal.mobilitymixx.nl
>
> I'm not sure I understand enough to know what the issues are here. Could you explain?

Both the localityName and stateOrProvinceName are Almere, while
the province is Flevoland.

What's also confusing is that the owner seems to have changed
from Mobility Mixx B.V. (NL) to Leaseplan Information Services
Limited (IE) and then back to Mobility Mixx B.V. (NL).

> > https://portal.payplaza.com
>
> https://crt.sh/?id=106229165
>
> I'm not sure I understand the issues enough to know what's wrong here?

It says "Noord-Holland" instead of "Noord-Brabant".


Kurt

[Mike vd Ent]

Ryan,

My answers on the particular issues are stated inline.
But the thing I want to address is how could (in this case Digicert) validate such data and issues certificates? I am investigation more of them and afraid even linked company names or registration numbers could be false. Shouldn't those certificates be revoked?

On Wednesday, 19 April 2017 21:28:26 UTC+2, Ryan Sleevi wrote:
> On Wednesday, April 19, 2017 at 3:13:36 PM UTC-4, Mike Pasarella wrote:
> > To add some more concerning this issue:
> >
> > https://xenapp.alpinvest.com/
>
> https://crt.sh/?id=42227446
>
> localityName of Amsterdam
> stateOrProvinceName of 19
> countryName of NL
>
> Problem has existed since 2013 - https://crt.sh/?id=6164627

But not solved? Shouldn't this certificate be revoked?

>
> > https://adoftheyear.com
>
> https://crt.sh/?id=55977126
>
> localityName of Rotterdam
> stateOrProvinceName of California
> countryName of NL

California is for sure not a province in The Netherlands.

>
> https://crt.sh/?id=5178826 goes back to at least 2014
>
> Previous (good) cert from Comodo at https://crt.sh/?id=36863825
>
> > https://secure.mobihealth.com
>
> https://crt.sh/?id=38952224
>
> localityName of Enschede
> stateOrProvinceName of 15
> countryName of NL
>
> Strangely, this cert had issues from 2013 - 2014 (see https://crt.sh/?id=734399 https://crt.sh/?id=3495854 https://crt.sh/?id=5271322 ), briefly fixed the issue (see https://crt.sh/?id=9342945 https://crt.sh/?id=10983769 https://crt.sh/?id=12915701 https://crt.sh/?id=36254431 ) then went back to the issue.
>
> It appears the distinction was DigiCert SHA2 Secure Server CA (which does the right thing) and DigiCert High Assurance CA-3 (which does the wrong thing)
>
> > https://portal.mobilitymixx.nl
>
> I'm not sure I understand enough to know what the issues are here. Could you explain?

Almere is a city (which is correct), but not the province.
https://en.wikipedia.org/wiki/Almere

>
> > https://mijn.nfu.nl
>
> https://crt.sh/?id=41866884
>
> localityName of Utrecht
> stateOrProvinceName of 03
> countryName of NL
>
> > https://portal.payplaza.com
>
> https://crt.sh/?id=106229165
>
> I'm not sure I understand the issues enough to know what's wrong here?

Eindhoven is not in the province Noord-Holland. Noord-Brabant (or Brabant) should be correct.

[Ryan Sleevi]

On Wed, Apr 19, 2017 at 3:47 PM, Mike vd Ent via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Ryan,
>
> My answers on the particular issues are stated inline.
> But the thing I want to address is how could (in this case Digicert)
> validate such data and issues certificates? I am investigation more of them
> and afraid even linked company names or registration numbers could be
> false. Shouldn't those certificates be revoked?
>

You are correct that it appears these certificates should not have issued.
Hopefully Jeremy and Ben from DigiCert can comment on this thread (
https://groups.google.com/d/msg/mozilla.dev.security.policy/DgeLqKMzIds/ig8UmHT2DwAJ
for the archive) with details about the issues and the steps taken.

[Jeremy Rowley]

I’m looking into it right now. I’ll report back shortly.



Jeremy



From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Wednesday, April 19, 2017 2:25 PM
To: Mike vd Ent <pasarel...@gmail.com>
Cc: mozilla-dev-security-policy <mozilla-dev-s...@lists.mozilla.org>; Jeremy Rowley <jeremy...@digicert.com>; Ben Wilson <ben.w...@digicert.com>
Subject: Re: CA Validation quality is failing

[Mike vd Ent]

I hope you could investigate it even further as this might be just the beginning.
I just did a random quick lookup so far. And I guess there are over a thousand or more Digicert certificates issued for Dutch websites and companies.

Does this mean the validation process is lacking proper validation or missing the tools and assets to know where to check for this information? For locations:

- Maps services from Google
- Wikipedia (although not favourable)
- Company registration agencies (kvk in The Netherlands), which already did the address check

[Peter Gutmann]

Kurt Roeckx via dev-security-policy <dev-secur...@lists.mozilla.org> writes:

>Both the localityName and stateOrProvinceName are Almere, while the province
>is Flevoland.

How much checking is a CA expected to do here? I know that OV and DV certs
are just "someone at this site responded to email" or whatever, but for an
EV cert how much further does the CA actually have to go? When e-Szignó
Hitelesítés-Szolgáltató in Hungary certifies Autolac Car Services, Av Los
Frutales 487 urb., Lima, Peru, are they expected to verify that it's really
in Av Los Frutales and not Los Tolladores, or do they just go ahead and
issue the cert? Can someone point to the bit of the BR that says that this
is obviously right or wrong?

Peter.

[Ryan Sleevi]

On Wed, Apr 19, 2017 at 6:41 PM, Peter Gutmann via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Kurt Roeckx via dev-security-policy <dev-secur...@lists.mozilla.org>
> writes:
>
> >Both the localityName and stateOrProvinceName are Almere, while the
> province
> >is Flevoland.
>
> How much checking is a CA expected to do here? I know that OV and DV certs
> are just "someone at this site responded to email" or whatever,

This is not correct. This can be easily answered by
https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.4.2.pdf

Section 3 governs validation, Section 7 governs the profile of how to use
that validated information

> but for an
> EV cert how much further does the CA actually have to go? When e-Szignó
> Hitelesítés-Szolgáltató in Hungary certifies Autolac Car Services, Av Los
> Frutales 487 urb., Lima, Peru, are they expected to verify that it's really
> in Av Los Frutales and not Los Tolladores, or do they just go ahead and
> issue the cert? Can someone point to the bit of the BR that says that this
> is obviously right or wrong?
>

For an EV cert, you look in
https://cabforum.org/wp-content/uploads/EV-V1_6_1.pdf

[Vincent Lynch]

Hi Peter,

EV requirements are actually dictated by a separate set of guidelines:
https://cabforum.org/extended-validation/

They do go into detail about how to verify applicant information. It covers
how you verify the company is legally established, where its physically
operating, etc. As you can imagine, its quite detailed. Here is a short
excerpt to give you an idea of the wording.

>From Section 11.4.1:

"... the CA MUST confirm that the Applicant's address, as listed in the EV
Certificate Request, is a valid business address for the Applicant or a
Parent/Subsidiary Company by reference to such QGIS, QIIS, or QTIS, and MAY
rely on the Applicant's representation that such address is its Place of
Business:"

(QGIS, QIIS, and QTIS are acronyms for different types of authoritative
sources, which the document also defines and details acceptable criteria
for)

-Vince

On Wed, Apr 19, 2017 at 6:41 PM, Peter Gutmann via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

> Kurt Roeckx via dev-security-policy <dev-secur...@lists.mozilla.org>
> writes:
>
> >Both the localityName and stateOrProvinceName are Almere, while the
> province
> >is Flevoland.
>
> How much checking is a CA expected to do here? I know that OV and DV certs

> are just "someone at this site responded to email" or whatever, but for an

> EV cert how much further does the CA actually have to go? When e-Szignó
> Hitelesítés-Szolgáltató in Hungary certifies Autolac Car Services, Av Los
> Frutales 487 urb., Lima, Peru, are they expected to verify that it's really
> in Av Los Frutales and not Los Tolladores, or do they just go ahead and
> issue the cert? Can someone point to the bit of the BR that says that this
> is obviously right or wrong?
>

> Peter.
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>



--
Vincent Lynch

[Kurt Roeckx]

On Wed, Apr 19, 2017 at 10:41:33PM +0000, Peter Gutmann via dev-security-policy wrote:
> Kurt Roeckx via dev-security-policy <dev-secur...@lists.mozilla.org> writes:
>
> >Both the localityName and stateOrProvinceName are Almere, while the province
> >is Flevoland.
>
> How much checking is a CA expected to do here? I know that OV and DV certs
> are just "someone at this site responded to email" or whatever, but for an
> EV cert how much further does the CA actually have to go?

For the EV cert we got we got a phone call asking if she could
speak to someone else to confirm that he works there.

That also wasn't what I expected. I expected that they would at
least check that he has the authority to do so, like asking the
CEO.

(It was a code sign certificate, but I expect if it's labeled EV
that the same things apply.)


Kurt

[Jeremy Rowley]

That was changed in ballot 127.

-----Original Message-----
From: dev-security-policy
[mailto:dev-security-policy-bounces+jeremy.rowley=digice...@lists.mozilla
.org] On Behalf Of Kurt Roeckx via dev-security-policy
Sent: Wednesday, April 19, 2017 5:54 PM
To: Peter Gutmann <pgu...@cs.auckland.ac.nz>
Cc: Ryan Sleevi <ry...@sleevi.com>;
mozilla-dev-s...@lists.mozilla.org
Subject: Re: CA Validation quality is failing

[Ryan Sleevi]

On Wed, Apr 19, 2017 at 7:53 PM, Kurt Roeckx via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> (It was a code sign certificate, but I expect if it's labeled EV
> that the same things apply.)
>

Not necessarily. A separate set of guidelines cover those -
https://cabforum.org/ev-code-signing-certificate-guidelines/

Neither Mozilla nor Google actively participate in the maintenance of those
documents.

[Kurt Roeckx]

On Wed, Apr 19, 2017 at 11:58:28PM +0000, Jeremy Rowley wrote:
> That was changed in ballot 127.

Which is adopted in july 2014. This was somewhere in 2016.

As I understood it, they didn't ask for the HR department, just
someone else. That might of course be a misunderstanding of what
was asked, which I guess is the reason for ballot 127.


Kurt

[Kurt Roeckx]

On Wed, Apr 19, 2017 at 09:00:22PM -0400, Ryan Sleevi wrote:
> On Wed, Apr 19, 2017 at 7:53 PM, Kurt Roeckx via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
> >
> > (It was a code sign certificate, but I expect if it's labeled EV
> > that the same things apply.)
> >
>
> Not necessarily. A separate set of guidelines cover those -
> https://cabforum.org/ev-code-signing-certificate-guidelines/

It really just points to the EV guidelines for the verification
requirments.


Kurt

[Jeremy Rowley]

FYI - still looking into this. I should have a report tomorrow.

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+jeremy.rowley=digice...@lists.mozilla.org] On Behalf Of Jeremy Rowley via dev-security-policy
Sent: Wednesday, April 19, 2017 2:27 PM
To: ry...@sleevi.com; Mike vd Ent <pasarel...@gmail.com>
Cc: Ben Wilson <ben.w...@digicert.com>; mozilla-dev-security-policy <mozilla-dev-s...@lists.mozilla.org>
Subject: RE: CA Validation quality is failing

I’m looking into it right now. I’ll report back shortly.



Jeremy



From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Wednesday, April 19, 2017 2:25 PM
To: Mike vd Ent <pasarel...@gmail.com>
Cc: mozilla-dev-security-policy <mozilla-dev-s...@lists.mozilla.org>; Jeremy Rowley <jeremy...@digicert.com>; Ben Wilson <ben.w...@digicert.com>

Subject: Re: CA Validation quality is failing

[Peter Gutmann]

Ryan Sleevi <ry...@sleevi.com> writes:

>For an EV cert, you look in https://cabforum.org/wp-content/uploads/EV-V1_6_1.pdf

It was meant as a rhetorical question, the OP asked whether doing XYZ in an
EV certificate was allowed and I was pointing out that the CAB Forum
guidelines should provide the answer. Vincent Lynch's reply was the appropriate
one, pointing out the text that covers this situation.

Peter.

[Jakob Bohm]

One thing:

Could this be a result of the common (among CAs) bug of requiring entry
of a US/Canada State/Province regardless of country, forcing applicants
to fill in random data in that field?

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Ryan Sleevi]

On Thu, Apr 20, 2017 at 6:42 AM Jakob Bohm via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:

>
> One thing:
>
> Could this be a result of the common (among CAs) bug of requiring entry
> of a US/Canada State/Province regardless of country, forcing applicants
> to fill in random data in that field?

That Is not common among CAs, because it's not how certificate information
is validated. Perhaps it would be best if you just waited for Jeremy to
respond, rather than attempting to speculate about the system. I appreciate
the eagerness to find answers, but those sorts of speculation don't really
help much.

[Jeremy Rowley]

Thanks Mike for bringing this up. We’ve looked into it and have an initial report to share.

After receiving the email on the Mozilla list, we investigated the identified certificates and discovered a couple of very related issues in our process that led to certificates with bad info:
1. As Jakob pointed out, part of the issue was that our intake form required a state if US was selected. As country is the last requested field, the state only became optional after the customer completed the rest of the form. This led to a lot of customers submitting bad data.
2. We have an old tool that generates information based on a customer’s location. This tool helps customers create CSRs and complete certificate requests. The tool inserts bad data into some fields if the field is left blank by the customer. The result was customers using this tool outside of the US had numbers included in the state field.
3. There was a gap in our validation process. This is the more important issue as the validation process should have caught the bad data inserted by the other two issues. Although we are obtaining validation documents from the appropriate government entity, the data submitted via the tool or intake form was not properly being updated with retrieved validated information. The end result was that the bad CSR data was submitted for signing instead of the data listed on the government document. This was a personnel problem, not a failure in our system as the validation staff was not appropriately updating the required signing fields.

So far, we have identified approximately 600 certificates that have the wrong state, zip, or locality. This is just a measurement of the problem scope and not the exact number as we are still reviewing are certificate database using a mapping API to determine whether the address exists. We expect the search to have a lot of false positives initially but provide a maximum scope of the problem. In parallel, we are contacting each customer with a non-compliant certificate, replacing the certificate, and revoking the certificate. All mis-matched certificates are being uploaded to the Google and DigiCert CT logs.

To fix our process, we are planning the following:
a. We are immediately deprecating our geolocation tool and updating our intake form to help reduce the amount of bad data.
b. We are updating our validation process to flag the differences in validation data and customer-provided data.
c. We are providing our validation staff with an extra mandatory tool that checks address information for accuracy. Part of our process going forward will be to use a separate source to verify that the city/state are actually in the country specified by the certificate.
4. Finally, we are implementing additional restrictions on our CA that prohibit signing of bad locality/state/zip information. We have a tool internally named “cert shield”. This tool is similar to CAB lint and that checks information submitted to the CA for compliance with the BRs and EV Guidelines. The rule set in cert shield is being updated to include additional restrictions designed to catch issues like numeric states and cities.

I’ll report back more on our progress next week with additional ideas. Please let me know if you have any questions.

Jeremy

[Jeremy Rowley]

Status Update:

We are still scanning our database to discover all certificates containing incorrect data. So far, the count is at 1510. The issues fall into two categories: 1) a failure to properly convey that BRs prohibit inclusion of meta data (BR 7.1.4.2.2.j) and 2) auto-population of data in the order that validation forgot to clear before issuing the certificate.

On the training issue, the validation team inserted characters like "-", "." or other similar information in approximately 1000 certificates. This information asserted that the field was not applicable. The remaining 500 included auto-population data. The auto-population took three formats (such as a number representing the country code) depending on the customer's location and access to our certificate management platform. Interestingly, the validation database contains the correct documentation. However, we failed to properly update the certificate information to match the validated data.

Since Mike reported the issue, we've patched our system to prevent meta-data and made substantial improvements to the validation process. These improvements help identify mis-matches between validation information and certificate data.

We also started the revocation process for the 500 certificates containing meta-data. However, we wanted to ask about the 1000 certificates containing data indicating the field was not applicable. We recognize these were not properly issued, but I am curious whether revocation is required by Mozilla in this case. Should we start revoking those certificates as well despite the certificate information clearly not indicating a state/province? My thought is yes because of BR 4.9.1.1:

9. The CA is made aware that the Certificate was not issued in accordance with these Requirements or the
CA’s Certificate Policy or Certification Practice Statement;

I don't think #10 applies because the information is accurate - the field is not applicable:
10. The CA determines that any of the information appearing in the Certificate is inaccurate or misleading;

Thoughts?

Jeremy

[Gervase Markham]

On 27/04/17 00:16, Jeremy Rowley wrote:
> We also started the revocation process for the 500 certificates
> containing meta-data. However, we wanted to ask about the 1000
> certificates containing data indicating the field was not applicable.
> We recognize these were not properly issued, but I am curious whether
> revocation is required by Mozilla in this case. Should we start
> revoking those certificates as well despite the certificate
> information clearly not indicating a state/province? My thought is
> yes because of BR 4.9.1.1:
>
> 9. The CA is made aware that the Certificate was not issued in
> accordance with these Requirements or the CA’s Certificate Policy or
> Certification Practice Statement;

What line in your CP or CPS is violated by these certs?

> I don't think #10 applies because the information is accurate - the
> field is not applicable: 10. The CA determines that any of the
> information appearing in the Certificate is inaccurate or
> misleading;

I agree that a "." or "-" instead of a field being empty is not
inaccurate or misleading. However, #10 also says "the CA determines", so
it's your view, not mine, which is most relevant :-)

Gerv

[Jeremy Rowley]

There isn't anything in our CPS directly. However, we state that we follow the baseline requirements in the CPS. The baseline requirements give a profile for the state field. We weren't sure this was strictly followed.

We finished our validation review over the weekend. There are about 3000 older certs with information indicating a field was not applicable (such as a "-", "N/A", etc). On top of this, we issued about 1000 certificates with mismatched validation information. The mismatched information can be divided into about 850 certificates with numbers in the state field. These numbers indicate a location code that was provided by the auto-populator. The remaining 150 are certificates with "Select", a state, or a postal code improperly included in the certificate. The root issue was a combination of the auto-populator inserting incorrect data into the cert request and our validation staff not properly updating the certificate information after completing the validation process.

Based on these results, we propose the following remediation plan:
1. We already removed the auto-populator from our CSR and certificate order generators.
2. We already blocked this information on the CA side from included in signed SSL/TLS objects.
3. We will revoke the 150 certificates with mismatched information.
4. We plan to let the remaining 3850 expire normally but will correct the certificate for all future orders (including rekeys).

Is this an acceptable plan? We are proposing not revoking the 3850 certificates because the information isn't misleading, the information is accurate, and there isn't a risk posed to Mozilla's users by inclusion of the numeric location code or not applicable indicator. Any thoughts?

Jeremy

-----Original Message-----
From: Gervase Markham [mailto:ge...@mozilla.org]
Sent: Thursday, April 27, 2017 2:41 AM
To: Jeremy Rowley <jeremy...@digicert.com>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: CA Validation quality is failing

On 27/04/17 00:16, Jeremy Rowley wrote:

> We also started the revocation process for the 500 certificates
> containing meta-data. However, we wanted to ask about the 1000
> certificates containing data indicating the field was not applicable.
> We recognize these were not properly issued, but I am curious whether
> revocation is required by Mozilla in this case. Should we start
> revoking those certificates as well despite the certificate
> information clearly not indicating a state/province? My thought is yes
> because of BR 4.9.1.1:
>
> 9. The CA is made aware that the Certificate was not issued in
> accordance with these Requirements or the CA’s Certificate Policy or
> Certification Practice Statement;

What line in your CP or CPS is violated by these certs?

> I don't think #10 applies because the information is accurate - the
> field is not applicable: 10. The CA determines that any of the
> information appearing in the Certificate is inaccurate or misleading;

[Ryan Sleevi]

(With a Google hat on)

Jeremy,

I think with the information you've shared so far, that sounds like a
reasonable plan from Google's perspective for the 3000 certificates. I
think there's at least a little concern about the EV nature for the 850
side, but just trying to understand more here what the implications would
be. Is this exclusively the state, or does it also extend to the
jurisdiction* fields? Or is this only for EV?

Would you be able to share a spreadsheet or details for those, in the
spirit of transparency? I think if you can share those details, it's
reasonable to avoid revoking, and anything specific that might represent a
security/compat risk to an Application Software Supplier (i.e. 4.9.1.1(15)
), we can look into separately.

Thank you for
1) Disclosing the details to a sufficient level of detail immediately
2) Providing regular updates and continued investigation
3) Confirming the acceptability of the plan before implementing it, and
with sufficient detail to understand the implications

These are several of the factors we weighed when considering the
implications/risk of not revoking.

[Jeremy Rowley]

Certainly happy to share more info. The search was for all EV and OV certs. Of the 4000 certificates detected, 101 certificates are EV. Of these EV certificates, 17 would be included in the proposed revocation. The rest have the “N/A” issue. This is just the locality/state/zip. The jurisdiction of incorporation processes separately and doesn’t have the same auto-populate tool.



More info:

78 certs had non-US states populated with US states (thanks to the auto-populator)

791 entities are affected by the entire range of certificates

257 entities are affected if we revoke the 1033 certs

82 entities are affected if we revoke just the 150 certs



What else would you like to know?

Jeremy



From: Ryan Sleevi [mailto:ry...@sleevi.com]
Sent: Monday, May 1, 2017 5:01 PM
To: Jeremy Rowley <jeremy...@digicert.com>
Cc: Gervase Markham <ge...@mozilla.org>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: CA Validation quality is failing

[Gervase Markham]

On 02/05/17 00:01, Ryan Sleevi wrote:
> Thank you for
> 1) Disclosing the details to a sufficient level of detail immediately
> 2) Providing regular updates and continued investigation
> 3) Confirming the acceptability of the plan before implementing it, and
> with sufficient detail to understand the implications

I echo Ryan's comments here. I'm happy with your remediation plan, and
think there's enough wiggle room in the BRs and Mozilla policy that
revocation of the certs with "N/A" etc. is avoidable.

I still think we need to address that 24-hour revocation requirement to
be a bit more nuanced, but that's a separate discussion :-)

Gerv

[Ryan Sleevi]

(Still wearing Google Hat in this context)

I think sharing a list (in CT) of the certs is good and can help verify the
assertions made here :)

But overall, I think this sounds right and the risk is minimal to our
users, so not revoking still sounds good :)

On Mon, May 1, 2017 at 11:53 PM, Jeremy Rowley <jeremy...@digicert.com>
wrote:

> Certainly happy to share more info. The search was for all EV and OV
> certs. Of the 4000 certificates detected, 101 certificates are EV. Of these
> EV certificates, 17 would be included in the proposed revocation. The rest
> have the “N/A” issue. This is just the locality/state/zip. The jurisdiction
> of incorporation processes separately and doesn’t have the same
> auto-populate tool.
>
>
>
> More info:
>
> 78 certs had non-US states populated with US states (thanks to the
> auto-populator)
>
> 791 entities are affected by the entire range of certificates
>
> 257 entities are affected if we revoke the 1033 certs
>
> 82 entities are affected if we revoke just the 150 certs
>
>
>
> What else would you like to know?
>
>
>
> Jeremy
>
>
>

> *From:* Ryan Sleevi [mailto:ry...@sleevi.com]
> *Sent:* Monday, May 1, 2017 5:01 PM
> *To:* Jeremy Rowley <jeremy...@digicert.com>
> *Cc:* Gervase Markham <ge...@mozilla.org>; mozilla-dev-security-policy@
> lists.mozilla.org
> *Subject:* Re: CA Validation quality is failing

>
>
>
>
>
>
>
> On Mon, May 1, 2017 at 3:41 PM, Jeremy Rowley via dev-security-policy <

> Thank you for
>
> 1) Disclosing the details to a sufficient level of detail immediately
>
> 2) Providing regular updates and continued investigation
>
> 3) Confirming the acceptability of the plan before implementing it, and
> with sufficient detail to understand the implications
>
>
>

[Alex Gaynor]

I know several CAs are using certlint (https://github.com/awslabs/certlint)
as a pre-issuance check that the cert they're about to issue doesn't have
any programmatically detectable deficiencies; if it doesn't already cover
some of these cases, it'd be great to add them as a technical means for
ensuring that this doesn't regress -- things like N/A should be an easy
enough check to add I'd think.

Cheers,
Alex

[Rob Stradling]

On 02/05/17 16:11, Alex Gaynor via dev-security-policy wrote:
> I know several CAs are using certlint (https://github.com/awslabs/certlint)
> as a pre-issuance check that the cert they're about to issue doesn't have
> any programmatically detectable deficiencies; if it doesn't already cover
> some of these cases, it'd be great to add them as a technical means for
> ensuring that this doesn't regress -- things like N/A should be an easy
> enough check to add I'd think.

Simple project idea (perhaps for https://github.com/cabforum):

A CSV file that contains 2 items per line:
1. An optional comma-separated list of Subject attribute shortnames.
2. A string that a CA should probably not encode as a complete Subject
attribute.

e.g.,
"OU,ST,L","N/A"
,"."
"O","Internet Widgits Pty Ltd"

Anyone (CA representatives, industry researchers, etc, etc) would be
able to submit PRs, CAs would be invited to consult this list when
evaluating certificate requests, and certlint would be able to report on
"violations".

--
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online

[Jeremy Rowley]

Thanks!

The revocation timeline changes are coming today/tomorrow morning.

-----Original Message-----
From: Gervase Markham [mailto:ge...@mozilla.org]

Sent: Tuesday, May 2, 2017 4:55 AM
To: ry...@sleevi.com; Jeremy Rowley <jeremy...@digicert.com>;
mozilla-dev-s...@lists.mozilla.org
Subject: Re: CA Validation quality is failing

On 02/05/17 00:01, Ryan Sleevi wrote:

> Thank you for
> 1) Disclosing the details to a sufficient level of detail immediately
> 2) Providing regular updates and continued investigation
> 3) Confirming the acceptability of the plan before implementing it,
> and with sufficient detail to understand the implications

[Jeremy Rowley]

Okay – we’ll add them all to CT over the next couple of days.

From: Ryan Sleevi [mailto:ry...@sleevi.com]

Sent: Tuesday, May 2, 2017 9:08 AM
To: Jeremy Rowley <jeremy...@digicert.com>

Cc: ry...@sleevi.com; Gervase Markham <ge...@mozilla.org>; mozilla-dev-s...@lists.mozilla.org
Subject: Re: CA Validation quality is failing

(Still wearing Google Hat in this context)



I think sharing a list (in CT) of the certs is good and can help verify the assertions made here :)



But overall, I think this sounds right and the risk is minimal to our users, so not revoking still sounds good :)

On Mon, May 1, 2017 at 11:53 PM, Jeremy Rowley <jeremy...@digicert.com <mailto:jeremy...@digicert.com> > wrote:

Certainly happy to share more info. The search was for all EV and OV certs. Of the 4000 certificates detected, 101 certificates are EV. Of these EV certificates, 17 would be included in the proposed revocation. The rest have the “N/A” issue. This is just the locality/state/zip. The jurisdiction of incorporation processes separately and doesn’t have the same auto-populate tool.



More info:

78 certs had non-US states populated with US states (thanks to the auto-populator)

791 entities are affected by the entire range of certificates

257 entities are affected if we revoke the 1033 certs

82 entities are affected if we revoke just the 150 certs



What else would you like to know?

Jeremy

From: Ryan Sleevi [mailto:ry...@sleevi.com <mailto:ry...@sleevi.com> ]
Sent: Monday, May 1, 2017 5:01 PM
To: Jeremy Rowley <jeremy...@digicert.com <mailto:jeremy...@digicert.com> >
Cc: Gervase Markham <ge...@mozilla.org <mailto:ge...@mozilla.org> >; mozilla-dev-s...@lists.mozilla.org <mailto:mozilla-dev-s...@lists.mozilla.org>
Subject: Re: CA Validation quality is failing

On Mon, May 1, 2017 at 3:41 PM, Jeremy Rowley via dev-security-policy <dev-secur...@lists.mozilla.org <mailto:dev-secur...@lists.mozilla.org> > wrote:

There isn't anything in our CPS directly. However, we state that we follow the baseline requirements in the CPS. The baseline requirements give a profile for the state field. We weren't sure this was strictly followed.

We finished our validation review over the weekend. There are about 3000 older certs with information indicating a field was not applicable (such as a "-", "N/A", etc). On top of this, we issued about 1000 certificates with mismatched validation information. The mismatched information can be divided into about 850 certificates with numbers in the state field. These numbers indicate a location code that was provided by the auto-populator. The remaining 150 are certificates with "Select", a state, or a postal code improperly included in the certificate. The root issue was a combination of the auto-populator inserting incorrect data into the cert request and our validation staff not properly updating the certificate information after completing the validation process.

Based on these results, we propose the following remediation plan:
1. We already removed the auto-populator from our CSR and certificate order generators.
2. We already blocked this information on the CA side from included in signed SSL/TLS objects.
3. We will revoke the 150 certificates with mismatched information.
4. We plan to let the remaining 3850 expire normally but will correct the certificate for all future orders (including rekeys).

Is this an acceptable plan? We are proposing not revoking the 3850 certificates because the information isn't misleading, the information is accurate, and there isn't a risk posed to Mozilla's users by inclusion of the numeric location code or not applicable indicator. Any thoughts?



(With a Google hat on)



Jeremy,



I think with the information you've shared so far, that sounds like a reasonable plan from Google's perspective for the 3000 certificates. I think there's at least a little concern about the EV nature for the 850 side, but just trying to understand more here what the implications would be. Is this exclusively the state, or does it also extend to the jurisdiction* fields? Or is this only for EV?



Would you be able to share a spreadsheet or details for those, in the spirit of transparency? I think if you can share those details, it's reasonable to avoid revoking, and anything specific that might represent a security/compat risk to an Application Software Supplier (i.e. 4.9.1.1(15) ), we can look into separately.

Thank you for

1) Disclosing the details to a sufficient level of detail immediately

2) Providing regular updates and continued investigation

3) Confirming the acceptability of the plan before implementing it, and with sufficient detail to understand the implications

[Jakob Bohm]

On 02/05/2017 17:30, Rob Stradling wrote:
> On 02/05/17 16:11, Alex Gaynor via dev-security-policy wrote:
>> I know several CAs are using certlint
>> (https://github.com/awslabs/certlint)
>> as a pre-issuance check that the cert they're about to issue doesn't have
>> any programmatically detectable deficiencies; if it doesn't already cover
>> some of these cases, it'd be great to add them as a technical means for
>> ensuring that this doesn't regress -- things like N/A should be an easy
>> enough check to add I'd think.
>
> Simple project idea (perhaps for https://github.com/cabforum):
>
> A CSV file that contains 2 items per line:
> 1. An optional comma-separated list of Subject attribute shortnames.
> 2. A string that a CA should probably not encode as a complete Subject
> attribute.
>
> e.g.,
> "OU,ST,L","N/A"
> ,"."
> "O","Internet Widgits Pty Ltd"
>
> Anyone (CA representatives, industry researchers, etc, etc) would be
> able to submit PRs, CAs would be invited to consult this list when
> evaluating certificate requests, and certlint would be able to report on
> "violations".
>

For simplicity and consistency with usual best development practices
("3rd normal form"), perhaps at most one attribute shortname in column
1.


e.g. Your example would be written as:

"OU","N/A"
"ST","N/A"

"L","N/A"
,"."
"O","Internet Widgits Pty Ltd"

>> ...

[Jeremy Rowley]

Okay - all certs were added to the CT log. We're now working through revocation.

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+jeremy.rowley=digice...@lists.mozilla.org] On Behalf Of Jeremy Rowley via dev-security-policy
Sent: Tuesday, May 2, 2017 12:09 PM
To: ry...@sleevi.com
Cc: mozilla-dev-s...@lists.mozilla.org; Gervase Markham <ge...@mozilla.org>
Subject: RE: CA Validation quality is failing

Okay – we’ll add them all to CT over the next couple of days.

From: Ryan Sleevi [mailto:ry...@sleevi.com]

Sent: Tuesday, May 2, 2017 9:08 AM
To: Jeremy Rowley <jeremy...@digicert.com>
Cc: ry...@sleevi.com; Gervase Markham <ge...@mozilla.org>; mozilla-dev-s...@lists.mozilla.org

Subject: Re: CA Validation quality is failing

(Still wearing Google Hat in this context)



I think sharing a list (in CT) of the certs is good and can help verify the assertions made here :)



But overall, I think this sounds right and the risk is minimal to our users, so not revoking still sounds good :)



On Mon, May 1, 2017 at 11:53 PM, Jeremy Rowley <jeremy...@digicert.com <mailto:jeremy...@digicert.com> > wrote:

Certainly happy to share more info. The search was for all EV and OV certs. Of the 4000 certificates detected, 101 certificates are EV. Of these EV certificates, 17 would be included in the proposed revocation. The rest have the “N/A” issue. This is just the locality/state/zip. The jurisdiction of incorporation processes separately and doesn’t have the same auto-populate tool.



More info:

78 certs had non-US states populated with US states (thanks to the auto-populator)

791 entities are affected by the entire range of certificates

257 entities are affected if we revoke the 1033 certs

82 entities are affected if we revoke just the 150 certs



What else would you like to know?

Jeremy

From: Ryan Sleevi [mailto:ry...@sleevi.com <mailto:ry...@sleevi.com> ]
Sent: Monday, May 1, 2017 5:01 PM
To: Jeremy Rowley <jeremy...@digicert.com <mailto:jeremy...@digicert.com> >
Cc: Gervase Markham <ge...@mozilla.org <mailto:ge...@mozilla.org> >; mozilla-dev-s...@lists.mozilla.org <mailto:mozilla-dev-s...@lists.mozilla.org>

Subject: Re: CA Validation quality is failing