Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Issuance with improper domain validation
444 views
Skip to first unread message
Jeremy Rowley
Aug 16, 2018, 2:17:16 PM8/16/18
to mozilla-dev-s...@lists.mozilla.org
I posted this to Bugzilla last night. Basically, we had an issue with
validation that resulted in some certs issuing without proper (post-Aug 1)
domain verification. Still working out how many. The major reason was lack
of training by the validation staff combined with a lack of strict document
controls in the early part of the Symantec-DigiCert transition.
1. How your CA first became aware of the problem (e.g. via a problem report
submitted to your Problem Reporting Mechanism, a discussion in
mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and
the time and date.
On 2018/08/07 at 17:00 UTC, a customer submitted a request for information
about our validation process for the verification of four of their domains.
Upon investigation, we found that the four domains were not properly
validated using a post-Aug 1 domain validation method. When attempting to
revalidated the domains prior to August 1, the random value was sent to an
address other than the WHOIS contact. This launched a broader investigation
into our overall revalidation efforts. This investigation is ongoing.
2. A timeline of the actions your CA took in response. A timeline is a
date-and-time-stamped sequence of all relevant events. This may include
events before the incident was reported, such as when a particular
requirement became applicable, or a document changed, or a bug was
introduced, or an audit was done.
>From approximately February through April 2018, DigiCert permitted some
legacy Symantec customers to use Method 1 to validate their domains. Use of
the method was subject to manager approval and reserved only for those
companies that had urgent replacement deadlines that could not be met with
an alternative validation method. Under this process, prior to approval, the
validation staff was required to match the WHOIS company information and
obtain approval using the WHOIS email address.
Around April, this process was modified to include a BR-compliant Random
Value that the validation staff sent using the WHOIS contact information.
Use of the random value indicated acceptance. Adding the random value
effectively transformed the validation from Method 1 to Method 2. The email
could include multiple domains with the understanding that the WHOIS contact
information had to match each domain listed.
We believe that in some cases either the validation staff failed to match
the WHOIS contact information for each domain listed, approving the
certificate solely based on the existing verified registrant info, or the
system did not check whether the WHOIS contact information matched the email
address used in the original confirmation.
On Aug 1, 2018, Ballot 218 took effect, deprecating Method 1.
On, August 7, 2018, a customer requested the audit trail of a certificate
issued using our new process. Upon review, validation management discovered
the validation was improper because the previously verified email contact
information did not match the WHOIS contact information. This discovery
created an escalation up to management.
On August 13, 2018, we stopped all issuance based on the process that
converted Method 1 validations to Method 2 validations.
We're currently investigating and will post an update when we know the
number of certificates and more about what went wrong. For now, we know the
number of impacted certificates is just under 2,500. We should have a
clearer picture shortly, after we have conducted a manual review of all
2,500 certificates.
3. Whether your CA has stopped, or has not yet stopped, issuing certificates
with the problem. A statement that you have will be considered a pledge to
the community; a statement that you have not requires an explanation.
On August 13, although most of these validations were likely properly
completed, we stopped issuance using information converted from Method 1
until completing a more thorough investigation.
4. A summary of the problematic certificates. For each problem: number of
certs, and the date the first and last certs with that problem were issued.
Approximately 2,500 certificates are under review for validation issues. We
wanted to get the incident report out quickly as an FYI while our
investigation continues. We'll update this section in a final report.
5. The complete certificate data for the problematic certificates. The
recommended way to provide this is to ensure each certificate is logged to
CT and then list the fingerprints or crt.sh IDs, either in the report or as
an attached spreadsheet, with one list per distinct problem.
Still under review. We will upload them to the bug once we have a complete
list. Because the error was human, we are reviewing each validation to
determine whether Method 2 was correctly used. Once we complete our review,
we'll post a Bugzilla attachment with the links for revoked certificates.
6. Explanation about how and why the mistakes were made or bugs introduced,
and how they avoided detection until now.
Preliminarily, we believe that the revalidation process that relied on
previously verified information failed because there were not technical
controls in place to ensure information integrity and because that the
validation staff received insufficient training on the process of uploading
WHOIS information., This resulted in validation staff adding some domains to
the approval email where the WHOIS contact information didn't always match
all the other domains listed.
When we performed revalidation of domains in preparation for Ballot 218's
effective date, we considered the existing domain information to be correct
and reusable until the normal expiration date. However, this inadvertently
blended poor quality validations due to the failure to check WHOIS contact
information with properly completed Method 2 validations, allowing certs to
issue post Aug 1 without proper domain validation.
7. List of steps your CA is taking to resolve the situation and ensure
such issuance will not be repeated in the future, accompanied with a
timeline of when your CA expects to accomplish these things.
We stopped all issuance relying on impacted domains until the domain control
is revalidated. We are also investigating each impacted domain's validation
to determine whether the email was sent to the appropriate WHOIS contact.
Any certs issued based on an improper validation will be revoked or
revalidated using an approved method.
We will update this issue and post to the Mozilla list when we have a
precise number of impacted certs.
We're still considering what additional action to take and will post an
update when we figure out more about what went wrong.
Jeremy
NEW
validation that resulted in some certs issuing without proper (post-Aug 1)
domain verification. Still working out how many. The major reason was lack
of training by the validation staff combined with a lack of strict document
controls in the early part of the Symantec-DigiCert transition.
1. How your CA first became aware of the problem (e.g. via a problem report
submitted to your Problem Reporting Mechanism, a discussion in
mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and
the time and date.
On 2018/08/07 at 17:00 UTC, a customer submitted a request for information
about our validation process for the verification of four of their domains.
Upon investigation, we found that the four domains were not properly
validated using a post-Aug 1 domain validation method. When attempting to
revalidated the domains prior to August 1, the random value was sent to an
address other than the WHOIS contact. This launched a broader investigation
into our overall revalidation efforts. This investigation is ongoing.
2. A timeline of the actions your CA took in response. A timeline is a
date-and-time-stamped sequence of all relevant events. This may include
events before the incident was reported, such as when a particular
requirement became applicable, or a document changed, or a bug was
introduced, or an audit was done.
>From approximately February through April 2018, DigiCert permitted some
legacy Symantec customers to use Method 1 to validate their domains. Use of
the method was subject to manager approval and reserved only for those
companies that had urgent replacement deadlines that could not be met with
an alternative validation method. Under this process, prior to approval, the
validation staff was required to match the WHOIS company information and
obtain approval using the WHOIS email address.
Around April, this process was modified to include a BR-compliant Random
Value that the validation staff sent using the WHOIS contact information.
Use of the random value indicated acceptance. Adding the random value
effectively transformed the validation from Method 1 to Method 2. The email
could include multiple domains with the understanding that the WHOIS contact
information had to match each domain listed.
We believe that in some cases either the validation staff failed to match
the WHOIS contact information for each domain listed, approving the
certificate solely based on the existing verified registrant info, or the
system did not check whether the WHOIS contact information matched the email
address used in the original confirmation.
On Aug 1, 2018, Ballot 218 took effect, deprecating Method 1.
On, August 7, 2018, a customer requested the audit trail of a certificate
issued using our new process. Upon review, validation management discovered
the validation was improper because the previously verified email contact
information did not match the WHOIS contact information. This discovery
created an escalation up to management.
On August 13, 2018, we stopped all issuance based on the process that
converted Method 1 validations to Method 2 validations.
We're currently investigating and will post an update when we know the
number of certificates and more about what went wrong. For now, we know the
number of impacted certificates is just under 2,500. We should have a
clearer picture shortly, after we have conducted a manual review of all
2,500 certificates.
3. Whether your CA has stopped, or has not yet stopped, issuing certificates
with the problem. A statement that you have will be considered a pledge to
the community; a statement that you have not requires an explanation.
On August 13, although most of these validations were likely properly
completed, we stopped issuance using information converted from Method 1
until completing a more thorough investigation.
4. A summary of the problematic certificates. For each problem: number of
certs, and the date the first and last certs with that problem were issued.
Approximately 2,500 certificates are under review for validation issues. We
wanted to get the incident report out quickly as an FYI while our
investigation continues. We'll update this section in a final report.
5. The complete certificate data for the problematic certificates. The
recommended way to provide this is to ensure each certificate is logged to
CT and then list the fingerprints or crt.sh IDs, either in the report or as
an attached spreadsheet, with one list per distinct problem.
Still under review. We will upload them to the bug once we have a complete
list. Because the error was human, we are reviewing each validation to
determine whether Method 2 was correctly used. Once we complete our review,
we'll post a Bugzilla attachment with the links for revoked certificates.
6. Explanation about how and why the mistakes were made or bugs introduced,
and how they avoided detection until now.
Preliminarily, we believe that the revalidation process that relied on
previously verified information failed because there were not technical
controls in place to ensure information integrity and because that the
validation staff received insufficient training on the process of uploading
WHOIS information., This resulted in validation staff adding some domains to
the approval email where the WHOIS contact information didn't always match
all the other domains listed.
When we performed revalidation of domains in preparation for Ballot 218's
effective date, we considered the existing domain information to be correct
and reusable until the normal expiration date. However, this inadvertently
blended poor quality validations due to the failure to check WHOIS contact
information with properly completed Method 2 validations, allowing certs to
issue post Aug 1 without proper domain validation.
7. List of steps your CA is taking to resolve the situation and ensure
such issuance will not be repeated in the future, accompanied with a
timeline of when your CA expects to accomplish these things.
We stopped all issuance relying on impacted domains until the domain control
is revalidated. We are also investigating each impacted domain's validation
to determine whether the email was sent to the appropriate WHOIS contact.
Any certs issued based on an improper validation will be revoked or
revalidated using an approved method.
We will update this issue and post to the Mozilla list when we have a
precise number of impacted certs.
We're still considering what additional action to take and will post an
update when we figure out more about what went wrong.
Jeremy
NEW
0 new messages