Incidents involving the CA WoSign

22830 views
Skip to first unread message

Gervase Markham

unread,
Aug 24, 2016, 9:09:02 AM8/24/16
to mozilla-dev-s...@lists.mozilla.org, Richard Wang
Dear m.d.s.policy,

Several incidents have come to our attention involving the CA "WoSign".
Mozilla is considering what action it should take in response to these
incidents. This email sets out our understanding of the situation.

Before we begin, we note that Section 1 of the Mozilla CA Certificate
Enforcement Policy[0] says: "When a serious security concern is noticed,
such as a major root compromise, it should be treated as a
security-sensitive bug, and the Mozilla Policy for Handling Security
Bugs should be followed." It is clear to us, and appears to be clear to
other CAs based on their actions, that misissuances where domain control
checks have failed fall into the category of "serious security concern".

Incident 0
----------

On or around April 23rd, 2015, WoSign's certificate issuance system for
their free certificates allowed the applicant to choose any port for
validation. Once validation had been completed, WoSign would
issue certificates for that domain. A researcher was able to obtain a
certificate for a university by opening a high-numbered port (>50,000)
and getting WoSign to use that port for validation of control.

This problem was reported to Google, and thence to WoSign and resolved.
Mozilla only became aware of it recently.

* Before the recent passage of Ballot 169 in the CAB Forum, which limits
the ports and paths which can be used, the Baseline Requirements said
that one acceptable method of domain validation was "Having the
Applicant demonstrate practical control over the FQDN by making an
agreed‐upon change to information found on an online Web page identified
by a uniform resource identifier containing the FQDN". This method
therefore did not violate the letter of the BRs. However, Mozilla
considers the basic security knowledge that ports over 1024 are
unprivileged should have led all CAs not to accept validations of domain
control on such ports, even when not documented in the BRs.

* The misissuance incident was not reported to Mozilla by WoSign as it
should have been (see above).

* This misissuance incident did not turn up on WoSign's subsequent BR
audit[1].

Incident 1
----------

In June 2015, an applicant found a problem with WoSign's free
certificate service, which allowed them to get a certificate for the
base domain if they were able to prove control of a subdomain.

The reporter proved the problem in two ways. They accidentally
discovered it when trying to get a certificate for med.ucf.edu and
mistakenly also applied for www.ucf.edu, which was approved. They then
confirmed the problem by using their control of
theiraccount.github.com/theiraccount.github.io to get a cert for
github.com, github.io, and www.github.io.

They reported this to WoSign, giving only the Github certificate as an
example. That cert was revoked and the vulnerability was fixed. However
recently, they got in touch with Google to note that the ucf.edu cert
still had not been revoked almost a year later.

* The lack of revocation of the ucf.edu certificate (still unrevoked at
time of writing, although it may have been by time of posting) strongly
suggests that WoSign either did not or could not search their issuance
databases for other occurrences of the same problem. Mozilla considers
such a search a basic part of the response to disclosure of a
vulnerability which causes misissuance, and expects CAs to keep records
detailed enough to make it possible.

* This misissuance incident was not reported to Mozilla by WoSign as it
should have been (see above).

* This misissuance incident did not turn up on WoSign's subsequent BR
audit[1].

Incident 2
----------

In July 2016, it became clear that there was some problems with the
StartEncrypt automatic issuance service recently deployed by the CA
StartCom. As well as other problems it had, which are outside the scope
of this discussion, changing a simple API parameter in the POST request
on the submission page changed the root certificate to which the
resulting certificate chained up. The value "2" made a certificate
signed by "StartCom Class 1 DV Server CA", "1" selected "WoSign CA Free
SSL Certificate G2" and "0" selected "CA 沃通根证书", another root
certificate owned by WoSign and trusted by Firefox.

Using the value "1" led to a certificate which had a notBefore date
(usage start date) of 20th December 2015, and which was signed using the
SHA-1 checksum algorithm.

* The issuance of certificates using SHA-1 has been banned by the
Baseline Requirements since January 1st, 2016. Browsers, including
Firefox, planned to enforce this[2] by not trusting certs with a
notBefore date after that date, but in the case of Firefox the fix had
to be backed out due to web compatibility issues. However, we are
considering how/when to reintroduce it, and CAs presumably know this.

* The issuance of backdated certificates is not forbidden, but is listed
in Mozilla's list of Problematic Practices[3]. It says "Minor tweaking
for technical compatibility reasons is accepted, but backdating
certificates in order to avoid some deadline or code-enforced
restriction is not."

* WoSign deny that their code backdated the certificates in order to
avoid browser-based restrictions - they say "this date is the day we
stop to use this code"[4]. If that is true, it is not clear to us how
StartCom came to deploy WoSign code that WoSign itself had abandoned.

* It seems clear from publicly available information that StartCom's
issuance systems are linked to WoSign's issuance systems in some way.
Nevertheless, it should not have been possible for an application for a
cert from StartCom to produce a cert signed by WoSign.

* This misissuance incident was not reported to Mozilla by WoSign as it
should have been.


Taking into account all these incidents and the actions of this CA,
Mozilla is considering what action to take. Your input is welcomed.

Gerv, Kathleen and Richard


[0]
https://www.mozilla.org/en-US/about/governance/policies/security-group/certs/policy/enforcement/
[1] https://cert.webtrust.org/SealFile?seal=2019&file=pdf
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=942515
[3]
https://wiki.mozilla.org/CA:Problematic_Practices#Backdating_the_notBefore_date
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=1293366

Eric Mill

unread,
Aug 24, 2016, 12:09:39 PM8/24/16
to Gervase Markham, Richard Wang, mozilla-dev-s...@lists.mozilla.org
For the thread's reference, here's the crt.sh link for the misissued GitHub
certificate:

https://crt.sh/?id=29647048

Valid for 3 years, for github.com. It's not in OneCRL, CRLset, or
Microsoft's disallowedcert.stl.
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>



--
konklone.com | @konklone <https://twitter.com/konklone>

Gervase Markham

unread,
Aug 24, 2016, 12:31:26 PM8/24/16
to Jeremy Rowley, Richard Wang
Hi Jeremy,

On 24/08/16 17:12, Jeremy Rowley wrote:
> On incident 0, its unclear whether a cert was actually mis-issued.
> Although they used a higher level port, did the researcher
> successfully bypass WoSign's domain validation process? Is the only
> concern that WoSign permitted higher level ports?

The result of the incident was that a certificate was issued to someone
who did not, in the normally understood sense of the word, have control
of the domain in question. Mozilla feels that even without a specific
injunction in the BRs, CAs should have known that ports > 1024 are not
privileged and not done control checks using them.

The severity of the problem, of course, is a matter for discussion here.

> On incident 2, it sounds like they are both using the same
> auto-generation script.

It seems like a bit more than that, doesn't it? Let's presume that
WoSign did not ship a copy of their intermediate cert's private key to
StartCom. Therefore, this cert must have been issued on the back end by
some sort of WoSign system. So either WoSign's back-end issuing service
has some form of authentication and the StartCom system had those
credentials (why?), or the WoSign system does not have any form of
authentication (concerning).

> Giving WoSign the benefit of the doubt, it
> sounds like maybe it was a bug in their software that permitted SHA1
> certs not an intentional back-dating issue. Is there any clarity
> around how this worked?

Perhaps WoSign would like to provide some :-)

Gerv

Peter Bowen

unread,
Aug 24, 2016, 12:44:44 PM8/24/16
to Gervase Markham, Richard Wang, mozilla-dev-s...@lists.mozilla.org, Jeremy Rowley
On Wed, Aug 24, 2016 at 9:30 AM, Gervase Markham <ge...@mozilla.org> wrote:
> On 24/08/16 17:12, Jeremy Rowley wrote:
>> On incident 2, it sounds like they are both using the same
>> auto-generation script.
>
> It seems like a bit more than that, doesn't it? Let's presume that
> WoSign did not ship a copy of their intermediate cert's private key to
> StartCom. Therefore, this cert must have been issued on the back end by
> some sort of WoSign system. So either WoSign's back-end issuing service
> has some form of authentication and the StartCom system had those
> credentials (why?), or the WoSign system does not have any form of
> authentication (concerning).

I think you are missing the most likely option: CA hosting. My
understanding is that it is not uncommon that one CA operator
contracts with another CA operator to run a CA on behalf of the first
operator. I don't think it has been clear what disclosure of this
practice is required. Given that I believe this is widespread, I
assumed that all of the issuing CAs in this case were operated by the
same entity.

Thanks,
Peter

Ryan Sleevi

unread,
Aug 24, 2016, 4:19:41 PM8/24/16
to Jeremy Rowley, Richard Wang, mozilla-dev-s...@lists.mozilla.org, Gervase Markham, Peter Bowen
On Wed, Aug 24, 2016 at 12:40 PM, Jeremy Rowley
<jeremy...@digicert.com> wrote:
> However, the fact a researcher was able to obtain a cert without proper domain
> validation is pretty serious. I'd like to hear more details about how this was
> accomplished. Ports 8080 and 8443 aren't that uncommon so penalizing someone
> merely for port use seems harsh when there wasn't a policy against it.

There was no restriction on ports at all. Any client-specified port
was accepted, and any HTTP-like response it gave back was accepted.

s...@gmx.ch

unread,
Aug 24, 2016, 8:22:59 PM8/24/16
to dev-secur...@lists.mozilla.org
Of course, adding the affected certs to OneCRL should be done immediately.

WoSign also has to be transparent about all (mis) issued certs in the
past and have to provide this info in the future.
If they can't, I think we may consider if the current certs that are
valid for 3 years should be restricted to a shorter period.

Regards,
Jonas
signature.asc

Richard Wang

unread,
Aug 24, 2016, 10:58:06 PM8/24/16
to Eric Mill, Gervase Markham, mozilla-dev-s...@lists.mozilla.org
this cert is revoked in the same once it is issued.
Thanks for posting to CT.


Best Regards,

Richard

From: Eric Mill [mailto:er...@konklone.com]
Sent: Thursday, August 25, 2016 12:08 AM
To: Gervase Markham <ge...@mozilla.org>
Cc: mozilla-dev-s...@lists.mozilla.org; Richard Wang <ric...@wosign.com>
Subject: Re: Incidents involving the CA WoSign

For the thread's reference, here's the crt.sh link for the misissued GitHub certificate:

https://crt.sh/?id=29647048

Valid for 3 years, for github.com<http://github.com>. It's not in OneCRL, CRLset, or Microsoft's disallowedcert.stl.
discovered it when trying to get a certificate for med.ucf.edu<http://med.ucf.edu> and
mistakenly also applied for www.ucf.edu<http://www.ucf.edu>, which was approved. They then
confirmed the problem by using their control of
theiraccount.github.com/theiraccount.github.io<http://theiraccount.github.com/theiraccount.github.io> to get a cert for
github.com<http://github.com>, github.io<http://github.io>, and www.github.io<http://www.github.io>.

They reported this to WoSign, giving only the Github certificate as an
example. That cert was revoked and the vulnerability was fixed. However
recently, they got in touch with Google to note that the ucf.edu<http://ucf.edu> cert
still had not been revoked almost a year later.

* The lack of revocation of the ucf.edu<http://ucf.edu> certificate (still unrevoked at
Taking into account all these incidents and the actions of this CA,
Mozilla is considering what action to take. Your input is welcomed.

dev-secur...@lists.mozilla.org<mailto:dev-secur...@lists.mozilla.org>
https://lists.mozilla.org/listinfo/dev-security-policy



--
konklone.com<https://konklone.com> | @konklone<https://twitter.com/konklone>

Richard Wang

unread,
Aug 24, 2016, 11:40:53 PM8/24/16
to Gervase Markham, Jeremy Rowley, mozilla-dev-s...@lists.mozilla.org
See below inline, thanks.





Best Regards,



Richard



-----Original Message-----

From: Gervase Markham [mailto:ge...@mozilla.org]

Sent: Thursday, August 25, 2016 12:31 AM

To: Jeremy Rowley <jeremy...@digicert.com>; mozilla-dev-s...@lists.mozilla.org

Cc: Richard Wang <ric...@wosign.com>

Subject: Re: Incidents involving the CA WoSign



Hi Jeremy,



On 24/08/16 17:12, Jeremy Rowley wrote:

> On incident 0, its unclear whether a cert was actually mis-issued.

> Although they used a higher level port, did the researcher

> successfully bypass WoSign's domain validation process? Is the only

> concern that WoSign permitted higher level ports?



The result of the incident was that a certificate was issued to someone who did not, in the normally understood sense of the word, have control of the domain in question. Mozilla feels that even without a specific injunction in the BRs, CAs should have known that ports > 1024 are not privileged and not done control checks using them.



The severity of the problem, of course, is a matter for discussion here.



R: We got report from Google, but don’t tell me if any mis-issued certificate happen, just tell us to forbidden this high level port.

We are searching our database to try to find if any mis-issued cert is issued. And will update to this mail list.





> On incident 2, it sounds like they are both using the same

> auto-generation script.



It seems like a bit more than that, doesn't it? Let's presume that WoSign did not ship a copy of their intermediate cert's private key to StartCom. Therefore, this cert must have been issued on the back end by some sort of WoSign system. So either WoSign's back-end issuing service has some form of authentication and the StartCom system had those credentials (why?), or the WoSign system does not have any form of authentication (concerning).



R: NOT this case you think. Due to root inclusion problem, WoSign root is cross signed by StartCom since 2011. And we shared some facility with StartCom like CRL and OCSP distribution etc. But not this case, as I declared in the previous email, this is a API parameter option that can post data to any server located in any place.





> Giving WoSign the benefit of the doubt, it sounds like maybe it was a

> bug in their software that permitted SHA1 certs not an intentional

> back-dating issue. Is there any clarity around how this worked?



Perhaps WoSign would like to provide some :-)



R: I think we said clearly in the related Bugzilla ☺





Gerv



Richard Wang

unread,
Aug 24, 2016, 11:42:44 PM8/24/16
to Peter Bowen, Gervase Markham, mozilla-dev-s...@lists.mozilla.org, Jeremy Rowley
Yes, correct.

Due to root inclusion problem, WoSign root is cross signed by StartCom since 2011. And we shared some facility with StartCom like CRL and OCSP distribution etc. But not this case, as I declared in the previous email, this is a API parameter option that can post data to any server located in any place.

Best Regards,

Richard

-----Original Message-----
From: Peter Bowen [mailto:pzb...@gmail.com]
Sent: Thursday, August 25, 2016 12:45 AM
To: Gervase Markham <ge...@mozilla.org>
Cc: Jeremy Rowley <jeremy...@digicert.com>; mozilla-dev-s...@lists.mozilla.org; Richard Wang <ric...@wosign.com>
Subject: Re: Incidents involving the CA WoSign

On Wed, Aug 24, 2016 at 9:30 AM, Gervase Markham <ge...@mozilla.org> wrote:
> On 24/08/16 17:12, Jeremy Rowley wrote:
>> On incident 2, it sounds like they are both using the same
>> auto-generation script.
>
> It seems like a bit more than that, doesn't it? Let's presume that
> WoSign did not ship a copy of their intermediate cert's private key to
> StartCom. Therefore, this cert must have been issued on the back end
> by some sort of WoSign system. So either WoSign's back-end issuing
> service has some form of authentication and the StartCom system had
> those credentials (why?), or the WoSign system does not have any form
> of authentication (concerning).

Richard Wang

unread,
Aug 25, 2016, 12:05:32 AM8/25/16
to s...@gmx.ch, dev-secur...@lists.mozilla.org
We revoked this certificate, and we know this certificate is for test only.

For transparency, WoSign announced full transparency for all SSL certificate from July 5th that post all issued SSL certificate to Google log server, browsers can distrust WoSign issued SSL certificate after that day if no SCT embedded data in the certificate.

And WoSign even plan to post the code signing certificate and client certificate to log server for full transparency for all certificates.

See this news if you missed: https://www.wosign.com/english/News/2016_wosign_CT.htm.

And we plan to setup an free alert service for worldwide users that if any SSL certificate for domain you care is issued from any CA, then you can get the alert email, this will benfit the ecosystem.


Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of s...@gmx.ch
Sent: Thursday, August 25, 2016 8:18 AM
To: dev-secur...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

Of course, adding the affected certs to OneCRL should be done immediately.

WoSign also has to be transparent about all (mis) issued certs in the past and have to provide this info in the future.
If they can't, I think we may consider if the current certs that are valid for 3 years should be restricted to a shorter period.

Regards,
Jonas


> For the thread's reference, here's the crt.sh link for the misissued
> GitHub
> certificate:
>
> https://crt.sh/?id=29647048
>
> Valid for 3 years, for github.com. It's not in OneCRL, CRLset, or

Matt Palmer

unread,
Aug 25, 2016, 2:48:30 AM8/25/16
to dev-secur...@lists.mozilla.org
On Thu, Aug 25, 2016 at 04:03:04AM +0000, Richard Wang wrote:
> For transparency, WoSign announced full transparency for all SSL
> certificate from July 5th that post all issued SSL certificate to Google
> log server, browsers can distrust WoSign issued SSL certificate after that
> day if no SCT embedded data in the certificate.

That would be slightly more reassuring if there wasn't a history of certs
being issued with seemingly misleading notBefore values...

Separately, do you have any thoughts on the reports that WoSign's BR auditor
did not note any of the misissuances? Also, what changes, exactly, has
WoSign implemented to its policies and procedures to ensure that all trust
programs in which WoSign is a participant are notified of future incidents,
in line with each program's requirements?

- Matt

--
"The user-friendly computer is a red herring. The user-friendliness of a
book just makes it easier to turn pages. There's nothing user-friendly about
learning to read."
-- Alan Kay

Richard Wang

unread,
Aug 25, 2016, 3:14:10 AM8/25/16
to Matt Palmer, dev-secur...@lists.mozilla.org
Thanks for your friendly reminder.

We can post all 2015 issued SSL certificate to CT log server if necessary.

For BR auditor, I think this issue is too technical that fewer auditor can find out this problem.

We will add the quality control system to PKI system before issuing the certificate, and will check the crt.sh or use the CABF lint and X590 Lint to check the certificate before and after the certificate is issued to prevent such case, if such case happen, we will notify all browsers instantly.


Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Matt Palmer
Sent: Thursday, August 25, 2016 2:48 PM
To: dev-secur...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Ryan Sleevi

unread,
Aug 25, 2016, 3:09:59 PM8/25/16
to mozilla-dev-s...@lists.mozilla.org
On Thursday, August 25, 2016 at 12:14:10 AM UTC-7, Richard Wang wrote:
> We can post all 2015 issued SSL certificate to CT log server if necessary.

Is there any reason not to do that proactively?

> For BR auditor, I think this issue is too technical that fewer auditor can find out this problem.

The audit letter included an attestation from Management that, during the time of the audit, management believed that the CA complied with the Baseline Requirements.

Management was aware of non-compliance, by virtue of revocation and system and procedural changes to align with compliance.

Thus, do you believe it was faithful and accurate for Management to warrant that the CA was operated in compliance with the BRs, given that Management was aware of incidents of non-compliance?

Matt Palmer

unread,
Aug 25, 2016, 7:35:39 PM8/25/16
to dev-secur...@lists.mozilla.org
On Thu, Aug 25, 2016 at 07:11:18AM +0000, Richard Wang wrote:
> We can post all 2015 issued SSL certificate to CT log server if necessary.

That doesn't provide any assurance, in the face of misleading notBefore
values in certificates. Without strong assurances that whatever failure of
systems or processes allowed misleading notBefore values to end up in
certificates has been corrected, the only way to be sure that there are no
shenanigans going on is for *all* certificates issued by WoSign,
*regardless* of their purported issuance date, to carry SCTs from a
qualified log (or have them presented at TLS establishment), and have the
"effective date" for the purposes of compliance with relevant standards and
guidelines to be the date of the earliest SCT presented. This is because
the notBefore value in the certificate, being produced by a flawed process
which allows misleading values to be used, is useless for the purposes of
determining when the certificate was *actually* issued. Any TLS connection
using a WoSign-issued certificate which does not present SCTs would have to
be considered completely untrustworthy.

> For BR auditor, I think this issue is too technical that fewer auditor can
> find out this problem.

I believe Ryan has done an excellent job of outlining the concerns with this
statement.

> We will add the quality control system to PKI system before issuing the
> certificate, and will check the crt.sh or use the CABF lint and X590 Lint
> to check the certificate before and after the certificate is issued to
> prevent such case, if such case happen, we will notify all browsers
> instantly.

This neither addresses the question I posed, nor does it contain sufficient
specificity to be reassuring. I asked:

> what changes, exactly, has WoSign implemented to its policies and
> procedures to ensure that all trust programs in which WoSign is a
> participant are notified of future incidents, in line with each program's
> requirements?

I'm after the specifics of the changes to WoSign's policies and procedures
regarding *notification*, not quality control. What were WoSign's previous
policies and procedures regarding notification (obviously there was
something in place, since Google was notified), and what changes have been
made to improve those policies to ensure that all root programs are notified
in line with each program's requirements in the future?

- Matt

--
I told [my daughter] that if I see her digging a hole that she might not be
able to crawl out of, my job isn't to stand back and say "That's a *real*
nice hole you're digging there".
-- Paul Tomblin, ASR

Ryan Sleevi

unread,
Aug 25, 2016, 8:16:07 PM8/25/16
to mozilla-dev-s...@lists.mozilla.org
On Thursday, August 25, 2016 at 4:35:39 PM UTC-7, Matt Palmer wrote:
> I'm after the specifics of the changes to WoSign's policies and procedures
> regarding *notification*, not quality control. What were WoSign's previous
> policies and procedures regarding notification (obviously there was
> something in place, since Google was notified), and what changes have been
> made to improve those policies to ensure that all root programs are notified
> in line with each program's requirements in the future?

Clarification: In none of these incidents was Google notified proactively by WoSign. Instead, Google received communication from internal or external researchers regarding these issues, either prior to resolution or much later after the fact, and subsequently contacted WoSign regarding them.

It was only when Google found out recently that other programs were NOT notified, proactively, as had been expected, that Google shared the details it was aware of regarding various CA incidents, including those of WoSign, mentioned in this thread.

Matt Palmer

unread,
Aug 25, 2016, 10:03:50 PM8/25/16
to dev-secur...@lists.mozilla.org
On Thu, Aug 25, 2016 at 05:15:58PM -0700, Ryan Sleevi wrote:
> On Thursday, August 25, 2016 at 4:35:39 PM UTC-7, Matt Palmer wrote:
> > I'm after the specifics of the changes to WoSign's policies and procedures
> > regarding *notification*, not quality control. What were WoSign's previous
> > policies and procedures regarding notification (obviously there was
> > something in place, since Google was notified), and what changes have been
> > made to improve those policies to ensure that all root programs are notified
> > in line with each program's requirements in the future?
>
> Clarification: In none of these incidents was Google notified proactively
> by WoSign. Instead, Google received communication from internal or
> external researchers regarding these issues, either prior to resolution or
> much later after the fact, and subsequently contacted WoSign regarding
> them.

Oh, wow. I totally misread the initial report. That's almost certainly
much *worse*, then, because it's not simply a matter of some adjustments to
an existing process to improve it, but likely the development and deployment
of an entirely new process.

- Matt

--
The main advantages of Haynes and Chilton manuals are that they cost $15,
where the factory manuals cost $100 and up, and that they will tell you how
to use two hammers, a block of wood, and a meerkat to replace "special tool
no. 2-112-A" -- Matt Roberds in asr.

Richard Wang

unread,
Aug 25, 2016, 11:05:46 PM8/25/16
to Ryan Sleevi, mozilla-dev-s...@lists.mozilla.org
See below inline, thanks.

Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Ryan Sleevi
Sent: Friday, August 26, 2016 3:10 AM
To: mozilla-dev-s...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

On Thursday, August 25, 2016 at 12:14:10 AM UTC-7, Richard Wang wrote:
> We can post all 2015 issued SSL certificate to CT log server if necessary.

Is there any reason not to do that proactively?

R: OK, we will post all 2015 issued SSL certificates to CT log server, but this take time since we issued 115K SSL certificate in 2015.
Now we are posting the (1) using higher level port validated orders related to incident 0, total 72 certificates. To be clear, those certificates are validated by website control validation method that using other port except 80 and 443; (2) Mis-issued certificate with un-validated subdomain related to incident 1, total 33 certificates. I will list all crt.sh URL to this mail thread.
Some certificates are revoked after getting report from subscriber, but some still valid, if any subscriber think it must be revoked and replaced new one, please contact us in the system, thanks.


> For BR auditor, I think this issue is too technical that fewer auditor can find out this problem.

The audit letter included an attestation from Management that, during the time of the audit, management believed that the CA complied with the Baseline Requirements.

Management was aware of non-compliance, by virtue of revocation and system and procedural changes to align with compliance.

Thus, do you believe it was faithful and accurate for Management to warrant that the CA was operated in compliance with the BRs, given that Management was aware of incidents of non-compliance?

R: This is my fault that I think it is not serious enough to state in the assertion letter, now I know and every related employee know how to do this in the future.

Richard Wang

unread,
Aug 25, 2016, 11:32:50 PM8/25/16
to Matt Palmer, dev-secur...@lists.mozilla.org
See below inline, thanks


Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Matt Palmer
Sent: Friday, August 26, 2016 7:35 AM
To: dev-secur...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

On Thu, Aug 25, 2016 at 07:11:18AM +0000, Richard Wang wrote:
> We can post all 2015 issued SSL certificate to CT log server if necessary.

That doesn't provide any assurance, in the face of misleading notBefore values in certificates. Without strong assurances that whatever failure of systems or processes allowed misleading notBefore values to end up in certificates has been corrected, the only way to be sure that there are no shenanigans going on is for *all* certificates issued by WoSign,
*regardless* of their purported issuance date, to carry SCTs from a qualified log (or have them presented at TLS establishment), and have the "effective date" for the purposes of compliance with relevant standards and guidelines to be the date of the earliest SCT presented. This is because the notBefore value in the certificate, being produced by a flawed process which allows misleading values to be used, is useless for the purposes of determining when the certificate was *actually* issued. Any TLS connection using a WoSign-issued certificate which does not present SCTs would have to be considered completely untrustworthy.

R: As I declared in the Bugzilla, only two certificate is the backdated certificate that not issued by normal system, it is a API bug that found and used by Computest to get the two backdates certificate, not WoSign normally issued this two cert. And this two test certificate are revoked after we got report, we stopped this API and delete the bug code.

As we stated in WoSign website, browsers can distrust WoSign issued certificate after July 5th that no SCT embedded data, this is for full transparency.


> For BR auditor, I think this issue is too technical that fewer auditor
> can find out this problem.

I believe Ryan has done an excellent job of outlining the concerns with this statement.

> We will add the quality control system to PKI system before issuing
> the certificate, and will check the crt.sh or use the CABF lint and
> X590 Lint to check the certificate before and after the certificate is
> issued to prevent such case, if such case happen, we will notify all
> browsers instantly.

This neither addresses the question I posed, nor does it contain sufficient specificity to be reassuring. I asked:

> what changes, exactly, has WoSign implemented to its policies and
> procedures to ensure that all trust programs in which WoSign is a
> participant are notified of future incidents, in line with each
> program's requirements?

I'm after the specifics of the changes to WoSign's policies and procedures regarding *notification*, not quality control. What were WoSign's previous policies and procedures regarding notification (obviously there was something in place, since Google was notified), and what changes have been made to improve those policies to ensure that all root programs are notified in line with each program's requirements in the future?

R: Sorry, I don't say it clear, please forgive my bad English since my native language is Chinese.
As I said this is my fault that we don't understand the Mozilla policy clearly that we don't think we need to report. But now we are clear that all mis-issued certificate case and any reported bug related system change also need to report. I and every related employee all clear now, then we can guarantee we will do it well in the future.
Why we log all SSL certificate from July 5th is for full transparency to let all related parties can report to us in the first time after the certificate is issued.


- Matt

--
I told [my daughter] that if I see her digging a hole that she might not be able to crawl out of, my job isn't to stand back and say "That's a *real* nice hole you're digging there".
-- Paul Tomblin, ASR

Richard Wang

unread,
Aug 25, 2016, 11:35:51 PM8/25/16
to Ryan Sleevi, mozilla-dev-s...@lists.mozilla.org
Yes, sorry for this.

As I admitted that this discussion gives us a big lesson that we know when we need to report incident to all browsers. We guarantee we will do it better.


Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Ryan Sleevi
Sent: Friday, August 26, 2016 8:16 AM
To: mozilla-dev-s...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

On Thursday, August 25, 2016 at 4:35:39 PM UTC-7, Matt Palmer wrote:
> I'm after the specifics of the changes to WoSign's policies and
> procedures regarding *notification*, not quality control. What were
> WoSign's previous policies and procedures regarding notification
> (obviously there was something in place, since Google was notified),
> and what changes have been made to improve those policies to ensure
> that all root programs are notified in line with each program's requirements in the future?

Clarification: In none of these incidents was Google notified proactively by WoSign. Instead, Google received communication from internal or external researchers regarding these issues, either prior to resolution or much later after the fact, and subsequently contacted WoSign regarding them.

It was only when Google found out recently that other programs were NOT notified, proactively, as had been expected, that Google shared the details it was aware of regarding various CA incidents, including those of WoSign, mentioned in this thread.

Richard Wang

unread,
Aug 25, 2016, 11:40:23 PM8/25/16
to Matt Palmer, dev-secur...@lists.mozilla.org
We know how to do in the future, and believe me we will do this better.


Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Matt Palmer
Sent: Friday, August 26, 2016 10:03 AM
To: dev-secur...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

On Thu, Aug 25, 2016 at 05:15:58PM -0700, Ryan Sleevi wrote:
> On Thursday, August 25, 2016 at 4:35:39 PM UTC-7, Matt Palmer wrote:
> > I'm after the specifics of the changes to WoSign's policies and
> > procedures regarding *notification*, not quality control. What were
> > WoSign's previous policies and procedures regarding notification
> > (obviously there was something in place, since Google was notified),
> > and what changes have been made to improve those policies to ensure
> > that all root programs are notified in line with each program's requirements in the future?
>
> Clarification: In none of these incidents was Google notified
> proactively by WoSign. Instead, Google received communication from
> internal or external researchers regarding these issues, either prior
> to resolution or much later after the fact, and subsequently contacted
> WoSign regarding them.

Oh, wow. I totally misread the initial report. That's almost certainly much *worse*, then, because it's not simply a matter of some adjustments to an existing process to improve it, but likely the development and deployment of an entirely new process.

- Matt

--
The main advantages of Haynes and Chilton manuals are that they cost $15, where the factory manuals cost $100 and up, and that they will tell you how to use two hammers, a block of wood, and a meerkat to replace "special tool
no. 2-112-A" -- Matt Roberds in asr.

percy...@gmail.com

unread,
Aug 26, 2016, 4:05:14 AM8/26/16
to mozilla-dev-s...@lists.mozilla.org
The news about possible sanction against WoSign was reported by Solidot http://www.solidot.org/story?sid=49448
(the Chinese version of Slashdot). Out of 12 comments in total (at the time of writing), 8 of them call for revocation of WoSign, the rest talks about the general bad security practices in China.

A quick intro of myself.
I'm Percy Alpha and I broke the news on GFW's MITM attack on iCloud, Outlook and Yahoo in 2014 and was later the victim of Great Cannon attack in 2015.

percy...@gmail.com

unread,
Aug 26, 2016, 4:15:57 AM8/26/16
to mozilla-dev-s...@lists.mozilla.org
In most Chinese institutions, most checks and verifications are just formality. Contracting to the case of CNNIC CA, I'm not advocating for an outright removal of WoSign (even though I revoked the CA personally). But the incorrect notBefore date suggests that a mandatory inclusion of CT of all certs ever issued is needed. Of course, WoSign needs to address other issues raised by Matt and Ryan in addition to the CT requirement.

Richard Wang

unread,
Aug 26, 2016, 4:26:26 AM8/26/16
to percy...@gmail.com, mozilla-dev-s...@lists.mozilla.org
This is the standard way in China Internet, if a west company say something to China company, all will support the west company.

PLEASE don’t move this technical problem to political issue, thanks.

Best Regards,

Richard

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of percy...@gmail.com
Sent: Friday, August 26, 2016 4:05 PM
To: mozilla-dev-s...@lists.mozilla.org
Subject: Re: Incidents involving the CA WoSign

233sec Team

unread,
Aug 26, 2016, 3:57:56 PM8/26/16
to mozilla-dev-s...@lists.mozilla.org
Wosign's Issue mechanism is high risking for large enterprise.
This is one prove:

https://gist.github.com/xiaohuilam/8589f2dfaac435bae4bf8dfe0984f69e

Alicdn.com is the cdn asset domain name of Taobao/tmall who belong to alibaba, which are Chinese biggest online shopping websites.
With the fake cert's middle man attack, password stealing, information leaking...

Jonathan Rudenberg

unread,
Aug 26, 2016, 4:24:44 PM8/26/16
to 233sec Team, mozilla-dev-s...@lists.mozilla.org
Here’s the crt.sh link for this certificate: https://crt.sh/?id=29884704

Can you provide more details about where this certificate came from? Did you issue it using one of the vulnerabilities discussed in this thread?

Richard Wang

unread,
Aug 29, 2016, 12:49:33 AM8/29/16
to Ryan Sleevi, mozilla-dev-s...@lists.mozilla.org
On Thursday, August 25, 2016 at 12:14:10 AM UTC-7, Richard Wang wrote:
> We can post all 2015 issued SSL certificate to CT log server if necessary.

Is there any reason not to do that proactively?

R: OK, we will post all 2015 issued SSL certificates to CT log server, but this take time since we issued 115K SSL certificate in 2015.

For incident 1 - mis-issued certificate with un-validated subdomain, total 33 certificates. We have posted to CT log server and listed in crt.sh, here is the URL. Some certificates are revoked after getting report from subscriber, but some still valid, if any subscriber think it must be revoked and replaced new one, please contact us in the system, thanks.
https://crt.sh/?id=7036355
https://crt.sh/?id=29805552
https://crt.sh/?id=7678955
https://crt.sh/?id=29805553
https://crt.sh/?id=29805554
https://crt.sh/?id=29805555
https://crt.sh/?id=29805556
https://crt.sh/?id=6798197
https://crt.sh/?id=29805558
https://crt.sh/?id=6798107
https://crt.sh/?id=29805559
https://crt.sh/?id=7728281
https://crt.sh/?id=29805560
https://crt.sh/?id=6639307
https://crt.sh/?id=29805561
https://crt.sh/?id=29805562
https://crt.sh/?id=6805650
https://crt.sh/?id=6739981
https://crt.sh/?id=7966229
https://crt.sh/?id=7094833
https://crt.sh/?id=29805563
https://crt.sh/?id=29805564
https://crt.sh/?id=29805565
https://crt.sh/?id=29805566
https://crt.sh/?id=29805567
https://crt.sh/?id=6843440
https://crt.sh/?id=29805568
https://crt.sh/?id=6999366
https://crt.sh/?id=29805569
https://crt.sh/?id=9534934
https://crt.sh/?id=29806448
https://crt.sh/?id=29813139
https://crt.sh/?id=29647048

For incident 0, the certificate issued related using higher level port validated, total 72 certificates. To be clear, those certificates are validated by website control validation method that using other port except 80 and 443. So we think those certificate no need to be revoked. The crt.sh link just for your reference.
https://crt.sh/?id=29805572
https://crt.sh/?id=7022909
https://crt.sh/?id=7564839
https://crt.sh/?id=29805573
https://crt.sh/?id=29805574
https://crt.sh/?id=29805575
https://crt.sh/?id=29805576
https://crt.sh/?id=29805577
https://crt.sh/?id=6969460
https://crt.sh/?id=29805578
https://crt.sh/?id=29805579
https://crt.sh/?id=29805580
https://crt.sh/?id=29805581
https://crt.sh/?id=29805582
https://crt.sh/?id=29805584
https://crt.sh/?id=29805585
https://crt.sh/?id=29805586
https://crt.sh/?id=9911443
https://crt.sh/?id=29805587
https://crt.sh/?id=7122803
https://crt.sh/?id=29805588
https://crt.sh/?id=29805589
https://crt.sh/?id=9985267
https://crt.sh/?id=29805590
https://crt.sh/?id=29805591
https://crt.sh/?id=29805592
https://crt.sh/?id=29805593
https://crt.sh/?id=29805594
https://crt.sh/?id=7224978
https://crt.sh/?id=10917791
https://crt.sh/?id=29805595
https://crt.sh/?id=29805596
https://crt.sh/?id=29805597
https://crt.sh/?id=6788465
https://crt.sh/?id=7224923
https://crt.sh/?id=9169568
https://crt.sh/?id=6836953
https://crt.sh/?id=29805598
https://crt.sh/?id=8172756
https://crt.sh/?id=29805599
https://crt.sh/?id=29805600
https://crt.sh/?id=7021184
https://crt.sh/?id=29805601
https://crt.sh/?id=29805602
https://crt.sh/?id=29805603
https://crt.sh/?id=29805604
https://crt.sh/?id=6927114
https://crt.sh/?id=6777468
https://crt.sh/?id=9793847
https://crt.sh/?id=29805605
https://crt.sh/?id=29805606
https://crt.sh/?id=29805607
https://crt.sh/?id=29805608
https://crt.sh/?id=9932344
https://crt.sh/?id=29805609
https://crt.sh/?id=29805610
https://crt.sh/?id=6880740
https://crt.sh/?id=29805611
https://crt.sh/?id=29805612
https://crt.sh/?id=7015627
https://crt.sh/?id=10008992
https://crt.sh/?id=29805613
https://crt.sh/?id=29805614
https://crt.sh/?id=29805615
https://crt.sh/?id=29805616
https://crt.sh/?id=7046181
https://crt.sh/?id=29805617
https://crt.sh/?id=29805618
https://crt.sh/?id=29805619
https://crt.sh/?id=7121749
https://crt.sh/?id=29805620
https://crt.sh/?id=6999366

Best Regards,

Richard

Gervase Markham

unread,
Aug 29, 2016, 4:27:59 AM8/29/16