WoSign: updated report and discussion

[Gervase Markham]

As noted by Richard Wang, WoSign have just published an updated Incident
Report:
https://www.wosign.com/report/WoSign_Incident_Report_Update_07102016.pdf

I think we are now in a position to discuss whether the plan proposed here:
https://docs.google.com/document/d/1C6BlmbeQfn4a9zydVi2UvjBGv6szuSB4sMYUcVrR8vQ/edit#
is still appropriate for WoSign.

Because it contains repeated or lightly-updated information about all of
the issues on the issues list, the updated Incident Report rather
"buries the lede" (hides the important news). Therefore I felt it might
be worth highlighting some of the things within it:

* WoSign admits that it has issued 64 back-dated SHA-1 certificates. The
cause was a mixture of intentional issuance using a created pathway, and
bugs which triggered that pathway by mistake.

* This includes admitting the misissuance of the certificates for
tyro.com by StartCom, which were the subject of Mozilla's most recent
investigation; this issuance was approved by Richard Wang.

* WoSign agrees it should have been more forthcoming about its purchase
of StartCom, and announced it earlier.

* WoSign and StartCom are to be legally separated, with the corporate
structure changed such that Qihoo 360 owns them both individually,
rather than WoSign owning StartCom.

* There will be personnel changes:

- StartCom’s chairman will be Xiaosheng Tan (Chief Security Officer
of Qihoo 360).
- StartCom’s CEO will be Inigo Barreira (formerly GM of StartCom
Europe).
- Richard Wang will be relieved of his duties as CEO of WoSign and
other responsibilities. It is not decided who will replace him.

* StartCom will soon provide a plan on how they will separate their
operations and technology from that of WoSign.

* In the light of these changes, Qihoo 360 request that WoSign and
StartCom be considered separately.


Mozilla is minded to agree that it is reasonable to at least consider
the two companies separately, although that does not preclude the
possibility that we might decide to take the same action for both of
them. Accordingly, Mozilla continues to await the full remediation plan
from StartCom so as to have a full picture. However, I think we can work
towards a conclusion for WoSign now.

Gerv

[Gervase Markham]

On 07/10/16 12:12, Gervase Markham wrote:
> Mozilla is minded to agree that it is reasonable to at least consider
> the two companies separately, although that does not preclude the
> possibility that we might decide to take the same action for both of
> them. Accordingly, Mozilla continues to await the full remediation plan
> from StartCom so as to have a full picture. However, I think we can work
> towards a conclusion for WoSign now.

My view is simple: notwithstanding the dismissal of Richard Wang, I am
not confident in the robustness of WoSign's technology or that it has an
appropriate culture of security, and think that the measures outlined
previously continue to be appropriate for the roots that they control.

Gerv

[Jakob Bohm]

On 07/10/2016 13:12, Gervase Markham wrote:
> ...

> * WoSign agrees it should have been more forthcoming about its purchase
> of StartCom, and announced it earlier.
>
> * WoSign and StartCom are to be legally separated, with the corporate
> structure changed such that Qihoo 360 owns them both individually,
> rather than WoSign owning StartCom.
>
> * There will be personnel changes:
>
> - StartCom’s chairman will be Xiaosheng Tan (Chief Security Officer
> of Qihoo 360).
> - StartCom’s CEO will be Inigo Barreira (formerly GM of StartCom
> Europe).

> ...

> * StartCom will soon provide a plan on how they will separate their
> operations and technology from that of WoSign.
>
> * In the light of these changes, Qihoo 360 request that WoSign and
> StartCom be considered separately.
>
>
> Mozilla is minded to agree that it is reasonable to at least consider
> the two companies separately, although that does not preclude the
> possibility that we might decide to take the same action for both of
> them. Accordingly, Mozilla continues to await the full remediation plan
> from StartCom so as to have a full picture. However, I think we can work
> towards a conclusion for WoSign now.
>

As an outsider, here is one question: If StartCom has not yet decided
on a technical separation plan, could one acceptable option for such a
plan be to reactivate the old (pre-acquisition) infrastructure and
software and take it from there?

An answer to that might help StartCom choose an acceptable plan.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Gervase Markham]

On 07/10/16 12:23, Jakob Bohm wrote:
> As an outsider, here is one question: If StartCom has not yet decided
> on a technical separation plan, could one acceptable option for such a
> plan be to reactivate the old (pre-acquisition) infrastructure and
> software and take it from there?
>
> An answer to that might help StartCom choose an acceptable plan.

Mozilla gave some guidance to the StartCom representatives at the
meeting as to what might be acceptable ways forward, and this was one of
them. However, it's not the only possible route and so we will wait to
see what they propose.

Gerv

[Patrick Figel]

On 07/10/16 13:23, Jakob Bohm wrote:
> On 07/10/2016 13:12, Gervase Markham wrote:

>> ... * WoSign agrees it should have been more forthcoming about its

>> purchase of StartCom, and announced it earlier.
>>
>> * WoSign and StartCom are to be legally separated, with the
>> corporate structure changed such that Qihoo 360 owns them both
>> individually, rather than WoSign owning StartCom.
>>
>> * There will be personnel changes:
>>
>> - StartCom’s chairman will be Xiaosheng Tan (Chief Security
>> Officer of Qihoo 360). - StartCom’s CEO will be Inigo Barreira

>> (formerly GM of StartCom Europe). ... * StartCom will soon provide

>> a plan on how they will separate their operations and technology
>> from that of WoSign.
>>
>> * In the light of these changes, Qihoo 360 request that WoSign and
>> StartCom be considered separately.
>>
>>
>> Mozilla is minded to agree that it is reasonable to at least
>> consider the two companies separately, although that does not
>> preclude the possibility that we might decide to take the same
>> action for both of them. Accordingly, Mozilla continues to await
>> the full remediation plan from StartCom so as to have a full
>> picture. However, I think we can work towards a conclusion for
>> WoSign now.
>>
>

> As an outsider, here is one question: If StartCom has not yet
> decided on a technical separation plan, could one acceptable option
> for such a plan be to reactivate the old (pre-acquisition)
> infrastructure and software and take it from there?
>
> An answer to that might help StartCom choose an acceptable plan.

I think a good approach for StartCom's remediation plan would be to
follow the conditions for readmission suggested by Mozilla:

> * A Point-In-Time Readiness Audit (PITRA) from a Mozilla-agreed
> WebTrust auditor;
> * A full code security audit of their issuing infrastructure from a
> Mozilla-chosen security auditor;
> * 100% embedded CT for all issued certificates, logged to at least
> one Google and one non-Google log not controlled by WoSign/StartCom;

[Ryan Sleevi]

Gerv,

Could you explain how Mozilla feels this line of reasoning and explanation is different than, say, how Mozilla handles Symantec inclusion requests?

For example, it seems you're suggesting that:
* This was the result of rogue employees acting outside of appropriate policies
* The people responsible have been sacked, and everyone will undergo retraining
* Systems will be updated to prevent misissuance in the future

This response is very similar to that offered by Symantec when news of their misissuance first came to light, but it was later revealed that the issues were much deeper, much more systemic, and continued to persist well beyond the dates it was claimed for remediation.

I mention this not to suggest that StartCom is the same as Symantec, but to highlight precisely how this seems like this seems to move from a neutral, consistent standard, into one that favors some CAs while not treating others equally.

For example, it was suggested that there would be a similar investigation into StartCom as there was with WoSign, with a gathering of issues for public discussion. Is that still the plan? It sounds like you're suggesting that Mozilla has reached some agreement for remediation with StartCom independent of WoSign, despite these two companies having the same management structure for the past year, despite having undergone significant infrastructure changes (which are trivially observable by outsiders, both via CT and via interactions with the CA), and, as evidenced by StartEncrypt, sharing significant development infrastructure. Both represent to the same parent company, which allowed these issues to happen in the first place, so what assurances do the public have that they won't happen again?

I appreciate the consideration to the ecosystem - particularly, the desire to avoid the need to distrust a CA, which might result in a poor user experience. However, I'd be curious to better understand how Mozilla prioritizes that over the ecosystem risk and implications of such a decision.

For example, this would suggest that, if a company wished to minimize risk of it's misissuances - or to actively engage in the practice while limiting liability - it might separate out certificates into subsidiary companies. They could then fully share infrastructure, development, etc - but only the one that actively "misissued" would be penalized, while the other sister companies (which share the same 'everything') could continue to be trusted, so long as the deck chairs were reshuffled after every iceberg.

I appreciate the sensitivities towards a situational approach - considering the facts around the misissuance and not just the policies themselves - but this does seem to create a dangerous precedent, if gone forward with, that could have significant negative consequences to the ecosystem in the future. The issues we've seen here are clearly systemic, and reflected throughout the organization - the development and security practices, the clear disregard for the stated policies and practices, and the obvious disconnect in messaging. I can appreciate that Qihoo 360 may not have realized how problematic some of these practices were with respect to WoSign, but that should moreso be a cause for concern, rather than reprieve, since we have no assurances things will improve.

In a separate thread regarding SHA-1 exception process, you highlighted that the ecosystem around root inclusion has failed to improve, resulting in risks being borne by the WebPKI, due to the lack of risk or cost for these "exceptional" situations where customers ignore communications and changes in the security landscape. I'm curious if Mozilla feel's that the CA market is different, as it would seem to be suggested here.

[Gervase Markham]

Hi Ryan,

I should start by reiterating what you already know, but might be a
useful reminder for others - no agreement has been made between Mozilla
and Qihoo/StartCom/WoSign. We gave them advice on what we thought the
community might like to see, but they are responsible for their plan,
and the Module Owner will take the final decision on what action Mozilla
will take. I have my own opinions, which clearly will carry some weight,
but the decision is not mine. So the following is an explanation of how
I myself am viewing the situation.

My view is that WoSign was not run in the way a CA should be run, in a
variety of ways. This was not generally true of StartCom, which was
reasonably well-run (although not perfect). Once WoSign bought StartCom
and StartCom started being influenced by WoSign technology and
management, things went downhill there too. But I feel that
re-separation of the two - at all levels, ownership, management and
technology - might allow StartCom to continue as a going concern. This
would imply StartCom has management Mozilla trusts, no ownership
connection through WoSign, and uses reliable technology not authored at
WoSign.

Someone else viewing the same evidence might conclude that it's too late
for such a separation, and the damage is done. I would understand that
conclusion, but it's not my view.

On 07/10/16 15:48, Ryan Sleevi wrote:
> Could you explain how Mozilla feels this line of reasoning and
> explanation is different than, say, how Mozilla handles Symantec
> inclusion requests?
>
> For example, it seems you're suggesting that:
> * This was the result of rogue employees acting outside of
> appropriate policies

Well, the employee in question set the policies. Misconduct at the CEO
level is somewhat more serious than misconduct at a much lower level.

> This response is very similar to that offered by Symantec when news
> of their misissuance first came to light, but it was later revealed
> that the issues were much deeper, much more systemic, and continued
> to persist well beyond the dates it was claimed for remediation.

I would say the key differences with Symantec are that WoSign's
misdemeanours involved actual misissuance to third parties (e.g the
github certificates) and provable lying, e.g. about ownership and SHA-1
backdating. I don't see either of those serious components in the
Symantec case. This is not to minimise what happened at Symantec, but I
do think the WoSign situation is more serious. But then, even before
this, I think it was already the case that you took a dimmer view of the
happenings at Symantec than I did (your view perhaps being based on more
complete information).

> I mention this not to suggest that StartCom is the same as Symantec,
> but to highlight precisely how this seems like this seems to move
> from a neutral, consistent standard, into one that favors some CAs
> while not treating others equally.

No decision has been made on how to treat StartCom; as soon as the plan
emerges (which needs to be soon) we can take a view. The kind of robust
challenge that you are providing is most welcome :-)

> For example, it was suggested that there would be a similar
> investigation into StartCom as there was with WoSign, with a
> gathering of issues for public discussion. Is that still the plan?

The last part of the "WoSign and StartCom" document said:

"We are open to accepting further evidence, including whether there are
any significant errors in the above which would affect the narrative,
whether there have been any other problems with WoSign or StartCom’s
certificate issuance, and also any data relevant to our current view
that it is appropriate to treat these two CAs together."

No-one has come forward with additional StartCom issues since then. We
know about StartEncrypt (WoSign code) and Tyro (authorized by the WoSign
CEO). If you feel that invitation was insufficiently publicised, I
hereby issue it again.

As I said in my previous email, Qihoo's plans are enough, I think, count
as "data relevant to our current view" and I think we should at least
consider the two CAs separately, although that doesn't preclude reaching
the same conclusion for each.

> It
> sounds like you're suggesting that Mozilla has reached some agreement
> for remediation with StartCom independent of WoSign, despite these
> two companies having the same management structure for the past year,
> despite having undergone significant infrastructure changes (which
> are trivially observable by outsiders, both via CT and via
> interactions with the CA), and, as evidenced by StartEncrypt, sharing
> significant development infrastructure. Both represent to the same
> parent company, which allowed these issues to happen in the first
> place, so what assurances do the public have that they won't happen
> again?

As noted above, no agreement has been reached. However, as the person
who took a meeting with Qihoo's Head of Security, who will now chair
StartCom, I feel that he does understand the issues and I am willing to
give his chairmanship and Inigo Barreia's CEO-ship an opportunity to
demonstrate they can run a CA well. Inigo's track record at Izenpe is
good - I'm not aware of any incidents involving them.

> For example, this would suggest that, if a company wished to minimize
> risk of it's misissuances - or to actively engage in the practice
> while limiting liability - it might separate out certificates into
> subsidiary companies. They could then fully share infrastructure,
> development, etc - but only the one that actively "misissued" would
> be penalized, while the other sister companies (which share the same
> 'everything') could continue to be trusted, so long as the deck
> chairs were reshuffled after every iceberg.

Indeed, this would be a bad situation - which is why, in my mind, a
different deal for StartCom would be predicated on them moving quickly
to a position where they share _nothing_ with WoSign, rather than
everything. The WoSign document announced, almost in passing, the
ownership and management changes planned, and I expect the StartCom
remediation plan to give details about the technical and infrastructure
changes planned.

> In a separate thread regarding SHA-1 exception process, you
> highlighted that the ecosystem around root inclusion has failed to
> improve, resulting in risks being borne by the WebPKI, due to the
> lack of risk or cost for these "exceptional" situations where
> customers ignore communications and changes in the security
> landscape. I'm curious if Mozilla feel's that the CA market is
> different, as it would seem to be suggested here.

I think that the plan we propose for WoSign is a clear statement that
the sort of behaviours seen there are unacceptable. I don't see them as
getting off lightly.

Gerv

[Ryan Sleevi]

On Friday, October 7, 2016 at 9:10:29 AM UTC-7, Gervase Markham wrote:
> I should start by reiterating what you already know, but might be a
> useful reminder for others - no agreement has been made between Mozilla
> and Qihoo/StartCom/WoSign. We gave them advice on what we thought the
> community might like to see, but they are responsible for their plan,
> and the Module Owner will take the final decision on what action Mozilla
> will take. I have my own opinions, which clearly will carry some weight,
> but the decision is not mine. So the following is an explanation of how
> I myself am viewing the situation.

Understood, and with that same disclaimer...

> My view is that WoSign was not run in the way a CA should be run, in a
> variety of ways. This was not generally true of StartCom, which was
> reasonably well-run (although not perfect).

One possible issue with this is that there hasn't been a similar question about StartCom's past practices. I think that, up until the discussion began, particularly around the backdating of certificates, it might have been said the same about WoSign - that is, the view that the issues were 'minor'.

> Once WoSign bought StartCom
> and StartCom started being influenced by WoSign technology and
> management, things went downhill there too. But I feel that
> re-separation of the two - at all levels, ownership, management and
> technology - might allow StartCom to continue as a going concern. This
> would imply StartCom has management Mozilla trusts, no ownership
> connection through WoSign, and uses reliable technology not authored at
> WoSign.

Well, there can never be a perfect separation - though the WoSign report indicates that there wasn't a complete close of financial disbursements, there was a significant organizational and operational shift made under the terms of the contract that StartCom is, at present, significantly enmeshed with WoSign, and that process happened over a year.

We know that the investors and principles have changed, and thus the priorities have changed. I think, in the case of many CAs, we might say that, prior to their misissuance, they were well-run CAs. But isn't that part of the problem? By taking into perspective a historical view, rather than an incident-based view, it naturally gravitates towards favoring incumbents (those who have been able to operate long enough to be considered 'mature' when a misissuance happens), and begins to introduce subjective-based weights into a process that is, ideally, largely objective.

The implication here - that factors such as management and technology bear into decision making - suggest that future inclusion or maintenance requests need to consider this. I don't disagree that these could be valuable axis' in determining trust, but I think historically, Mozilla's Root Store has erred on the side of objectivity. For example, we see discussions about particular countries views' on wiretaps brought up from time to time, or particular companies' associated businesses providing wiretap/intercept capabilities, but on the whole, these associations are rejected as being influential in the decision to include or not - instead, it's based on (ideally) objective evaluations against audit criteria and technical standards.

So now when we have a failure to abide by those criteria, why shouldn't we adhere to that same set of standards? Why introduce these subjective elements into the decision making?

Clearly, there's a set of priorities at play here - what are the ultimate goals for Mozilla's Root Store? What are the things to prioritize?

As a thought experiment, I'm trying to get to the heart of the reasoning of why a CA (and all affiliated entities) should or should not be distrusted when an incident occurs.

An argument for distrust is that it strongly signals to the ecosystem that there is a serious risk in non-compliance. As such, the greater the fiduciary or business risk, the greater justification there is for investments into policies, practices, and technologies to minimize that risk. If you eliminate any risks, what incentives are there to align on proper behaviour, especially given the economic structures of CA trust, such that there is no long-term reputational brand damage for misissuance, nor is there any way to discover it (from people 'outside the know'). That is, to an end user, there's no distinction in trust between "Honest Achmed's Gently Used Certificates" and "Best CA in the world" - and as such, there's no incentives for site operators to consider one or the other.

An argument against distrust is generally that of user impact. Distrust options, at present, vary in proportionality of user impact (generally with the size of the CA), and thus the larger the CA, the more difficult it can be seen to take action.

But is that Mozilla's justification or set of priorities? It might be useful to understand what you (and other module contributors, to be particular, since we all may have different views) prioritize.

To me, I value long-term ecosystem health, and the desire for objectivity, more than user impact. This may not be entirely fair, as we see in past incidents, but the fact that we keep having incidents I believe suggests that something more than the status quo is needed.

> I would say the key differences with Symantec are that WoSign's
> misdemeanours involved actual misissuance to third parties (e.g the
> github certificates) and provable lying, e.g. about ownership and SHA-1
> backdating. I don't see either of those serious components in the
> Symantec case. This is not to minimise what happened at Symantec, but I
> do think the WoSign situation is more serious. But then, even before
> this, I think it was already the case that you took a dimmer view of the
> happenings at Symantec than I did (your view perhaps being based on more
> complete information).

This is a disconcerting viewpoint for me. It suggests that you view "We've sacked the people responsible" as an acceptable response, so long as we can't prove the CA is lying. While I agree that we should take into consideration that which we definitely know, and the facts surrounding the issue, I highlight this because, in the majority of cases, we don't know or won't be able to definitively prove the misissuance.

In these scenarios, should our position be to 'fail open' (and continue to trust the CA, with some set of remediation plan) or 'fail closed' (and remove the CA until it can prove it's in compliance).

The issue I have with 'fail open' is that you can see, throughout the engagement with CAs, significant leeway is given to them for slipping past compliance dates and practices, so much that it's become the rule, rather than the exception, that a CA will miss dates. I appreciate the need to be sensitive to business pressures, but I think it meaningfully impairs improvements in the WebPKI.

While I realize this sounds closer to a 'zero tolerance' policy than even perhaps I'd be comfortable with, I think the risk - to global user safety and trust - is significant here.

> As I said in my previous email, Qihoo's plans are enough, I think, count
> as "data relevant to our current view" and I think we should at least
> consider the two CAs separately, although that doesn't preclude reaching
> the same conclusion for each.

I'm uncomfortable with this, because it's a promise, with no timeline for delivery, and significant risk until it's met. This gets back to the question of priorities and goals for the root program. Would we accept a new CA that presented serious issues but promised to resolve them? I think history shows quite clearly that no - the community rejects such applications until the CA is able to resolve, and demonstrate they resolve them. The fact that this causes delays to the CA's application therefore incentivizes getting it correct.

If the view is that once you're in, there's a higher bar to get out, then it sets a double standard as to how the program is maintained, and one that I fear may put users at risk.

> As noted above, no agreement has been reached. However, as the person
> who took a meeting with Qihoo's Head of Security, who will now chair
> StartCom, I feel that he does understand the issues and I am willing to
> give his chairmanship and Inigo Barreia's CEO-ship an opportunity to
> demonstrate they can run a CA well. Inigo's track record at Izenpe is
> good - I'm not aware of any incidents involving them.

Does this suggest that Mozilla would be willing to meet with CA applicants' CSO/CEOs, to get a 'gut sense' about whether they're "serious", and then decide to overlook failures to abide by Mozilla's inclusion requirements? I ask this, because I think this highlights some of the disconcerting disconnect with "Oh, I know Jim, Jim's a good guy" sort of approaches.

It may be that the answer is yes, which would be a consistent standard. However, if the question is "maybe", then it's useful to work through when it would and would not be an acceptable situation, otherwise the risk is that the standard of trust is varied, rather than consistent.

> Indeed, this would be a bad situation - which is why, in my mind, a
> different deal for StartCom would be predicated on them moving quickly
> to a position where they share _nothing_ with WoSign, rather than
> everything.

How, if ever, will the community be assured of this? This is something that's quite intangible - which is the very problem.

> I think that the plan we propose for WoSign is a clear statement that
> the sort of behaviours seen there are unacceptable. I don't see them as
> getting off lightly.

No, but it suggests that if you play the game of organizational structuring, you can reduce risk of consequence. It suggests that, rather than integrate systems (such as Symantec has done with brands, or as Entrust is doing with AffirmTrust), you maintain them as "arms-distance" (legally speaking) entities, while the elements that the public cannot see are shared. Is that uncertainty worth introducing into the ecosystem? Is it something we already accept? I'm not sure, but I'm quite uncomfortable with the implications of the line of thinking.

Then again, I assume CAs will do the worst thing possible, and every loophole possible. While perhaps not true of all CAs, in aggregate, the market incentives are so aligned that the worst behaviour reaps the most reward, and that's disturbing.

[Andrew Ayer]

On Fri, 7 Oct 2016 12:12:58 +0100
Gervase Markham <ge...@mozilla.org> wrote:

> * WoSign and StartCom are to be legally separated, with the corporate
> structure changed such that Qihoo 360 owns them both individually,
> rather than WoSign owning StartCom.
>
> * There will be personnel changes:
>

> - StartCom___s chairman will be Xiaosheng Tan (Chief Security Officer
> of Qihoo 360).
> - StartCom___s CEO will be Inigo Barreira (formerly GM of StartCom

> Europe).
> - Richard Wang will be relieved of his duties as CEO of WoSign and
> other responsibilities. It is not decided who will replace him.
>
> * StartCom will soon provide a plan on how they will separate their
> operations and technology from that of WoSign.
>
> * In the light of these changes, Qihoo 360 request that WoSign and
> StartCom be considered separately.
>

> Mozilla is minded to agree that it is reasonable to at least consider
> the two companies separately

Consider the following hypothetical: Honest Achmed's Used Cars and
Certificates operates two roots, Honest Achmed Root A and Honest Achmed
Root B. The two roots share much of the same infrastructure, and over
the same period of time, both roots have serious incidents, including
Honest Achmed himself approving the backdating of SHA-1 certificates
under both roots.

After the incidents come to light, Honest Achmed's majority owner,
Uncle Mehmet, fires Honest Achmed and places Root A and Root B under
the control of two separate companies. He asks that Mozilla consider
the fate of Root A and Root B separately.

That seems like a very unreasonable request to me - a mismanaged CA
shouldn't be able to save some of their roots by spinning them off into
a separate company after they're caught. How is WoSign/StartCom
different? It doesn't matter that at one point in the past WoSign and
StartCom were separate companies. During the time that the incidents
occurred, StartCom and WoSign were for all intents and purposes the
same company, one wholly owned by the other, both managed by the same
disgraced CEO, and sharing significant infrastructure. They should
therefore be treated as the same company when responding to these
incidents.

Any restructuring and personnel changes at this point could influence
Mozilla's future consideration of StartCom and WoSign (e.g. during root
inclusion requests) but it cannot change the past and therefore should
not alter how Mozilla responds to what happened in the past.

Regards,
Andrew

[Ryan Hurst]

All,

What I am about to say represents my own personal beliefs and are not necessarily the same beliefs of my employer, Google, or Let’s Encrypt where I am an advisor.

I have been involved in the WebPKI since its inception. In WebPKI, a Certificate Authority has a conflicted role, it is responsible for acting in the best interest of the Relying Party but its existence is dependent on that of the Subscriber.

This is true of all CAs, even Let’s Encrypt, where its existence is dependent on donations from large organizations who, in many cases, utilize their service.

This model only works when there is a consequence for CA’s that violate the interests of the Relying Party.

That begs the question of what are the interests of a Relying Party in the context of the WebPKI? I would say the relying party expects CAs:
- To understand the deeply technical nature of X.509 and the web,
- To deploy products and services that securely support secure communications on the web,
- To operate their services in such a way that they are verifiable by third-parties,
- To act in a trustworthy and transparent way.

As I look at what has happened in this particular case, despite recent gestures, it is clear to me that WoSign has not lived up to these expectations.

While I have a ton of admiration for Eddy and the way that the independent StartCom operated, StartCom is a corporate entity and not an individual. Moreover, given that for the last year it has had numerous technical lapses, and its leadership misrepresented the material facts about its operation, it also has largely failed on these points.

This begs the question of what should be done in this case. I believe the answer there is buried in the role of the browsers in the WebPKI ecosystem where they represent the interests of relying parties.

To this end, when I managed the Microsoft Root Program I did my best to guide my decisions by the following tenets:
- I fight for the relying party,
- I fight for the WebPKI ecosystem,
- I must be predictable and fair,
- I must encourage the ecosystem to evolve to meet changing needs,
- I must comply with all legal and regulatory obligations.

In this case, it seems to me that WoSign’s purchase of StartCom, short of the lies and subterfuge (which I do not mean to trivialize), is not materially different than Symantec’s ownership of Thawte, RapidSSL, or GeoTrust brands.

In past actions, against Symantec, there were no carve outs for the different brands. As such, it would seem that to do so for WoSign and StartCom would not be an action consistent with the principals I tried to live by as a manager of a root program.

That then takes us to the structural changes proposed by Qihoo. I should say that I personally have faith in Inigo as a leader who would do the right thing for the WebPKI and believe that overall these changes seem like the right gestures to be making. They do not, however, negate the facts in question.

It seems to me based on this thread that Mozilla, or more specifically Gerv, is inclined to treat StartCom differently, I can assume this is because:
- StartCom prior to its acquisition had a positive brand reputation,
- He agrees that the new leadership would likely act in the right interest of the WebPKI.

The problem is that this sets a dangerous precedent. Let’s assume a similar situation happens in the future with another CA who owns multiple brands. Would you ignore the violations of the rules and allow them to carve off one brand because you liked who they would let manage it if you do?

I would hope the answer is no.

I would say that holding them equally accountable is the right thing to do, since for the time in question, they were equivalently managed and operated.

To offer much more than that would not be fair or in the best interest of the WebPKI ecosystem.

Ryan

[okaphone.e...@gmail.com]

It seems to me that, considering WoSign and StarCom were when it happened actually the same entity, the things that went wrong should be weighed equally against them. So if only what has happened is relevant, it makes no sense to deal with them separately.

If however what they propose/promise to do is also relevant, then it does make sense to see if they should be treated differently. For that may be where the two differ.

CU Hans

[Han Yuwei]

在 2016年10月7日星期五 UTC+8下午7:13:42，Gervase Markham写道：

According to the previous discussion, I think that maybe a community-driven program of CA's issue system could be established. Code is open and everyone can audit it. But what's more important is to build the culture of reporting issue like civil aviation and civil nuclear system.

[Jakob Bohm]

I would say that it is only natural that when Mozilla or other root
programs act a bit like an enforcement court that it is reasonable that
the root programs consider the same kinds of "soft" circumstances that
a regular court would consider when measuring out punishments.

While it is probably too late at this hour (it is already Saturday in
China, it is already Sabbath in Israel (sunset), and it is late Friday
night on the British isles), some things that could potentially be
added to increase the justification of treating a reborn B better than
A might be (I have absolutely no decision power in this, just arguing
in general):

- Something that involves a significant economic cost to Uncle Mehmet,
thus providing a strong economic disincentive to other CA owners that
might want to participate in a race to the bottom. Each of the
following suggestions would involve at least some such cost.

- Removing ownership of B from the organization that owned it during
its bad year, just in case someone higher up was complicit in ways
other than trusting Nephew Honest Achmed. For example this could
involve selling B at a significant loss.

- A promise that the new / rebooted leaders of B will before a
specified date, and at B's or Mehmet's cost go through the
records from when they first started talking to Achmed until the
reform, looking for mis-issued certificates and/or any way in which
Achmed could have issued certificates not on their records (for
example, was Achmed or his minions given access to the private key or
to some other way of signing certificates outside the control of the
HSM? Did the HSM ever stop counting issued certificates such that the
number of issued certificates is no longer provable? Did the HSM ever
issue certificates that can neither be found nor revoked due to
unknown serial number for example?).

- A promise that before another specified date, an outside auditor
chosen by noone from the A/B/M family will do the same checks as
above, and be paid a specified fee for doing so.

- A promise that new B roots will be spun up and all genuine
certificates reissued at B's or Mehmet's cost before a specified date,
such that 1 month after that date, all the old B roots can be
distrusted.

- A condition that B issues no certificates for the next 15 months,
maintaining a perfect record of functional revocation services during
that time, only then being allowed to reenter with new root keys.
This may or may not be combined with permission to let another
(independent, well-established) CA to run the B brand as seen by
subscribers, with all vetting and security handled by that independent
CA, but with a contractual condition that said independent CA is not
allowed to actively steal customers over to its own brand unless B
closes permanently or is refused reentry to the root programs. Thus B
would loose 15 months of income while keeping up significant
operational costs just for the hope of maybe getting readmitted.

[uri...@gmail.com]

Did anyone ever determine if "Andy Ligg" is in fact a real person?
(As discussed here
https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/0pqpLJ_lCJQ/7QRQ7oqGDwAJ )

If he in fact was a pseudonym for a WoSign employee, then arguably there was no distinction between WoSign and StartCom. (And of course the issue that no-one from StartCom objected to a pseudonym being used to represent them.)

[Honest Achmed]

Andrew Ayer [ag...@andrewayer.name] yazdi:

Consider the following hypothetical: Honest Achmed's Used Cars and
Certificates operates two roots, Honest Achmed Root A and Honest Achmed
Root
B. The two roots share much of the same infrastructure, and over the same
period of time, both roots have serious incidents, including Honest Achmed
himself approving the backdating of SHA-1 certificates under both roots.

Well this would never happen since backdating would imply that Honest Achmed
is not Honest. Since Achmed is by definition Honest, he would never wind
back
the odometer on a car, backdate a SHA-1 certificate, or patch up a rusted
panel in a lovely light blue 1972 Anadol A2 with only one previous owner
using
body filler, no matter what Cousin Husamettin says.

After the incidents come to light, Honest Achmed's majority owner, Uncle
Mehmet, fires Honest Achmed

Honest Achmed is the sole owner of Honest Achmed's Used Cars and CA.
Achmed's
Uncle Mehmet passed away several years ago and never held any controlling
interest. Nor did Cousin Ligg or Uncle Wang. Pay no attention to the holding
company in the UK or the as-yet-undiscovered shell company in Malta.

How is WoSign/StartCom different?

I do not see how WoSign has done wrong. The purpose of a CA is to sell
certificates and make money. If a client comes to the CA and asks for a
certificate, the CA sells them one and makes a profit. This is what WoSign
has done. This is what a CA does. What is the problem?

[谭晓生]

Dear All,
This is the information that would be released by Inigo in the coming week, Percy asked me to answer the question, so, it is here:

Business Supporting Software:
There are 5 components of StartCom’s business supporting software:
1. Official Website + Ordering system
Code: Independent code, totally different with WoSign’s one.
Server: Dedicate server, no sharing.
Location: Hosted in Qihoo 360 Los Angeles, U.S China Telecom America IDC, WoSign’s one is hosted in China.
Business Process/Logic: Different with WoSign’s business process/logic, keeping the same business process/logic with the old system before the acquisition.
UI of Web: Different with WoSign’s, Revised version of StartCom’s original version. Developed by WoSign RD team.

2. CMS (Certificate Management System)
Code: Independent code, totally different with WoSign’s one.
Server: Dedicate server, no sharing.
Location: The primary server is hosted in Qihoo 360 head quarter’s data center in Beijing, there is a backup server in WoSign’s office in Shenzhen.
Business Process/Logic: Different with WoSign’s business process/logic, keeping the same business process/logic with the old system before the acquisition.

3. PKI – signing service
Code: Same code with WoSign’s one.
Server: Shared Server.
Location: The primary one is hosted in Qihoo 360 head quarter’s data center in Beijing since Dec 2015, there is a backup server in Wosign’s office in Shenzhen.
Business Process: Same

4. CRL/OCSP
Code: Same code with WoSign’s one.
Server: Dedicate server, no sharing.
Location: StartCom and WoSign CRL/OCSP source server is located in Qihoo 360 USA IDC, the backup source server is hosted in Qihoo 360 head quarter’s data center in Beijing.
Cache Service: by Akamai for oversea visitors, Qihoo 360 CDN for China visitors
Business Process: Same

5. TSA
Code: Same code with Wosign’s one.
Server: Dedicate server, no sharing.
Location: StartCom TSA: http://tsa.startssl.com is located in Qihoo 360 Los Angeles IDC, WoSign TSA: http://timestamp.wosign.com is hosted in Qihoo 360 China IDC.
Business Process: Same

So, For StartCom and WoSign’s infrastructure, only the PKI servers were/are shared, the CRL/OCSP, TSA code were cloned but hosted in different IDC, nothing were/are shared on Official Website, Ordering System and CMS.

Data Center:
For Qihoo 360 Los Angeles IDC, it is a co-location with China Telecom North America company, located at 600 W 7TH ST Suite 570, LOS ANGELES CA 90017, governed by U.S Law.

Employee:
For employees, the StartCom and WoSign shared the software development team which paid by WoSign, but with independent validation team, customer care team and tech support team which were/are paid by StartCom China, StartCom U.K and StartCom IL, StartCom’s teams were/are English speaking employees, WoSign’s were/are Chinese speaking employees.

More information will be available in the SC report which will be released by Inigo soon.

Thanks，
Xiaosheng Tan, Chief Security Officer of Qihoo 360



在 2016/10/9 上午7:02，“dev-security-policy 代表 Percy”<dev-security-policy-bounces+tanxiaosheng=360...@lists.mozilla.org 代表 percy...@gmail.com> 写入:

His writing style is very similar to StartCom's website which is produced in China. As we're examining the infrastructure of the two companies, could Mozilla ask Qihoo 360 to disclose the current personnel and technical infrastructure shared between WoSign and StartCom.
WoSign has denied that they shared those infrastructures but we know WoSign lied. So I want to ask Qihoo 360 the same question and see whether Qihoo has a different answer.
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

[Peter Bowen]

On Fri, Oct 7, 2016 at 9:50 AM, Ryan Sleevi <ry...@sleevi.com> wrote:

> On Friday, October 7, 2016 at 9:10:29 AM UTC-7, Gervase Markham wrote:
>> I have my own opinions, which clearly will carry some weight,
>> but the decision is not mine. So the following is an explanation of how
>> I myself am viewing the situation.
>

> with that same disclaimer...

I will add my own disclaimer: opinions here are my own and do not
necessarily represent those of my employer.

>> Once WoSign bought StartCom
>> and StartCom started being influenced by WoSign technology and
>> management, things went downhill there too. But I feel that
>> re-separation of the two - at all levels, ownership, management and
>> technology - might allow StartCom to continue as a going concern. This
>> would imply StartCom has management Mozilla trusts, no ownership
>> connection through WoSign, and uses reliable technology not authored at
>> WoSign.
>

> Well, there can never be a perfect separation - though the WoSign report indicates that there wasn't a complete close of financial disbursements, there was a significant organizational and operational shift made under the terms of the contract that StartCom is, at present, significantly enmeshed with WoSign, and that process happened over a year.

I think that this is key. A CA operator is a legal entity run by
people. Which people own what percentage of the operator is one
relevant input to the discussion. How much they have been paid or not
been paid might be an interesting story for tabloids (maybe I should
pitch it to NetCraft), but should have no relevance.

What is relevant is who has operational management over the CA. There
is no question at this point that WoSign exercised control over
StartCom. The report clearly states that the CEO of WoSign directed
StartCom to issue a certificate in June 2016 that was on contravention
to the StartCom CPS and StartCom complied. I don't think there can be
much stronger evidence. And, just to be clear, WoSign's own reports
clearly show that the CEO of WoSign had operational control of WoSign.
In the 4 September 2016 report from WoSign, Figures 4 and 5 clearly
show that the CEO approved all changes to systems. So there should be
no question of shared operational control of StartCom and WoSign in
addition to WoSign owning StartCom.

>> As I said in my previous email, Qihoo's plans are enough, I think, count
>> as "data relevant to our current view" and I think we should at least
>> consider the two CAs separately, although that doesn't preclude reaching
>> the same conclusion for each.
>

> I'm uncomfortable with this, because it's a promise, with no timeline for delivery, and significant risk until it's met. This gets back to the question of priorities and goals for the root program. Would we accept a new CA that presented serious issues but promised to resolve them? I think history shows quite clearly that no - the community rejects such applications until the CA is able to resolve, and demonstrate they resolve them. The fact that this causes delays to the CA's application therefore incentivizes getting it correct.
>
> If the view is that once you're in, there's a higher bar to get out, then it sets a double standard as to how the program is maintained, and one that I fear may put users at risk.
>

>> As noted above, no agreement has been reached. However, as the person
>> who took a meeting with Qihoo's Head of Security, who will now chair
>> StartCom, I feel that he does understand the issues and I am willing to
>> give his chairmanship and Inigo Barreia's CEO-ship an opportunity to
>> demonstrate they can run a CA well. Inigo's track record at Izenpe is
>> good - I'm not aware of any incidents involving them.
>

> No, but it suggests that if you play the game of organizational structuring, you can reduce risk of consequence. It suggests that, rather than integrate systems (such as Symantec has done with brands, or as Entrust is doing with AffirmTrust), you maintain them as "arms-distance" (legally speaking) entities, while the elements that the public cannot see are shared. Is that uncertainty worth introducing into the ecosystem? Is it something we already accept? I'm not sure, but I'm quite uncomfortable with the implications of the line of thinking.

I think that the plans to both change the ownership structure and
operational control should be seen a positive steps and the beginning
of the process for applying to rejoin as two separate CAs.

"Organizational Structuring", as you put it, is very relevant to
determining whether two entites should be treated as independent when
it comes to their membership in the Mozilla CA program. We know that
Thawte, Inc., GeoTrust, Inc. and Symantec Corporation are each legal
entities (see the EV certs on the websites for confirmation). However
they have clearly decided to operate as one, based their interactions
with Mozilla.

On the other hand, I think there are legitmate indepedence cases where
two or more CAs share some part of their operations. For example,
EJBCA from PrimeKey is apparently a commonly used CA software. I
multiple CAs in the Mozilla program use this and effectively share
development costs by paying PrimeKey to develop the software. I'm
sure there are also cases where two CAs use the same data center or
the same HSM provider. Just because there is shared technical
infrastructure does not mean there is not independence.

I think the proposal from 360 to operate WoSign and StartCom as
separate subsidiaries is interesting and something that is well worth
reviewing if/when they apply to rejoin the program. However that does
not change the past. WoSign and StartCom were, at least as of a month
ago, under common control with WoSign owning and directing operations
of StartCom. Therefore I think they must be treated as one when
reviewing what actions to take as a result of their past behavior.

Thanks,
Peter

[Matt Palmer]

On Sun, Oct 09, 2016 at 08:47:59AM -0700, Peter Bowen wrote:
> I think the proposal from 360 to operate WoSign and StartCom as
> separate subsidiaries is interesting and something that is well worth
> reviewing if/when they apply to rejoin the program. However that does
> not change the past. WoSign and StartCom were, at least as of a month
> ago, under common control with WoSign owning and directing operations
> of StartCom. Therefore I think they must be treated as one when
> reviewing what actions to take as a result of their past behavior.

This is my stance, too. StartCom and WoSign have shared, and currently
share, technical, administrative, and management functions. If their
operations are, in the future, functionally separated, then they can be
considered for reinclusion separately. However, for the purposes of what to
do about them over *past* actions, when they were a single operational
entity, their actions should be considered as such.

- Matt

[谭晓生]

I also said that the official website, ordering system, certificate management system are different and independent, which is the major cause of the bugs from technical perspective, that’s why Wosign suffered the incidents of bugs but StartCom haven’t.
The validation team, customer care team and tech support team are also independent, that is important for the quality control for the business, that’s also the important reason that StartCom did well except the 2 backdated certificates that instructed by Richard Wang directly.
StartCom as a CA for 17 years, contributed more or less to the industry and community, we do hired an in-proper person to manage the company and it have been fixed, fortunately, the ordering process, CMS process still keep the same with the original one of StartCom, we are changing the software soon, the time table will be released in this week.
Please give a chance to StartCom.

Thanks，
Xiaosheng Tan



在 2016/10/10 上午6:43，“dev-security-policy 代表 Percy”<dev-security-policy-bounces+tanxiaosheng=360...@lists.mozilla.org 代表 percy...@gmail.com> 写入:

Tan said, for StartCom and WoSign’s infrastructure, the PKI servers were/are shared, the CRL/OCSP, TSA code were cloned and the StartCom and WoSign shared the software development team.

Also some management team are shared I assume since Richard Wang approved Tyro's backdated cert from StartCom.

As we saw most problems discovered are either due to software development(issue F,H,L,N,V) or management (issue S,P,R). And those team were shared between WoSign and StartCom at the time of the incidents. Consequently, at the time of the incidents, they're the same entity with regards to those issues. So I agree with the opinion that " If their

operations are, in the future, functionally separated, then they can be
considered for reinclusion separately. However, for the purposes of what to
do about them over *past* actions, when they were a single operational
entity, their actions should be considered as such. "

[in...@matthijsmelissen.nl]

On Saturday, October 8, 2016 at 8:18:09 AM UTC+2, uri...@gmail.com wrote:
> Did anyone ever determine if "Andy Ligg" is in fact a real person?
> (As discussed here
> https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/0pqpLJ_lCJQ/7QRQ7oqGDwAJ )

I believe Andy Ligg is a pseudonym of Richard Wang.

Have a look at this Bugzilla thread: https://bugzilla.mozilla.org/show_bug.cgi?id=851435 At 2015-03-12 08:43:09, some information related to Wosign is posted on behalf of Andy Li. About two minutes later, the exact same message is posted by Richard Wang. I can only understand that as a case of Richard Wang mixing up two pseudonyms, accidentally using the Startcom pseudonym Andy Li for Wosign matters. To me this implies Andy Li and Richard Wang are the same person.

>From Andy Li to Andy Ligg is likely a step to further Europeanize the name, inspired by the name of Eddy Nigg.

-- Matthijs

[Gervase Markham]

I don't believe this aspect of things is worth spending time on. However:

On 10/10/16 09:44, in...@matthijsmelissen.nl wrote:
> On Saturday, October 8, 2016 at 8:18:09 AM UTC+2, uri...@gmail.com
> wrote:
>> Did anyone ever determine if "Andy Ligg" is in fact a real person?
>> (As discussed here
>> https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/0pqpLJ_lCJQ/7QRQ7oqGDwAJ
>> )
>
> I believe Andy Ligg is a pseudonym of Richard Wang.
>
> Have a look at this Bugzilla thread:
> https://bugzilla.mozilla.org/show_bug.cgi?id=851435 At 2015-03-12
> 08:43:09, some information related to Wosign is posted on behalf of
> Andy Li.

This Bugzilla account was created in November 2014, presumably in order
to file this bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1106390

The email address associated with it, as anyone with a Bugzilla account
can see, is wosign at outlook dot com. Therefore, the Andy Li in
Bugzilla (not the same name as Andy Ligg, of course) claims to be
connected to WoSign, and was so long before they acquired StartCom.

Gerv

[Gervase Markham]

Hi Ryan,

I agree with your five tenets. And you ask a very important question:

On 07/10/16 18:43, Ryan Hurst wrote:
> The problem is that this sets a dangerous precedent. Let’s assume a
> similar situation happens in the future with another CA who owns
> multiple brands. Would you ignore the violations of the rules and
> allow them to carve off one brand because you liked who they would
> let manage it if you do?
>
> I would hope the answer is no.

I think the answer is that it depends on what happened.

What is now Symantec, ex-Verisign, has bought several CAs over the
years. Thawte was fully integrated; GeoTrust, AIUI, was not - they still
have separate issuing systems. Therefore, if there was a catastrophic
failure of technology in those GeoTrust systems, there would at least be
a case for any sanctions to apply only to GeoTrust-operated roots. If
(hypothetically!) there was evidence of conspiracy to misissue by the
common management, there would be little case for treating the brands
differently.

As Xiaosheng Tan has posted, the technical situation with StartCom and
WoSign is complex. Some systems are shared (cloned code), some are not.

> I would say that holding them equally accountable is the right thing
> to do, since for the time in question, they were equivalently managed
> and operated.

See my other message for why I don't think that is totally true. Many of
the incidents on the list, including the major cert-to-the-wrong-person
Issues L and N, occurred before WoSign bought StartCom. StartEncrypt had
a hole allowing this, although StartCom did have additional "high value
domain" checking and no-one actually achieved it. Opinions may
reasonably vary on whether this is equally as bad as Issue N.

Gerv

[Gervase Markham]

On 09/10/16 23:43, Percy wrote:
> Tan said, for StartCom and WoSign’s infrastructure, the PKI servers
> were/are shared, the CRL/OCSP, TSA code were cloned and the StartCom
> and WoSign shared the software development team.
>
> Also some management team are shared I assume since Richard Wang
> approved Tyro's backdated cert from StartCom.
>
> As we saw most problems discovered are either due to software
> development(issue F,H,L,N,V) or management (issue S,P,R). And those
> team were shared between WoSign and StartCom at the time of the
> incidents.

That's not so for issues F, H, L, N or P. These all happened before the
date when WoSign took legal ownership of StartCom (Nov 1st 2015) and
before the technical changes to use some WoSign code (Dec 18-22 2015). R
relates to the purchase; S and V were afterwards.

Gerv

[Gervase Markham]

Hi Xiaosheng.

On 09/10/16 14:54, 谭晓生 wrote:
> There are 5 components of StartCom’s business supporting software:

It might be useful if you were to explain what function in the
certificate issuance process is performed by each of these five components.

> 3. PKI – signing service
> Code: Same code with WoSign’s one.
> Server: Shared Server.
> Location: The primary one is hosted in Qihoo 360 head quarter’s data center in Beijing since Dec 2015, there is a backup server in Wosign’s office in Shenzhen.
> Business Process: Same

Presumably the fact that this code is shared with WoSign explains why
the StartCom serial numbers changed to be "WoSign-style" in December 2015.

> 5. TSA
> Code: Same code with Wosign’s one.
> Server: Dedicate server, no sharing.
> Location: StartCom TSA: http://tsa.startssl.com is located in Qihoo 360 Los Angeles IDC, WoSign TSA: http://timestamp.wosign.com is hosted in Qihoo 360 China IDC.
> Business Process: Same

Is this server involved in SSL certificate issuance at all?

Gerv

[Gervase Markham]

On 07/10/16 18:25, Andrew Ayer wrote:
> Consider the following hypothetical: Honest Achmed's Used Cars and

I would note in passing that amusing hypotheticals can sometimes work to
obscure the actual point you are trying to make, because it's not clear
which aspects of the hypothetical are pertinent and which aren't. So I
will skip past that and engage with your point.

> During the time that the incidents
> occurred, StartCom and WoSign were for all intents and purposes the
> same company, one wholly owned by the other, both managed by the same
> disgraced CEO, and sharing significant infrastructure. They should
> therefore be treated as the same company when responding to these
> incidents.

This is not correct, for a complete value of "time the incidents
occurred". I believe the evidence shows that WoSign took organizational
control of StartCom in November 2015, and operational control in late
December 2015 when StartCom's systems were taken down for 4 days to
"upgrade" them to use the WoSign infrastructure.

Issues D, F, H, J, L, N (significantly - this is a big one), O, and P on
the WoSign list all occurred before WoSign took control of StartCom.

Issue R refers to the purchase itself, and the lack of disclosure.

Issue T turned out not to be WoSign's fault.

There's no evidence that issue X applies to StartCom infra (although
there is no evidence that it doesn't).

That leaves issue S, the backdated SHA-1 certs (WoSign backdated 60-odd,
StartCom backdated 2) and issue V, StartEncrypt (where StartCom deployed
some terrible WoSign-authored code).

So I think it is not accurate to say that "during the time the incidents
occurred, they were the same company". During the time that _some_
incidents occurred, one wholly owned and effectively controlled the other.

Gerv

[Gervase Markham]

On 07/10/16 17:50, Ryan Sleevi wrote:
> One possible issue with this is that there hasn't been a similar
> question about StartCom's past practices. I think that, up until the
> discussion began, particularly around the backdating of certificates,
> it might have been said the same about WoSign - that is, the view
> that the issues were 'minor'.

Hear ye, hear ye, if anyone do have any evidence of additional
malfeasance by StartCom, ye are to declare it now, or forever hold your
peace!

:-) In all seriousness, we remain open to additional submissions of
evidence which might change how we should view these things.

> We know that the investors and principles have changed, and thus the
> priorities have changed. I think, in the case of many CAs, we might
> say that, prior to their misissuance, they were well-run CAs. But
> isn't that part of the problem? By taking into perspective a
> historical view, rather than an incident-based view, it naturally
> gravitates towards favoring incumbents (those who have been able to
> operate long enough to be considered 'mature' when a misissuance
> happens), and begins to introduce subjective-based weights into a
> process that is, ideally, largely objective.

I don't think you can avoid that entirely. Let's say CA Foo and CA Bar
both simultaneously mis-issue a cert for mozilla.org due to a bug in
their respective home-grown software stacks, and it was discovered a
week later. I think it would be weird to treat those two events exactly
the same in terms of possible sanction even if:

* CA Foo had no history of more minor incidents, but CA Bar had a long one;

* CA Foo had carefully grown its business slowly to avoid scaling
issues, but CA Bar had just served every customer who came knocking;

* CA Foo had, over the years, used best practices for secure code
development, but CA Bar had not;

* CA Foo immediately did root cause analysis and fixed the bug, but CA
Bar merely revoked the certificate and carried on.

I think these would all be relevant factors.

> The implication here - that factors such as management and technology
> bear into decision making - suggest that future inclusion or
> maintenance requests need to consider this. I don't disagree that
> these could be valuable axis' in determining trust, but I think
> historically, Mozilla's Root Store has erred on the side of
> objectivity. For example, we see discussions about particular
> countries views' on wiretaps brought up from time to time, or
> particular companies' associated businesses providing
> wiretap/intercept capabilities, but on the whole, these associations
> are rejected as being influential in the decision to include or not -
> instead, it's based on (ideally) objective evaluations against audit
> criteria and technical standards.

I think that latter example is slightly different, because such
objections are normally lodged with the absence of any evidence
whatsoever that the CA in question has done anything wrong.

> Clearly, there's a set of priorities at play here - what are the
> ultimate goals for Mozilla's Root Store? What are the things to
> prioritize?

I rather liked Ryan Hurst's five principles. :-)

> An argument for distrust is that it strongly signals to the ecosystem
> that there is a serious risk in non-compliance. As such, the greater
> the fiduciary or business risk, the greater justification there is
> for investments into policies, practices, and technologies to
> minimize that risk. If you eliminate any risks, what incentives are
> there to align on proper behaviour, especially given the economic
> structures of CA trust, such that there is no long-term reputational
> brand damage for misissuance, nor is there any way to discover it
> (from people 'outside the know'). That is, to an end user, there's no
> distinction in trust between "Honest Achmed's Gently Used
> Certificates" and "Best CA in the world" - and as such, there's no
> incentives for site operators to consider one or the other.

I agree with that, in general. But I don't think the suggestion of
having one plan for WoSign (the one outlined in my paper, which involves
a year's dis-trust, security audits and other major business disruption)
and another for StartCom (which involves significant expenditure on
their part, but does not involve a period of dis-trust) counts as
"eliminating any risk of non-compliance"!

> An argument against distrust is generally that of user impact.
> Distrust options, at present, vary in proportionality of user impact
> (generally with the size of the CA), and thus the larger the CA, the
> more difficult it can be seen to take action.
>
> But is that Mozilla's justification or set of priorities? It might be
> useful to understand what you (and other module contributors, to be
> particular, since we all may have different views) prioritize.

I try very hard to avoid considering the size of the CA, and I don't
think I've done so in this case. I favour sanctions which can be
implemented independent of CA size. I'm glad that the ecosystem is
moving towards trusted timestamps embedded in certificates, because that
will make it even easier to retire a CA, of any size, without too much
ecosystem impact.

>> I would say the key differences with Symantec are that WoSign's
>> misdemeanours involved actual misissuance to third parties (e.g
>> the github certificates) and provable lying, e.g. about ownership
>> and SHA-1 backdating. I don't see either of those serious
>> components in the Symantec case. This is not to minimise what
>> happened at Symantec, but I do think the WoSign situation is more
>> serious. But then, even before this, I think it was already the
>> case that you took a dimmer view of the happenings at Symantec than
>> I did (your view perhaps being based on more complete
>> information).
>
> This is a disconcerting viewpoint for me. It suggests that you view
> "We've sacked the people responsible" as an acceptable response, so
> long as we can't prove the CA is lying.

I think that's an over-generalisation of my position :-) Whether sacking
people is an acceptable response depends on what has happened.

> In these scenarios, should our position be to 'fail open' (and
> continue to trust the CA, with some set of remediation plan) or 'fail
> closed' (and remove the CA until it can prove it's in compliance).

I think that yoyoing CAs in and out of trustedness is disruptive to
customers. Killing a CA entirely is actually less disruptive. Removing
CAs from trustedness for minor or even medium-severity non-compliance
issues, pending compliance, is not a good strategy IMO.

>> As I said in my previous email, Qihoo's plans are enough, I think,
>> count as "data relevant to our current view" and I think we should
>> at least consider the two CAs separately, although that doesn't
>> preclude reaching the same conclusion for each.
>
> I'm uncomfortable with this, because it's a promise, with no timeline
> for delivery, and significant risk until it's met.

I hear you, but I want to wait to see what they have to say. (Soon, I
very much hope.)

>> As noted above, no agreement has been reached. However, as the
>> person who took a meeting with Qihoo's Head of Security, who will
>> now chair StartCom, I feel that he does understand the issues and I
>> am willing to give his chairmanship and Inigo Barreia's CEO-ship an
>> opportunity to demonstrate they can run a CA well. Inigo's track
>> record at Izenpe is good - I'm not aware of any incidents involving
>> them.
>
> Does this suggest that Mozilla would be willing to meet with CA
> applicants' CSO/CEOs, to get a 'gut sense' about whether they're
> "serious", and then decide to overlook failures to abide by Mozilla's
> inclusion requirements? I ask this, because I think this highlights
> some of the disconcerting disconnect with "Oh, I know Jim, Jim's a
> good guy" sort of approaches.

I think that assessing the trustworthiness of people is an unavoidable
part of assessing the trustworthiness of companies (who are made up of
people). If Richard Wang started a new CA, when it applied to the
Mozilla root program would it make a difference in the process that it
was him running it? I think it would.

>> Indeed, this would be a bad situation - which is why, in my mind,
>> a different deal for StartCom would be predicated on them moving
>> quickly to a position where they share _nothing_ with WoSign,
>> rather than everything.
>
> How, if ever, will the community be assured of this? This is
> something that's quite intangible - which is the very problem.

That's a good question. What do you think is a good way? I'd suggest a
third party system audit, with the auditor chosen by Mozilla and paid
for by StartCom. They could either verify the use of specific COTS
software, or verify the use of original StartCom software, or give a
thorough going-over to anything left that had been touched by WoSign.

Gerv

[Andrew Ayer]

On Mon, 10 Oct 2016 12:11:49 +0100
Gervase Markham <ge...@mozilla.org> wrote:

> > During the time that the incidents
> > occurred, StartCom and WoSign were for all intents and purposes the
> > same company, one wholly owned by the other, both managed by the
> > same disgraced CEO, and sharing significant infrastructure. They
> > should therefore be treated as the same company when responding to
> > these incidents.
>

> This is not correct, for a complete value of "time the incidents
> occurred". I believe the evidence shows that WoSign took
> organizational control of StartCom in November 2015, and operational
> control in late December 2015 when StartCom's systems were taken down
> for 4 days to "upgrade" them to use the WoSign infrastructure.
>
> Issues D, F, H, J, L, N (significantly - this is a big one), O, and P
> on the WoSign list all occurred before WoSign took control of
> StartCom.
>
> Issue R refers to the purchase itself, and the lack of disclosure.
>
> Issue T turned out not to be WoSign's fault.
>
> There's no evidence that issue X applies to StartCom infra (although
> there is no evidence that it doesn't).
>
> That leaves issue S, the backdated SHA-1 certs (WoSign backdated
> 60-odd, StartCom backdated 2) and issue V, StartEncrypt (where
> StartCom deployed some terrible WoSign-authored code).
>
> So I think it is not accurate to say that "during the time the
> incidents occurred, they were the same company". During the time that
> _some_ incidents occurred, one wholly owned and effectively
> controlled the other.

We agree that misissuances occurred under both roots during a time at
which one company wholly owned and controlled the other. One of these
misissuances - backdated SHA-1 certificates - is severe and was
approved by the CEO of WoSign himself. The fact that some of the
incidents occurred before one company controlled the other doesn't
change my point that they were effectively one company at the time they
were seriously mismanaged.

To approach this matter from a different angle: if Qihoo can make a
proposal for continuing to operate the StartCom roots which Mozilla
would find acceptable, why not allow the WoSign roots to continue
operating under the same proposal?

Regards,
Andrew

[谭晓生]

Dear Gervase,

Yes, the certificate issuance process is performed by each of these five components, except, TSA is used for code issuance and PDF issuance, not related with SSL certificates issuance.

Thanks，
Xiaosheng Tan



在 2016/10/10 下午7:11，“Gervase Markham”<ge...@mozilla.org> 写入:

Hi Xiaosheng.

On 09/10/16 14:54, 谭晓生 wrote:

> There are 5 components of StartCom’s business supporting software:

It might be useful if you were to explain what function in the
certificate issuance process is performed by each of these five components.

> 3. PKI – signing service
> Code: Same code with WoSign’s one.
> Server: Shared Server.
> Location: The primary one is hosted in Qihoo 360 head quarter’s data center in Beijing since Dec 2015, there is a backup server in Wosign’s office in Shenzhen.
> Business Process: Same

Presumably the fact that this code is shared with WoSign explains why
the StartCom serial numbers changed to be "WoSign-style" in December 2015.

> 5. TSA
> Code: Same code with Wosign’s one.
> Server: Dedicate server, no sharing.
> Location: StartCom TSA: http://tsa.startssl.com is located in Qihoo 360 Los Angeles IDC, WoSign TSA: http://timestamp.wosign.com is hosted in Qihoo 360 China IDC.
> Business Process: Same

[Gervase Markham]

On 10/10/16 16:47, 谭晓生 wrote:
> Yes, the certificate issuance process is performed by each of these
> five components, except, TSA is used for code issuance and PDF
> issuance, not related with SSL certificates issuance.

Right :-) But can you explain what each component does specifically? E.g.:

1) The user visits the website (component 1) and uploads a CSR.
2) ...
3) ...

Then it would be clear what particular steps are currently fulfilled by
code from StartCom, what by code from WoSign, and so on.

Gerv

[Nick Lamb]

On Monday, 10 October 2016 12:49:37 UTC+1, Gervase Markham wrote:
> I think that's an over-generalisation of my position :-) Whether sacking
> people is an acceptable response depends on what has happened.

I'm very doubtful that it is ever really relevant to the relying parties or trust stores. It might make good business sense to terminate somebody, but the responsibility for the problem always lies with the organisation, never an individual no matter how senior. We need to be quite firm in making that point.

The failure is an organisational failure, even if one identifiable individual is the only one to actively do bad things, the organisation failed to detect those things in a timely fashion and tell us about them. Firing people, whether it's a security guard or a CEO doesn't fix that. The consequences have to be visited on the organisation, and it must not be possible to "dodge" those consequences through a paperwork exercise such as "firing" an employee.

> I think that yoyoing CAs in and out of trustedness is disruptive to
> customers. Killing a CA entirely is actually less disruptive. Removing
> CAs from trustedness for minor or even medium-severity non-compliance
> issues, pending compliance, is not a good strategy IMO.

Agreed that Yoyoing _keys_ is annoying for the subscribers and the relying parties, without necessarily achieving very much, although I don't think we should entirely rule it out. Don't agree about the CA itself.

I don't see why we'd want to trust the existing StartCom keys. Why wouldn't someone who directly lies to Mozilla's investigators and to their own employer also lie about the integrity of the keys? No HSM is impervious to attack by its custodians, the protection in an HSM is against inadvertent or momentary compromise, not a malicious actor with total physical control over a prolonged period of time.

Would anybody here _seriously_ be shocked to read next month that a black hat group is auctioning some StartCom private keys ? On the evidence available we have to assume that the keys underpinning both WoSign and StartCom may turn out to be compromised, which surely means if StartCom is to be resuscitated so that QiHoo 360 can recover some of their investment, they need to generate new roots and start over mathematically as well as from a manpower point of view.

> I think that assessing the trustworthiness of people is an unavoidable
> part of assessing the trustworthiness of companies (who are made up of
> people). If Richard Wang started a new CA, when it applied to the
> Mozilla root program would it make a difference in the process that it
> was him running it? I think it would.

I doubt Mozilla's ability to reliably identify whether Richard Wang exercises effective control over any particular CA. It has been our practice from the outset to allow sovereign entities, private companies and public companies to operate Certificate Authorities, from almost anywhere in the world.

Sovereign entities are opaque basically everywhere, the effective control over such a "government" CA may lay with a civil servant, an appointed official, or an elected official, and I don't believe Mozilla asks for or receives any notification if that changes, as it must have done many times for the CAs in the trust store today.

Private companies are also opaque. In most of the world they're not obliged to disclose who really exercises control. In places where they notionally are obliged to disclose this, many either lie or obfuscate their answers, for example citing control as lying with another private company, based somewhere that has no disclosure rules or with an opaque legal trust operated by lawyers who'll just tell you client confidentiality stands in the way of answering.

Public companies are at least obliged in most of the world to tell you of their largest shareholders and senior executives. Nevertheless it remains possible for practical day-to-day control to lie with someone who isn't listed on paper, so long as large shareholders either don't suspect this or are complicit.

It would be great if Mozilla _did_ know the key individuals behind every CA (as opposed to having contact details which may turn out to be for a "mere employee"). But I suspect that getting to there from here would be difficult. In particular I suspect that because Mozilla is physically a English-language biased US-based outfit any mechanism by which Mozilla tried to obtain confidence in the identities of the people behind each CA would be very open to the charge of trying to shut out non-English non-US groups from a global Internet.

[Kathleen Wilson]

I greatly appreciate the significant amount of effort that you all have been putting into this investigation and discussion.

As Gerv pointed out, since I am Mozilla's CA Certificate Module owner, I have the responsibility of making some decisions... I am continuing to mull over all of your input in this discussion forum.

I would like to remind everyone that when making decisions about what to do about CA mis-issuance, it is expressly *not* a goal for me to mete out punishment. Rather, my primary goal is to help keep end-users safe, based on the information that is available.

Thanks,
Kathleen

[Ryan Sleevi]

On Monday, October 10, 2016 at 11:39:19 AM UTC-7, Kathleen Wilson wrote:
> I would like to remind everyone that when making decisions about what to do about CA mis-issuance, it is expressly *not* a goal for me to mete out punishment. Rather, my primary goal is to help keep end-users safe, based on the information that is available.

Kathleen,

Even as a module peer, this reply somewhat surprises me. While I understand the desire to avoid punitive treatment, at the same time, the entire functioning of the ecosystem is based on the rather intangible property of "trust", which we attempt to quantify through both objective policies and through the evaluations of third-parties, at the individual level, but also through the lens of economic game theory.

That is, if we accept that the sole role of the program is to respond to ensure incidents are contained, then that suggests that there is zero incentive for a CA to comply with Mozilla's policies in general, because only the most serious infractions might necessitate distrusting a CA.

Consider, for example, the comparison of this discussion (with respect to StartCom's misissuance) with that of CNNIC's, a CA which Mozilla moved to distrust. CNNIC, as you know, had an otherwise unblemished record, until a lapse in policies by senior management resulted in the issuance of a CA certificate to MCS Holdings. However, by the time Mozilla took action, the certificate issued to MCS Holdings was already expired - it's ability to cause damage was already constrained.

If we take the view that the primary goal is to keep end users safe, then we could say at the time of expiration, that goal was met, and no further action was necessary. The one incident that CNNIC had done had been resolved (and, as CNNIC's management itself attested, was done in a time limited fashion specifically to reduce risk and ensure compatibility for Mozilla clients in a future sub-CA arrangement).

Following that incident, CNNIC's management was confident they understood the issue, and its seriousness, and yet Mozilla still went ahead. Under what logic can we attribute that to?

My own belief is that there is an aspect to keeping end-users safe, much like there is with respect to enforcing laws - that is, without consequence, there is no rule of law, and thus there is, purely from an economic perspective, zero incentive to abide by the rules of inclusion with Mozilla's program. It's only because the spectre of consequence looms over CAs that there is an incentive to abide by the policies - failure could result in distrust, which could result in a disruption of business.

While understanding and having a remediation plan is important, we don't exactly practice a judicial system in which the guilty party proposes their sentence - again, because the economic incentives there are to take the least impactful operation.

I do hope you consider your response in this light - that to take no action, as proposed, would be a strong signal to the ecosystem - both of CAs and to Mozilla's users - that any CA who callously and crassly violates Mozilla's policies can escape without (meaningful) consequence, provided that they put the right people 'in front', or provide the necessary legal structuring to avoid detection.

Though it may be argued that the proposed remediation for StartCom, put forward by Gerv, is not "without consequence," on a purely economic matter, it really largely is. As seen during the discussion of the management structure, the numbers we're talking about in terms of overall profits and revenues are measured in billions of dollars, and while this might represent some cost to achieve, it allows a full recognition of profits throughout the period, and avoids any meaningful sanction or stigma. This, in turn, can be seen as the base 'cost to violate' - and many CAs, particularly those with state backing, could easily absorb such costs.

I'm trying to avoid too many political parallels, but one might consider, say, the response to the global banking crisis and corruption, and whether or not the sanctions - designed to protect consumers by censuring inappropriate behaviour - meaningfully accomplish that. Likewise, it might be useful to compare such Scandavian models of proportionate impact - http://www.theatlantic.com/business/archive/2015/03/finland-home-of-the-103000-speeding-ticket/387484/ - and their effects on deterring behaviours that put others at risk.

Though we should not strive to have CA's "yo-yo in", as Gerv put it - and I am a strong believer that *any* future trust *must* involve new keys - we know we have a number of failures, through a single shared organization, and I believe that to offer anything short of an equivalent action upon both those roots under the "WoSign" branding and those under the "StartCom" branding - whatever their historic operational separation - is to send a strong signal to CAs that Mozilla's enforcement is in name, but not in practice, and that the rules only apply to those who can't pay their way out of them.

The last piece I'll leave you with, as you thoughtfully consider the (rather uniform) feedback to date, consider research such as http://freakonomics.com/2013/10/23/what-makes-people-do-what-they-do/ - I link to the more accessible version, rather than the scholarly citations. The encouragement of 'small' fines - which I believe is exactly what the proposal for StartCom represents - can easily encourage more of the very behaviour you're expecting to deter. It is only when the risk is truly great that any deterrence begins to be introduced - and the only meaningful consequence of bad behaviour, within the CA ecosystem, is distrust, as many are calling for.

[Matt Palmer]

On Mon, Oct 10, 2016 at 10:33:15AM -0700, Nick Lamb wrote:
> Would anybody here _seriously_ be shocked to read next month that a black
> hat group is auctioning some StartCom private keys ? On the evidence
> available we have to assume that the keys underpinning both WoSign and
> StartCom may turn out to be compromised,

Say what-now? I don't recall anything that suggested private key
*compromise*. The need to roll the keys, from what I can see, is because
the existing chains have done "things" that are shady, and we can never be
sure there isn't more shady things lurking in the shadows. Hence, we
distrust the keys entirely to prevent any of the old shady from leaping out
in a year's time and laying waste to the landscape once again.

- Matt

[Ryan Hurst]

Gerv,

Again, this mail represents my own personal beliefs and does not necessarily represent the beliefs of my employer, Google, or Let’s Encrypt where I am an advisor.

I agree an appropriate response depends on the facts, so as you say, it depends.

I also believe there are a few core questions that are relevant to “what it depends on”, these include:
Is it reasonable for the operational and technical failures StartCom made prior to the acquisition to be handled as a separate incident?
Did the operational changes that occurred after the acquisition impact the trustworthiness of StartCom as an independent entity?
How severe is the failure of both WoSign and StartCom to notify the root programs of the change of ownership?
Should the misrepresentation of facts regarding the acquisition and other issues, by both parties, have an impact on the faith in any claims made by the two organizations?

On the first question, I can see arguments in both directions.

When a company is purchased, you inherit both the assets and liabilities of that organization. This is why due diligence is such an important part of acquisitions. In short, under this line of reasoning, if Qihoo/Wosign failed to do sufficient due diligence as part of the acquisition, this is their problem and not the problem of the WebPKI. In other words, with this line of thinking treating both sets of issues as one “incident” could be seen as reasonable and expected.

The alternative view would be to say that the most severe issues were a function of WoSign’s leadership and technical practices. This, combined with StartCom’s past good practices, might carry sufficient weight to justify special casing the StartCom issues.

I struggle with this second view. To understand why let’s look at DigiCert’s acquisition of the Verizon PKI business. We all know how poorly Verizon managed that infrastructure, it was, a liability to the WebPKI.

I am confident that if DigiCert had not taken on the burden to repair their dysfunction Verizon would have been distrusted. In this respect my view is that DigiCert spent the trust and goodwill they had earned in the past for a grace period to clean up the Verizon mess.

In the case of Qihoo/WoSign/Startcom the prior goodwill is, in what is for all intents and purposes, a non-existent organization (Startcom). I say this because it is now under new ownership and new management. In other words, the new management has no equivalent goodwill to spend.


On the second question, based on Xiaosheng’s email, it seems the CA and OCSP services have been under the administrative and operational control of WoSign since December 2015. It also seems the RA (the CMS) system has been in a shared control situation for what we can only assume is the same period.

These are the material systems covered by Webtrust audits, the others while potentially relevant are arguably not material to the issuance of SSL certificates.

Since the most severe issues boil down to the operational and technical practices of WoSign, and the systems were under the control of WoSign since last year, it seems it was only luck of the draw that saved the involvement of StartCom in the other issues.


On the third question, I would argue that this is the smallest of the identified issues since both organizations were members of the root program, had active WebTrust audits, and contracts in place with various root stores. I say this because I believe that given these facts it is likely Mozilla and Microsoft would have raised no concerns and as a result this would have been a non-issue.

This is not to say their total failure to notify is acceptable, just that the larger issue in my mind is the repeated misrepresentation about this transaction.


On the fourth question, while this is not the technical or contractual requirement of the Mozilla root program, being truthful is the foundation of any good relationship. The “goodwill” one gets in a relationship is always a function of the quality of that relationship.

It is also my understanding Qihoo, WoSign, and Startcom were all voting in the CAB/Forum during this period, in essence, giving one organization three votes. This may have been an oversight but it also puts into question the integrity of these organizations.

As such, it seems to me, that offering goodwill to organizations that has a history of acting in bad faith (on purpose or otherwise) without other mitigating factors sets a bad precedent.


In summary, I am still inclined to say the right response is to treat the two sets of incidents as one. The gestures being made by Qihoo are the right ones to be made, but they do not nullify the past actions.

Instead, I believe, the good faith steps being made by Qihoo to address this situation should be given heavy weight in any resubmission process.

I would add that Mozilla should update its policies to make it clear how important the ownership notification process is to maintaining trust in the WebPKI ecosystem.

Ryan

[Kathleen Wilson]

On Monday, October 10, 2016 at 1:08:24 PM UTC-7, Ryan Sleevi wrote:
> On Monday, October 10, 2016 at 11:39:19 AM UTC-7, Kathleen Wilson wrote:
> > I would like to remind everyone that when making decisions about what to do about CA mis-issuance, it is expressly *not* a goal for me to mete out punishment. Rather, my primary goal is to help keep end-users safe, based on the information that is available.
>

<snip>

> That is, if we accept that the sole role of the program is to respond to ensure incidents are contained,

I did not say that my goal was solely to contain incidents! :-(

I said that my goal is to help keep end-users safe. Here's what I mean by that... If a CA has lost my faith in their ability to uphold Mozilla's policies, then I don't think they should be able to continue issuing certs chaining up to root certs in NSS until they re-gain my faith in their ability to uphold Mozilla's policies.

>
> If we take the view that the primary goal is to keep end users safe, then we could say at the time of expiration, that goal was met, and no further action was necessary.

I completely disagree!

CNNIC lost my confidence in their ability to uphold Mozilla's policies. For me, the resulting action items were not about punishing CNNIC, and not solely about containment of the incident. It was about taking that CA out of the system until they could re-gain my confidence in their ability to meet and uphold Mozilla's policies.

Based on the information that I have seen regarding WoSign, I believe that WoSign intentionally bent the rules in order to continue issuing SHA-1 SSL certs, when they knew full well that was no longer allowed. WoSign has lost my confidence in their ability and intention to follow Mozilla's policies.

> Though we should not strive to have CA's "yo-yo in", as Gerv put it - and I am a strong believer that *any* future trust *must* involve new keys

I think what you are saying is that the CA needs to re-apply for inclusion with new root certificates (not their old root certs). Correct?
If yes, I am inclined towards that idea too.
I've heard that it would cause continuity issues, but I don't get that. Seems to me that *if* the CA were to get their new root approved/included, that they could resume issuing from their new root instead of their old root.

In any case, Gerv's proposal outlined a set of things that the CA would need to do in order to re-apply for inclusion.

I'm inclined to agree with much of Gerv's proposal, but I'm still pondering this: "This distrust would remain for a minimum of 1 year. After that time, WoSign/StartCom may be readmitted to the Mozilla trust program, under the following conditions:..."

Why do we need a minimum of 1 year?
What purpose does that serve?
If they meet all our requirements earlier, why couldn't we discuss it earlier than 1 year?

Cheers,
Kathleen

[Ryan Sleevi]

On Monday, October 10, 2016 at 5:04:14 PM UTC-7, Kathleen Wilson wrote:
> Based on the information that I have seen regarding WoSign, I believe that WoSign intentionally bent the rules in order to continue issuing SHA-1 SSL certs, when they knew full well that was no longer allowed. WoSign has lost my confidence in their ability and intention to follow Mozilla's policies.

I suppose at core question here is whether it's believed StartCom is any different, and why that is. It seems uncontroversial that this is true for WoSign, but the proposal to distinguish StartCom, a wholly owned subsidiary that actively participated in this deception and misissuance, be treated distinctly.

This is where I think many of my arguments center around - the moral hazard it invites for participating CAs in the program seems, to many people commenting, to outweigh the potential benefits of not taking commiserate action against all of WoSign's branded CAs.

> I think what you are saying is that the CA needs to re-apply for inclusion with new root certificates (not their old root certs). Correct?

Yes

> If yes, I am inclined towards that idea too.
> I've heard that it would cause continuity issues, but I don't get that. Seems to me that *if* the CA were to get their new root approved/included, that they could resume issuing from their new root instead of their old root.

CAs would and could address that continuinity by signing their new root with their old (distrusted) root, and only issuing certificates with the new root, while the old root fades into obsolecence.

This offers continuity because the certs issued by new-root could be trusted by clients that only trust old-root, by cross-signing new-root with old-root, while still offering the assurances to the public that old-root can safely be distrusted.

It does create the issue that, in the absence of trusted timestamps or whitelist, there's no way to maintain continuity for old-root's existing certificates, and that's a difficult decision, but in the goal of protecting users, we should take the conservative approach of "assume the worst" - precisely because so much of what goes on in the CA ecosystem is unobserved, even in a world with CT, due to the reliance on auditors whose fiduciary duty is to the CA, not to the public.

> Why do we need a minimum of 1 year?
> What purpose does that serve?
> If they meet all our requirements earlier, why couldn't we discuss it earlier than 1 year?

In general, we've seen CAs hastily react to things, and in the process, introduce new vulnerabilities. We've also seen CAs fail to do the necessary due diligence to holistically understand the failure and to design safe-guards to mitigate the issue.

By setting a time limit, it serves two purposes:
1) As a consistent and neutral deterrent for others to engage in such behaviour, due to the economic and business risk that failing to act in a trustworthy manner would entail
2) As an expression of what the community believes is a reasonable minimum for a successful remediation of the issues in a way that will be thorough and sufficient enough to reconsider trust

[Gervase Markham]

On 10/10/16 23:00, Ryan Hurst wrote:
> I also believe there are a few core questions that are relevant to
> “what it depends on”, these include: Is it reasonable for the
> operational and technical failures StartCom made prior to the
> acquisition to be handled as a separate incident?

I presume you mean "WoSign" here? I'm not aware of significant failures
at StartCom prior to the acquisition. But then you go on to talk about
due diligence in acquisition, so I'm confused. What failures at StartCom
pre-acquisition are you thinking of?

> Since the most severe issues boil down to the operational and
> technical practices of WoSign, and the systems were under the control
> of WoSign since last year, it seems it was only luck of the draw that
> saved the involvement of StartCom in the other issues.

Or that they used a different codebase for the CMS. But saying "it's
just luck" is an un-refutable statement. StartCom was not involved in
most of the issues; many of the ones on the list happened even before
the acquisition. We can only work with the issues we have, not ones that
might have hypothetically happened if the "luck" had been different.

> On the third question, I would argue that this is the smallest of the
> identified issues since both organizations were members of the root
> program, had active WebTrust audits, and contracts in place with
> various root stores. I say this because I believe that given these
> facts it is likely Mozilla and Microsoft would have raised no
> concerns and as a result this would have been a non-issue.
>
> This is not to say their total failure to notify is acceptable, just
> that the larger issue in my mind is the repeated misrepresentation
> about this transaction.

I agree - it was the misrepresentation when directly challenged which
concerned me on a trust level more than the lack of notification.

> It is also my understanding Qihoo, WoSign, and Startcom were all
> voting in the CAB/Forum during this period, in essence, giving one
> organization three votes. This may have been an oversight but it also
> puts into question the integrity of these organizations.

I think this is a matter for the CAB Forum.

Gerv

[Nick Lamb]

On Tuesday, 11 October 2016 01:04:14 UTC+1, Kathleen Wilson wrote:
> Why do we need a minimum of 1 year?
> What purpose does that serve?
> If they meet all our requirements earlier, why couldn't we discuss it earlier than 1 year?

The exact period of one year is of course arbitrary. However I believe there are two useful things achieved by setting this period a little longer than the minimum achievable

1. This ensures the Certificate Authority's new management are able to plan out their activities over a reasonable period without pressure to bring things forward in order to meet commercial goals. This is the foundation of (we hope) a long-lived successful CA, it's not about rushing a minimum viable product to market with the plan to fix any deficiencies later. An external organisation like Mozilla is better placed to give management this cover than any internal promise from QiHoo 360 although if the arbitrary period is reasonable I hope QiHoo 360 will accept that it's ultimately to everyone's benefit to have this.

2. In this particular case we get to see QiHoo 360 wind down the existing CA safely and carefully in parallel with the new CA being founded. This is another opportunity to demonstrate good will and competence, including reaching out to existing subscribers to inform them of what happened, what Mozilla and QiHoo 360 are doing about it, and what steps they need to take to retain trust from third parties. In the event that during wind-down QiHoo 360 find other issues not previously detected by Mozilla, it's also a chance to act transparently and disclose those, for example if an undisclosed WoSign intermediate CA from 2015 is found on a smart card in somebody's desk drawer, reporting that rather than just quietly incinerating the smart card helps to demonstrate the organisation has learned to tell us about problems, not hide them. Winding down may take a while to complete, and it would be a shame to rush to instantiate a new CA without seeing the old one decommissioned properly.

[Eddy Nigg]

Hi Kathleen,

On 10/10/2016 09:39 PM, Kathleen Wilson wrote:
> I would like to remind everyone that when making decisions about what to do about CA mis-issuance, it is expressly *not* a goal for me to mete out punishment. Rather, my primary goal is to help keep end-users safe, based on the information that is available.

Allow me to add some of my thoughts into the discussion. I've read most
of the comments and arguments made here so far and assume that most
participants have the relevant information so that I don't have to
repeat them again....

I was directly responsible for StartCom for many years and gained some
experience in running a certificate authority, writing policies and
implementations thereof. I've helped to draft important guidelines and
requirements for CAs and in general learned about the mesh of software
vendors, certificate authorities and (web) PKI. I'm probably one of the
faces of this industry and would offer my two cents in this capacity
hereby....

The problematic issue in relation to StartCom is obviously the _two
backdated SHA1 certificates_ - however from the strictly technical point
of view I don't think that the user-base of Mozilla in general and the
relying parties in particular were much more at risk than relying on any
other SHA1 certificate that was obtained legitimately before the 1st of
January 2016. The risks were probably minimal since the certificate
properties besides that were validated correctly.

However, a completely different matter must be considered here - that of
compliance to the requirements set forth by the relevant bodies and
software vendors. Besides the _loss of trust_ in this particular case,
non-compliance happens many times due to _insufficient controls_. Being
it either that the requirements weren't correctly understood (not the
case here), or insufficient controls to prevent such non-compliance and
wrongful certificate issuance.

The remediation and corrections StartCom proposed are significant in
this respect - basically Mr. Richard Wang has been removed from his
position and unfortunately for him is paying a high price for
overstepping his authority. The parent company (WoSign) too has been
released of all its responsibilities and a full separation has been set
into motion.

The choice of Inigo Barreira as the new CEO of StartCom is probably a
good one and we all assume that he wouldn't approve the backdating of
certificates judging from his long-time record....however one of the
immediate tasks of Inigo will be to implement controls that will make
such abuse impossible - not by him and not by anybody else. I believe
this is the _core issue and remediation_ here, which was the failure in
first place.

(Obviously he'll have to review also other areas and implement controls
in case they were lacking or insufficient, something he's doing as we
speak).

But by looking at StartCom's performance besides that, I believe that
some of the voices and arguments haven't been reasonable during this
discussion! Was there a CA certificate compromise? Has StartCom lost
control of its issuance processes? Has StartCom in general failed to
validate certificate properties correctly? Has StartCom lost its ability
to abide and comply to the policies and requirements set forth? Has and
does StartCom present an undue risk to the user-base of Mozilla (and
relying parties in general)?

I believe that none of the above applies which would warrant such
dramatic steps on part of the software vendors and StartCom is generally
operating correctly. The particular failure that did happen can be dealt
with properly, firmly and professionally as proposed; _without knocking
StartCom out of business_. I believe StartCom is still an important part
of today's SSL landscape and it shouldn't be in anybody's interest to
remove it as a viable alternative to current mix of the established
certificate authorities - except if somebody is looking for revenge or
other personal matters....

--
Regards
Signer: Eddy Nigg, Founder
StartCom Ltd. <http://www.startcom.org>
XMPP: star...@startcom.org <xmpp:star...@startcom.org>

[Gervase Markham]

On 11/10/16 01:04, Kathleen Wilson wrote:
> I think what you are saying is that the CA needs to re-apply for
> inclusion with new root certificates (not their old root certs).
> Correct? If yes, I am inclined towards that idea too. I've heard that
> it would cause continuity issues, but I don't get that. Seems to me
> that *if* the CA were to get their new root approved/included, that
> they could resume issuing from their new root instead of their old
> root.

The issue with continuity is this.

Let's say we decide on a 1-year distrust for CA Foo's current root, Root
A, and that they must then re-apply with a new root (Root B). The
timeline, even in the best case for CA Foo where the dis-trust lasts
only exactly one year, might look like this:

31st October 2016: Firefox stops trusting CA Foo Root A certs with
notBefore after 2016-10-31
1st June 2017: CA Foo re-applies for inclusion with Root B
31st Sept 2017: Root B included in NSS
31st October 2017: CA Foo Root B is shipped in Firefox

In order to try and make their certs work in more places, such as every
browser issued before 31st October 2016, CA Foo cross-signs Root B with
Root A, and tells sites to put the cross-cert in the SSL handshake.

However, all copies of Firefox distributed between 31st October 2016 and
31st October 2017 will not trust their new certs. They won't chain
properly up to Root A via the cross-cert, because the notBefores are too
late and the block will stop them. And they won't chain up to Root B,
because these Firefoxes know nothing about Root B. So there is a
continuity gap, and a ubiquity issue for CA Foo until all those
Firefoxes stop being used - which might be years.

Does that make sense?

This is not to say we should make one decision or the other, but it is
to point out that unless you say "you have to have new roots, but you
can generate them and we will include them straight away", you are going
to have a gap. And if you do say that, there's a problem because surely
the problem is you don't trust their infra/management/whatever right
now, so why would you let them generate new roots and fast-track them
into your product?

I guess you could ask a trusted competitor to generate them on new
hardware and hold the HSMs securely, then you include the roots in
Firefox straight away, and then only tell the competitor to release the
HSMs to CA Foo once CA Foo had completed inclusion. But that seems
complicated!

> In any case, Gerv's proposal outlined a set of things that the CA
> would need to do in order to re-apply for inclusion.
>
> I'm inclined to agree with much of Gerv's proposal, but I'm still
> pondering this: "This distrust would remain for a minimum of 1 year.
> After that time, WoSign/StartCom may be readmitted to the Mozilla
> trust program, under the following conditions:..."
>
> Why do we need a minimum of 1 year? What purpose does that serve? If
> they meet all our requirements earlier, why couldn't we discuss it
> earlier than 1 year?

This is a good question; I remember writing an answer to someone who
asked this, but I can't remember where, so I'll repeat it here.

I picked the 1 year minimum for a number of reasons. The primary one
was, with WoSign-the-company and WoSign-the-infrastructure in mind, it
was clear to me that both the management and the technical infra had
lost my confidence. Restoring that was going to require changes of
personnel at many levels, a complete internal system audit, some
significant rewriting and testing of code, and then lastly an external
audit by a Mozilla-chosen auditor - with all the earlier steps being
necessary for them to have a hope of passing the last one. I felt that
if the WoSign knew that the 1-year distrust was happening anyway, there
would not be a temptation to rush this process.

My suspicions on this were borne out when, in the meeting in London,
Richard Wang (perhaps not being quite on the same page as everyone else
in the meeting) suggested that WoSign was ready to undergo the external
security audit now and prove their competence. Both I and the Qihoo
representatives politely suggested that this was not an appropriate
course of action!

1 year gives WoSign, if it ends up wishing to continue in business with
its own infrastructure, some guaranteed breathing room to take the
actions necessary to restore trust in a calm and unhurried manner, and
removes the temptation to cut corners.

Secondarily, it was compatible with the ban given to CNNIC, whose crimes
were arguably less severe.

Thirdly, while yanking a CA entirely can be done using OneCRL very
quickly, changing the code to trust and un-trust CAs by notBefore takes
time. Therefore, short bans make less sense. Hence my comment about the
inadvisability of "yo-yoing".

Gerv

[Gervase Markham]

On 11/10/16 02:55, Ryan Sleevi wrote:
> CAs would and could address that continuinity by signing their new
> root with their old (distrusted) root, and only issuing certificates
> with the new root, while the old root fades into obsolecence.
>
> This offers continuity because the certs issued by new-root could be
> trusted by clients that only trust old-root, by cross-signing
> new-root with old-root, while still offering the assurances to the
> public that old-root can safely be distrusted.

What do you say to my point that in practice there would be a set of
browsers which trusted neither - those released during the dis-trust period?

Gerv

[Gervase Markham]

Hi Eddy,

While I have sympathy with what you say, your analysis is incomplete in
one respect.

On 11/10/16 09:41, Eddy Nigg wrote:
> The problematic issue in relation to StartCom is obviously the _two
> backdated SHA1 certificates_

There is also the case of StartEncrypt. While no known
cert-to-wrong-person misissuance occurred because the researchers in
question used domains they already controlled to prove their point, but
there seemed to be multiple holes by which this might be possible.

https://www.computest.nl/blog/startencrypt-considered-harmful-today/

Of course, people can reasonably disagree on the seriousness of the
issue (standalone, and by comparison with e.g. WoSign issue N), and it
is true that StartCom took this codebase wholesale from WoSign, but it
seems incomplete to leave this out entirely.

> But by looking at StartCom's performance besides that, I believe that
> some of the voices and arguments haven't been reasonable during this
> discussion! Was there a CA certificate compromise? Has StartCom lost
> control of its issuance processes? Has StartCom in general failed to
> validate certificate properties correctly? Has StartCom lost its ability
> to abide and comply to the policies and requirements set forth? Has and
> does StartCom present an undue risk to the user-base of Mozilla (and
> relying parties in general)?

Eddy: does StartCom currently have any fully-automated certificate
issuance mechanisms, or does every certificate request pass by human
eyes before issuance?

Gerv

[谭晓生]

Process to apply a SSL certificate of StartCom:
Step 1. StartCom customer sign-in his/her account on official website of StartCom;
Step 2. Customer do the domain validation via “Validations Wizard”;
Step 3. PKI validation system send the verification code to domain name whois admin email, the subscriber pastes the verification to the validation page;
Step 4. If the domain validated successfully, subscriber using “Certificates Wizard” to choose the validated domain and post CSR to CMS for pending process;
Step 5. If the domain name is not in the “Manual Review list” and subscriber apply for a free SSL certificate, the order will be sent to the PKI server to issue the Certificate automatically, go to Step 8;
Step 6. If the domain name is in the “Manual Review List” or subscriber apply for OV and EV certificate, the order will be sent to CMS for manual process;
Step 7. CMS (Certificate Management System) is the internal order process system to review the order, do the identify validation, approve the order to PKI for pending issuance;
Step 8. The PKI signing server will get the serial number from serial number generator;
Step 9: PKI use the right intermediate CA key to sign the certificate, and return the issued certificate to CMS;
Step 10. CMS push the certificate to ordering system, and send email to user to retrieve the certificate from the official website.

The Official website & Ordering System, CMS and PKI system are involved in the process, the process is different with Wosign’s one.

CRL/OCSP distribution
1. Once the certificate is issued, the certificate serial number and other related info will post to OCSP source server for CDN distribution;
2. PKI signing the CRL at fixed period and send it to CRL source server for CDN distribution.

Thanks，
Xiaosheng Tan



在 2016/10/11 上午12:10，“Gervase Markham”<ge...@mozilla.org> 写入:

On 10/10/16 16:47, 谭晓生 wrote:

> Yes, the certificate issuance process is performed by each of these
> five components, except, TSA is used for code issuance and PDF
> issuance, not related with SSL certificates issuance.

[Nick Lamb]

On Tuesday, 11 October 2016 09:47:20 UTC+1, Gervase Markham wrote:
> I guess you could ask a trusted competitor to generate them on new
> hardware and hold the HSMs securely, then you include the roots in
> Firefox straight away, and then only tell the competitor to release the
> HSMs to CA Foo once CA Foo had completed inclusion. But that seems
> complicated!

Some of the major root trust stores (e.g. Microsoft, Apple) also operate their own root CA, which they include in that store, for internal purposes at least. I believe none of them is trusted by another root trust store but in principle they could be.

Mozilla could choose to do that too, and agree that when a new CA is added to NSS it will use the Mozilla CA (trusted but never used to issue end entity certificates) to cross sign the new CA. The resulting certificate could be included in chains for the new CA's end entity certificates, allowing them to be trusted by older Firefox versions immediately.

Does that work?

[Peter Bowen]

On Tue, Oct 11, 2016 at 7:08 AM, Nick Lamb <tiala...@gmail.com> wrote:
>
> Some of the major root trust stores (e.g. Microsoft, Apple) also operate their own root CA, which they include in that store, for internal purposes at least. I believe none of them is trusted by another root trust store but in principle they could be.

I believe that both of the ones you list have full WebTrust for CA and
WebTrust for BR audits and have CAs cross-signed by other CA
operators. It is possible they have additional non-cross certified
roots included, but I think all are WebTrust audited.

Thanks,
Peter

[Gervase Markham]

On 11/10/16 15:08, Nick Lamb wrote:
> Mozilla could choose to do that too, and agree that when a new CA is
> added to NSS it will use the Mozilla CA (trusted but never used to
> issue end entity certificates) to cross sign the new CA. The
> resulting certificate could be included in chains for the new CA's
> end entity certificates, allowing them to be trusted by older Firefox
> versions immediately.
>
> Does that work?

Technically, it does, but it's not a scalable solution for all root
stores, as each would need their own cross-sign and it would bloat the
number of certs a site would need to send by one per store. Also,
Mozilla would need to spin up the infra and security (or pay someone
else to host it) for the HSM for such a seriously valuable key. That's
not something that would be an easy sell.

Gerv

[Ryan Hurst]

On Tuesday, October 11, 2016 at 1:28:42 AM UTC-7, Gervase Markham wrote:

> I presume you mean "WoSign" here? I'm not aware of significant failures
> at StartCom prior to the acquisition. But then you go on to talk about
> due diligence in acquisition, so I'm confused. What failures at StartCom
> pre-acquisition are you thinking of?

No I meant Startcom. I was not referring to a specific issue. I was simply stating that when you buy something, you get the good, and the bad and that includes you tainting your purchase with your own issues. This is not unique to the WebPKI ecosystem, this is the way acquisitions work.

>> Or that they used a different codebase for the CMS. But saying "it's
>> just luck" is an un-refutable statement. StartCom was not involved in
>> most of the issues; many of the ones on the list happened even before
>> the acquisition. We can only work with the issues we have, not ones that
>> might have hypothetically happened if the "luck" had been different.

Given how manual the process was and that the keys were under both logically and physically the control of WoSign the different code base is somewhat immaterial. Control is control.

My statement about “luck” is an attempt to speak to that.

>> I think this is a matter for the CAB Forum.

I think that is a bad position to take. Certainly the trustworthiness of an operator is of paramount importance to Mozilla when considering to make an accommodation on behalf of the issuer?

[Peter Kurrasch]

I agree that it probably is not worth dwelling on the "Andy Ligg question" in particular but I think there is a broader issue at play which is worth addressing: deception.

I think there is ample evidence that WoSign engaged in a deliberate, persistent, and extensive campaign of deception committed against many different parties within the PKI ecosystem. In some cases the deception was committed by Richard Wang himself while in other cases it's less clear if the perpetrator was Richard or someone under his supervision.

I'd like to see something included in the summary report, although I'm the first to admit I don't know how best to do that. It seems to me the level of deceptive activity here falls well outside the norm of something more innocent, like being coy to protect a company's proprietary information. I don't think we've seen anything like this from other CA representatives in this forum.

If someone reads the report without having also participated in these discussions it's possible that he or she will not appreciate the difficulty we've had at times in getting at the truth of what has transpired. In fact, I think we continue to struggle to understand the extent of damage committed precisely because of the deception.

Again, I'm not sure the best way to capture this whole idea but I think it's something that should not be left unsaid. 


  Original Message  
From: Gervase Markham
Sent: Monday, October 10, 2016 5:45 AM
To: in...@matthijsmelissen.nl; mozilla-dev-s...@lists.mozilla.org
Subject: Re: WoSign: updated report and discussion

I don't believe this aspect of things is worth spending time on. However:

On 10/10/16 09:44, in...@matthijsmelissen.nl wrote:
> On Saturday, October 8, 2016 at 8:18:09 AM UTC+2, uri...@gmail.com
> wrote:
>> Did anyone ever determine if "Andy Ligg" is in fact a real person?
>> (As discussed here
>> https://groups.google.com/forum/#!msg/mozilla.dev.security.policy/0pqpLJ_lCJQ/7QRQ7oqGDwAJ
>> )
>
> I believe Andy Ligg is a pseudonym of Richard Wang.
>
> Have a look at this Bugzilla thread:
> https://bugzilla.mozilla.org/show_bug.cgi?id=851435 At 2015-03-12
> 08:43:09, some information related to Wosign is posted on behalf of
> Andy Li.

This Bugzilla account was created in November 2014, presumably in order
to file this bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1106390

The email address associated with it, as anyone with a Bugzilla account
can see, is wosign at outlook dot com. Therefore, the Andy Li in
Bugzilla (not the same name as Andy Ligg, of course) claims to be
connected to WoSign, and was so long before they acquired StartCom.

Gerv
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

[Jakob Bohm]

On 09/10/2016 15:54, 谭晓生 wrote:
> Dear All,
> This is the information that would be released by Inigo in the coming week, Percy asked me to answer the question, so, it is here:
>
> ...

>
> 3. PKI – signing service
> Code: Same code with WoSign’s one.
> Server: Shared Server.
> Location: The primary one is hosted in Qihoo 360 head quarter’s data center in Beijing since Dec 2015, there is a backup server in Wosign’s office in Shenzhen.
> Business Process: Same
>

Wait: Does this mean that WoSign has a copy of the StartCOM root
private key at the WoSign office?

Are there any technical obstacles to ensure that Richard Wang or his
underlings have not used that key in ways not logged in the log files
and databases now controlled by the new StartCOM?


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[谭晓生]

The HSM is stored offline, in the Vault of Qihoo 360’s head quarter, a little bit surprised by this question, I don’t know if there other CAs put their Root Certificates online?
If anybody have evident to say “Wosign have the private key of StartCom”, please show us here.

Thanks，
Xiaosheng Tan


在 2016/10/13 上午6:49，“dev-security-policy 代表 Percy”<dev-security-policy-bounces+tanxiaosheng=360...@lists.mozilla.org 代表 percy...@gmail.com> 写入:

" PKI – signing service
> Code: Same code with WoSign’s one.
> Server: Shared Server.
> Location: The primary one is hosted in Qihoo 360 head quarter’s data center in Beijing since Dec 2015, there is a backup server in Wosign’s office in Shenzhen.
> Business Process: Same
"

As Jakob said, WoSign might have StartCom's private key. Xiaosheng Tan, perhaps you can clarify what the backup server process and whether HSM is "backed up" as well.

[Gervase Markham]

On 13/10/16 01:40, Percy wrote:
> (Hmm, my previous comment about two faced WoSign disappeared from
> Google group probably due to anti-spam. Gerv, can you recover it for
> me?)

I have that message via the news interface, so it did get posted. It's
not in the spam filter.

Gerv

[Eddy Nigg]

On 10/11/2016 11:57 AM, Gervase Markham wrote:
>
> There is also the case of StartEncrypt. While no known
> cert-to-wrong-person misissuance occurred because the researchers in
> question used domains they already controlled to prove their point, but
> there seemed to be multiple holes by which this might be possible.

I haven't forgotten it and mentioned that Inigo has several tasks at hand:

"/... he'll have to review also other areas and implement controls in
case they were lacking or insufficient, something he's doing as we speak/"

This includes obviously development cycles and other areas, even if no
issues have been detected or reported.

> Of course, people can reasonably disagree on the seriousness of the
> issue (standalone, and by comparison with e.g. WoSign issue N), and it
> is true that StartCom took this codebase wholesale from WoSign, but it
> seems incomplete to leave this out entirely.

It wasn't my intention to ignore it, but I understand that this issue
has been quickly contained at that time.

>
> Eddy: does StartCom currently have any fully-automated certificate
> issuance mechanisms, or does every certificate request pass by human
> eyes before issuance?

Generally speaking it's semi-automated with a flagging system that
forces about 20% of all lower level certificates for a manual review and
approval by the verification team. Of course EV and code signing
certificates are issued only manually. The rest is issued automatically.

[Richard Wang]

Percy,

I think your English is too bad! And the English translation from Chinese is very bad.
The report said: "Richard Wang will be relieved of his duties as CEO of WoSign", it is "will be". Now I am the Acting CEO of WoSign till the new CEO arrives. Anybody that is interesting in this hot position of the CEO of WoSign can send your Resume to me.

WoSign will resell other trusted CA's SSL certificate to our customers to provide best product and best service to our customers.

Again, your English translation from Chinese is very bad, I copy the Chinese sentence here that I wish somebody can translate it correctly.
沃通(WoSign)能取得今天的成绩，除了需要感谢公司创始人/CEO王高华先生带领全体员工一起努力拼搏和全力奉献以外，更应该感谢多年来广大用户对沃通(WoSign)品牌的厚爱，我们对这样的信任倍感珍惜，将不负使命克服各种困难为广大用户提供更好的产品和更优质的服务。

Best Regards,

Richard Wang

-----Original Message-----
From: dev-security-policy [mailto:dev-security-policy-bounces+richard=wosig...@lists.mozilla.org] On Behalf Of Percy
Sent: Thursday, October 13, 2016 8:25 AM
To: mozilla-dev-s...@lists.mozilla.org
Subject: Re: WoSign: updated report and discussion

WoSign has so far announced nothing about those incidents or immediate distrust (Apple and Mozilla) to its end users. On the contrary, WoSign had a press release dated Oct 8th (https://www.wosign.com/news/netcraft-ssl-oct.htm) titled "WoSign SSL certs reaches almost 50% market share in China". In the press release, it stated that "WoSign's achievement today is due to company founder and CEO Richard Wang leads all staff to be dedicated". WoSign is depicted as this positive, strong growing company. Nothing about its distrust in the immediate future is mentioned.

In Oct 7th, in the incident report in English, WoSign admitted multiple intentional mistakes and deceptions (https://www.google.com/url?q=https%3A%2F%2Fwww.wosign.com%2Freport%2FWoSign_Incident_Report_Update_07102016.pdf&sa=D&sntz=1&usg=AFQjCNGRzAxwYrEEiA_SN5gfcsftSst0nA) and that the CEO Richard Wang to be relieved of its duties.

I'm calling WoSign out on this two-faced behavior towards Chinese end users and foreign security researchers.

[uri...@gmail.com]

>WoSign will resell other trusted CA's SSL certificate to our customers to provide best product and best service to our customers.

Is the plan to resell StartCom certificates?

[Han Yuwei]

在 2016年10月13日星期四 UTC+8下午9:09:11，uri...@gmail.com写道：

Wosign's initial business is reselling other's certificates.

[Jakob Bohm]

On 13/10/2016 04:36, 谭晓生 wrote:
> The HSM is stored offline, in the Vault of Qihoo 360’s head quarter, a little bit surprised by this question, I don’t know if there other CAs put their Root Certificates online?
> If anybody have evident to say “Wosign have the private key of StartCom”, please show us here.
>

Thanks for that information. Your previous post saying the "PKI
server" was completely duplicated at WoSign HQ without saying that
server did not contain the the private key was what made me ask about
that possibility.

For additional clarification, the HSM that is only in the Qihoo 360 HQ
vault, is this the HSM for the StartCOM CA root, and/or the HSM for the
Intermediary certificates?

[Gervase Markham]

On 29/10/16 22:42, Percy wrote:
> However, on the official website
> (https://www.wosign.com/about/Why_WoSign.htm) WoSign stated that "沃通是
> 中国唯一一家也是全球唯一一家能签发全球信任的采用国产加密算法(SM2) 的SSL证书和代码签名证书的商业CA。" WoSign is
> the only commercial CA in China -- only commercial CA in the world
> that can Sign SM2 SSL certs/code signing cert that is globally
> trusted.

Well, that statement is either false or very misleading, because in
order for an SM2 certificate to be useful "globally", there needs to be
wide browser support. I don't know exactly which browsers support SM2,
but I know that Firefox, Chrome, Safari, IE and Edge do not.

> This means that WoSign is not only signing SM2 certs for testing but
> also signing SM2 from the globally trusted roots in production. I
> suspect that there are SM2 certs from trusted root WoSign certs used
> in the wild.

Can you find one?

Gerv

[Han Yuwei]

在 2016年10月30日星期日 UTC+8下午9:15:48，Gervase Markham写道：

SM2 is widely used in Chinese government websites. There is a openssl branch (https://github.com/guanzhi/GmSSL) who implemented SM2/SM3/SM4. And I don't see any other depolyment in HTTPS.

[Gervase Markham]

On 30/10/16 19:47, Han Yuwei wrote:
> SM2 is widely used in Chinese government websites. There is a openssl
> branch (https://github.com/guanzhi/GmSSL) who implemented
> SM2/SM3/SM4. And I don't see any other depolyment in HTTPS.

Right, but my question remains: can you find a site with a WoSign SM2
certificate, which chains up to a root Mozilla trusts?

Gerv

[Han Yuwei]

在 2016年10月31日星期一 UTC+8下午11:50:46，Gervase Markham写道：

I am sorry that I can't provide such a certificate for I am not involved in these systems. And I am not likely think there could be a SM2 certificate because major broswers don't implemented SM2/SM3/SM4 so the server would only send RSA/ECC certificates.

[Gervase Markham]

On 31/10/16 18:25, Percy wrote:
> According to http://se.360.cn/event/gmzb.html, the browser needs to send a
> http header Accept-Protocal: SM-SSL.

That seems like an odd mechanism, because SSL connection establishment
happens before HTTP header transmission. Does this header mean "Next
time you contact me, use SM2"?

Gerv

[Han Yuwei]

在 2016年11月1日星期二 UTC+8下午6:43:53，Gervase Markham写道：

Yes. And it may be decided by a USB-key.