StartCom inclusion request: next steps

[Gervase Markham]

The Mozilla CA Certificates team has been considering what the
appropriate next steps are for the inclusion request from the CA
"StartCom".[0] As readers will know, this CA has previously been removed
from trust[1], and so a re-application obviously involves particular
scrutiny. In addition, several questions have been raised about the way
in which the new StartCom PKI has been operated technically[2]. This is
a proposal for the way forward, on which comments are invited.

Mozilla's considered view is as follows:

* It should have been obvious to StartCom that testing of their new
systems needed to be done using a parallel testing hierarchy. That it
was not obvious, is deeply concerning. It is also concerning that
someone can sit at a terminal and issue random certificates with
variable values in lots of fields, in what is to become a
publicly-trusted hierarchy. It's not about numbers (e.g. "40 out of
50000"), it's about the process.

As JC Jones wrote:

"This is a professional PKI operation, being overseen by industry
veterans. If something as concrete as the issuance process had such a
glaring quality assurance methodology failure, why should anyone believe
that something much harder -- subscriber validation -- is going to be
done correctly?"

* The key for their new root certificate was also used in a couple of
intermediates (one revoked as it was done incorrectly - again, lack of
testing!). While this is probably not a policy violation, it's not good
practice.

* StartCom's infrastructure audit, performed by Cure53, was frankly a
security disaster. (They are using EJBCA for CA operations; this was an
audit of their front end and customer management systems, which were
rewritten by a team from their new owner, Qihoo 360.) The (PHP) codebase
was full of holes, poorly commented, had few or no tests, and showed
every evidence of being hacked together in an enormous rush. This does
not inspire confidence. Cure53 say they retested a couple of months
later and most of the holes they found were fixed - although they found
quite a few more. All this does not bode well - Cure53 are not
infallible, an audit is not a substitute for secure coding practices,
and the initial results show that the software was clearly not built by
people who understand software security. The summary of their results is
attached to their Action Items bug[3], but it does leave out some of the
more critical passages of commentary from the original, and of course
does not show the particular holes found and their scope and severity.

* The WT/BR/EV audits on StartCom's website are significantly qualified,
and they include lack of controls on issuance. They should have clean
ones done before we permit any inclusion request to proceed. The
qualifications include:

- Risk analysis process defined but not implemented
- Business continuity plan defined but not implemented
- Audit logs not guaranteed to have integrity
- Monitoring system cannot detect security-related changes to
Certificate Systems

* Certnomis chose to cross-sign StartCom while StartCom had audits with
significant qualifications, and allowed them to recommence
publicly-trusted issuance before they had demonstrated to Mozilla that
they had met the remediation conditions required. While this may not
have been against the letter of our requirements for StartCom to restart
trusted operations, we feel it was not in the spirit of them.

All in all, this attempt to start a new CA compares poorly with other
recent executions of this process, such as those by Google and Amazon.
While those companies do have significantly more resources than
StartCom, many of the issues raised are questions of good practice, not
of money.

Conclusion: StartCom's attempt to restart the CA was rushed. One could
speculate why that was; perhaps due to a requirement to start generating
income again. But a process of building a production PKI by trial and
error, revoking your mistakes, is entirely inappropriate. The qualified
audits include missing or unimplemented processes, and audit/monitoring
failures which lead to uncertainty as to how well the new roots were
protected. This all shows that StartCom were not ready to start up the
production PKI when they did. And yet Firefox today trusts tens of
thousands of certificates issued by this PKI.

Considering all this, our proposal is to require that StartCom begin
again with new-new roots. These roots should be generated inside an
already-security-validated infrastructure, as part of a new WT/BR audit
process, the end results of which are clean because they already have
all the policies and processes in place before the roots are generated.
They should also build and use a parallel testing hierarchy, so that
major operations done on the production PKI are done right, first time.
Once they have generated new-new roots and intermediates, and got clean
audits, they can re-re-apply for inclusion.

No-one should be allowed to cross-sign this new hierarchy until, at
minimum, Mozilla has pronounced itself satisfied that the 5 (or 6)
remediation conditions which were imposed have been met. To permit
otherwise is to allow the bypassing of Mozilla's requirements.

We should add the existing Certnomis cross-signs to OneCRL to revoke
all the existing certificates. As of 10th August (now a month ago)
StartCom said they have 50000 outstanding SSL certs which are valid due
to the Certnomis cross-sign. Revoking them all by adding intermediates
to OneCRL would therefore lead to non-negligible disruption. But these
were issued by an org whose most recent audits are qualified, which is
under sanction, and about whose issuance practices and process safety
there is a reasonable amount of doubt. We may allow a grace period for
customers to replace them with certs from a trusted provider.

We are not sure what to do about StartCom's poor quality PHP code. While
continued use of it would cause us concern, we are not really in a
position to request particular changes to it, or a complete rewrite, in
a verifiable way. On the other hand, a security audit is a remediation
condition, and the current codebase can hardly be said to have passed
with flying colours.

We feel some sympathy for StartCom CEO Inigo Barreira, who has been
placed in a difficult position since he took on the role, but we need to
treat all CAs equally and fairly, according to our professional
judgement. We would not accept this set of circumstances from other CAs,
and so we feel we cannot accept it here.

Comments on this proposal are welcomed.

Gerv


[0] https://bugzilla.mozilla.org/show_bug.cgi?id=1381406
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1311832
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1369359
https://bugzilla.mozilla.org/show_bug.cgi?id=1386894
https://bugzilla.mozilla.org/show_bug.cgi?id=1386891
https://bugzilla.mozilla.org/show_bug.cgi?id=1381406#c5
and other comments in m.d.s.p.
[3] https://bugzilla.mozilla.org/attachment.cgi?id=8886970

[Jonathan Rudenberg]

> On Sep 14, 2017, at 04:49, Gervase Markham via dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
>
> We should add the existing Certnomis cross-signs to OneCRL to revoke
> all the existing certificates. As of 10th August (now a month ago)
> StartCom said they have 50000 outstanding SSL certs which are valid due
> to the Certnomis cross-sign. Revoking them all by adding intermediates
> to OneCRL would therefore lead to non-negligible disruption. But these
> were issued by an org whose most recent audits are qualified, which is
> under sanction, and about whose issuance practices and process safety
> there is a reasonable amount of doubt. We may allow a grace period for
> customers to replace them with certs from a trusted provider.

I’m not yet convinced a grace period is necessary. StartCom does not list Firefox as a compatible browser on their website (they have the logos for Internet Explorer, Microsoft Edge, Android, and Windows).

Additionally, there are multiple steps in the StartCom issuance flow that contain the following in red text:

> Notice:
> 1. Mozilla and Google decided to distrust all StartCom root certificates as of 21st of October, 2016, meaning that since January all the SSL certificates issued from that date will no longer be trusted in Firefox and Chrome newest releases.
> Besides, Google has gone further and as explained here:
> https://security.googleblog.com/2016/10/distrusting-wosign-and-startcom.html will not trust even those SSL certificates issued before that date until the final disruption.
> Apple's decision announced on Nov 30th, 2016 was to distrust all StartCom root certificates as of 1st of December, meaning that SSL cert issued after December 1st, 2016 will no longer be trusted in Apple’s systems.
> 2. Any subscribers that paid the validation fee after Oct. 21st, 2016 can get full refund by request.
> 3. Currently StartCom is able to provide an interim solution for organization users in case of requested.
> Meanwhile StartCom is updating all systems and is following all requirements requested by Mozilla to regain the trust in these browsers and re-apply after the 6 months time penalty.

Given this explicit notification, I do not believe that subscribers would ever have had the expectation that their certificates would work with Firefox.

Jonathan

[Inigo Barreira]

All,

Obviously this is not the message we would like to read and will try to explain and rebate as much as possible some of the comments posted here.

>
> The Mozilla CA Certificates team has been considering what the appropriate
> next steps are for the inclusion request from the CA "StartCom".[0] As readers
> will know, this CA has previously been removed from trust[1], and so a re-
> application obviously involves particular scrutiny. In addition, several
> questions have been raised about the way in which the new StartCom PKI has
> been operated technically[2]. This is a proposal for the way forward, on which
> comments are invited.
>
> Mozilla's considered view is as follows:
>
> * It should have been obvious to StartCom that testing of their new systems
> needed to be done using a parallel testing hierarchy.

Those tests were done to check the CT behaviour, there was any other testing of the new systems, just for the CT. Those certs were under control all the time and were lived for some minutes because were revoked inmediately after checking the certs were logged correctly in the CTs. It´s not a mis-issuance by means of we didn´t know what happened, we had to investigate, etc. It was not a good practice and I can´t excuse for that, but it was not related to the regular issuance procedure as someone suggested. We provided a report in which indicated all that happened and what we did to not happen this again, updating the EJBCA roles permissions.

> That it was not obvious, is deeply concerning. It is also concerning that someone can sit at a terminal
> and issue random certificates with variable values in lots of fields, in what is
> to become a publicly-trusted hierarchy.

Well, it was possible at that time, but only the CA administrator could do it and under many requirements. It´s not like sitting at a terminal and start issuing certificates, there were and are security mechanisms to avoid "someone" could do that and I can list many. Probably most of the CA administrators of the rest of CAs had this capacity (maybe not now) because the majority of the PKI software allows it and it´s needed when building a hierarchy.

> It's not about numbers (e.g. "40 out of 50000"), it's about the process.
>

This number of 40 is about the total of "mis-issuances" discovered, not only related to these ones for the CT testing. And some other times, discussed in this list, the number matters. Even more, for those 40, most of them were "discovered" by us and acted accordingly as per the BRs. We revoked the majority of them within the 24 hours of being notified internally. When those were posted in the bugzilla, as said, most were revoked and started the investigation on what happened and what actions needed to be done. Some of these "mis-issuances" were due to some incongruencies between the BRs and the Mozilla policy, such as the use of different curves (allowed by the BRs but not for some browsers), or about pre-certificates in which is not clear if they fall under what requirement as a discussion started by Jeremy on the list. For example, is it necessary to revoke also the pre-certificates when a certificate is revoked? Are they need to be considered certificates and meet the BRs and/or Mozilla policy?
Or about the use of Unicode vs punnycode which is still under discussions, even a ballot failed in the CABF. So, those errors we made were also made by some others, and not being as an excuse, but it seems that it was not clear for the CAs.

We updated our procedure issuance to avoid these issues happening in mid July. What did we do?
- Restrict the use of eliptic curves only to those admitted by Mozilla
- Change the certificate profile for not having differences with the key encipherment and key agreement
- update the internal db for country codes
- update the sytems for changing all domain names to punnycode
- and recently develop a csr checking tool to avoid the issue with RSA parameters because EJBCA didn´t have it at that time (it comes now with the new release 6.9.0)

> As JC Jones wrote:
>
> "This is a professional PKI operation, being overseen by industry veterans. If
> something as concrete as the issuance process had such a glaring quality
> assurance methodology failure, why should anyone believe that something
> much harder -- subscriber validation -- is going to be done correctly?"

Well, this is an opinion. And I fully respect but none is free of failures and let´s encrypt (and many others) is also having them as we´re seeing recently with weak keys, etc. and I´m none to say they are not professional, or not having a quality assurance methodology, ... or for not believing they are not acting correctly. For sure they are, but same as us. I can´t critize anyone and not because we´re in a weak position at Startcom in which everyone is looking deeply what we do, scrutinizing deeply.
For all these failures we acted quickly because our internal procedures worked well and as said, at the time of discovering most of them were revoked and solutions were ongoing. It´s not like "thanks for letting us know, we didn´t know, we are going to investigate, etc."

I can put other opinions here. For example. Matthew Hardeman wrote:
"If Inigo has prior CA management experience and is running the technical picture at Startcom now, why not allow them to proceed under this new PKI infrastructure with past issues set aside and take a serious stance to any issues going forward.

As far as I know, the current manager of Startcom has not been previously accused of deception or bad action. Far more than has been problematic in this early testing phase of their new PKI has been forgiven by the root programs before.

Nothing disastrous or intentionally dishonest has been done in their new PKI. Why not grant them a gentleman's chance to proceed and address any further issues with great scrutiny?"

>
> * The key for their new root certificate was also used in a couple of
> intermediates (one revoked as it was done incorrectly - again, lack of
> testing!). While this is probably not a policy violation, it's not good practice.
>

Yes, it´s not a policy violation. As explained, this was a problem in the EJBCA with the UTF8 encoding. It´s not related to a lack of testing, we generated intermediates in our development and QA system, it´s the same procedure and we followed it, nothing happened in the others but this one had this issue, so we had to revoke and create a new one. This happened in April.

> * StartCom's infrastructure audit, performed by Cure53, was frankly a security
> disaster. (They are using EJBCA for CA operations; this was an audit of their
> front end and customer management systems, which were rewritten by a
> team from their new owner, Qihoo 360.) The (PHP) codebase was full of
> holes, poorly commented, had few or no tests, and showed every evidence of
> being hacked together in an enormous rush. This does not inspire confidence.
> Cure53 say they retested a couple of months later and most of the holes they
> found were fixed - although they found quite a few more. All this does not
> bode well - Cure53 are not infallible, an audit is not a substitute for secure
> coding practices, and the initial results show that the software was clearly not
> built by people who understand software security. The summary of their
> results is attached to their Action Items bug[3], but it does leave out some of
> the more critical passages of commentary from the original, and of course
> does not show the particular holes found and their scope and severity.

Yes, it´s true. The first security audit didn´t go very well. That was mentioned also during the CABF F2F meeting at Cisco. In our remediation plan we imposed ourselves a very tight timing for this task and we failed. It was a very hard task in very few time but the people at 360 tried everything to get it done by that date, end of december 2016, and yes, we reached the date but with many failures. I may think that everyone has suffered this type of situations and none can write code at the first time without errors. So, of course, we went for "another round" because that was not aceptable. The second audit as you mention reflects that the issues found in the first one were fixed and some new ones cameo ut, but which were also fixed later on. Since then, the RD team and Security team have evolved the system and right now is robust.
In any case, until we had the OK from Cure 53, we didn´t go further and didn´t go live. Later on we generated the subCAs in production and then started to issue certificates.
And for sure Cure 53 is not infalible, but it´s true that those security audits gone very deep, and that the security team at 360 has continued improving the system.

>
> * The WT/BR/EV audits on StartCom's website are significantly qualified, and
> they include lack of controls on issuance. They should have clean ones done
> before we permit any inclusion request to proceed. The qualifications include:
>
> - Risk analysis process defined but not implemented
> - Business continuity plan defined but not implemented
> - Audit logs not guaranteed to have integrity
> - Monitoring system cannot detect security-related changes to
> Certificate Systems
>

Yes, this is also true. Our webtrust audits have findings but those are not so significant according to the auditors who signed the reports, so I assume the auditors thought that the system is good enough to have the audit report in place. Of course, I had wanted to have a clean one but it´s also true that the reports indicate that most of them are fixed, and I think it´s a matter of transparency.
We prepared a Corrective Action Plan providing solutions for all the findings and indicating time of application. We sent to the auditors with all the evidences showing that most were fixed rightly.
I´d like to explain those you mention.

Risk analysis: the risk analysis was defined and was implemented. We had the 2016 risk analysis done and the new 2017 risk analysis was scheduled in october this year, following the agenda set. The auditors requested the 2017 one and hence we did it just after the audit. But in any case, the 2017 risk analysis is done and sent to the auditors.

Business Continuity Plan: the BCP was defined and implemented, but partially. We were very optimistic to meet everything we wrote and at the time of the audit some things that were in the document were not finished in time, so, that´s the finding. Of course, I could have written a BCP less onerous and then have met what we had in place and not having the finding, but I wanted a very good one, and thus, didn´t mind that finding. BTW, we´ve finished our BCP according to the document.

Audit logs integrity: all logs were and are signed internally in the PKI system (it´s part of the configuration of the EJBCA) and provided all the evidences but they also requested to do the same with some other external components and not all products come with that feature, so had to develop it. It was applied at the beginning of june and sent the evidences to the auditors.

Monitoring: Startcom´s monitoring system was focused mainly on the monitoring of the server status (CPU, load, memory, etc.) of all servers under the PKI infrastructure. Furthermore, all services provided were also monitored, to check that even the server hosting the service was ok, the service had to be also live and running. So checking the service availability.

This finding is about updating the monitoring configuration alerting StartCom specific teams/people to check if there are some sensitive configuration files of the infrastructure that are being changed. Internally, StartCom has a manual procedure when some changes/updates to these files are requested in which need the approval by the service manager, then modify/update the system, and the auditors wanted to have this done also automatically, so updated the monitoring to alert inmediately specific teams in case of sensitive configuration files are changed. We extended the scope to cover all systems, DB, webservers, nginx, ... and are using a tool called inotify, which monitor and check all the writing operations. This was done at the beginning of June.

> * Certnomis chose to cross-sign StartCom while StartCom had audits with
> significant qualifications,

Well, the auditors explained in the reports that most of them were fixed, so even they are in the report because it´s what they found and despite some discussions about all them (I wasn´t agree with some), you can´t consider them as significant (I know it´s an opinion), or at least it´s contrary to auditor´s opinion. You can contact the auditors if you wish to request for their opinion about the audit report, what they found, what we provided, the CAP, etc.

> and allowed them to recommence publicly-trusted issuance before they had demonstrated to Mozilla that they had met the
> remediation conditions required. While this may not have been against the
> letter of our requirements for StartCom to restart trusted operations, we feel
> it was not in the spirit of them.
>

Not sure how to interpretate this. We followed our remediation plan and the Mozilla requirements, once we met all of them, we reapplied to be included in Mozilla, which it was in july (time after the mis-issuances, security reports and audit findings) following the steps as any other CA. We´ve seen in the past some others doing the same. Certinomis put us some requirements, like having the WT audit certificates, but maybe some others don´t put any requirement, just cross-sign the new CA and this one, in the meantime, gets its audit for example.

> All in all, this attempt to start a new CA compares poorly with other recent
> executions of this process, such as those by Google and Amazon.
> While those companies do have significantly more resources than StartCom,
> many of the issues raised are questions of good practice, not of money.
>

I don´t know how these you mention have applied, but I remember lots of issues regarding Google and the acquisition of the Globalsign roots and how they proceeded.
Again I don´t know these examples so can´t speak for them and don´t like to talk about others but it´s different to start when you have customers requesting certificates, asking for all that happened, etc. rather than starting from scratch if you don´t have customers yet or very few, or when you have other roots accepted in the root programs from which you could provide your services normally. We were in a very difficult situation, with lots of pressure even from the mozilla community, because even though being distrusted (we didn´t exist) and not applied yet for re-inclussion we were all the time asked/suggested/questioned many things that maybe are not done with other CAs.
And yes, at the end of the day, it´s a matter of money. You can´t do many things, or have to be put on hold new projects if you have no resources or these are limited. I´d like to do many things at startcom at the same time, I have lots of projects in my mind to develop but can´t be done because of it (lack of money). I don´t think this is only a matter of good practice because all is related.

> Conclusion: StartCom's attempt to restart the CA was rushed.

Yes, I admit it.

> One could speculate why that was; perhaps due to a requirement to start generating
> income again.

Well, finally this is a business and I don´t think none on this list is working for free. At the end everyone has his/her salary, etc. But that was not the main reason because getting included in the root programs takes time but wanted to provide our customers which gave us support for what happened with the distrust (which IMHO in the case of Startcom was very aggressive) a solution generating a new fresh and clean system.

> But a process of building a production PKI by trial and error, revoking your mistakes, is entirely inappropriate.

I don´t think we´ve built a trial and error PKI system. As you´ve said, 40 of 50000 in this question matters. If everything was a trial and error system, of course, it´s inaceptable, but I think this is not the case. As said, some of those 40 are due to some different "interpretations" which are still being discussed in the mozilla list. Revoking is the first thing to do for a CA, and start investigating the issue and propose countermeasures to avoid that happening again. And this is what we have done.
And, using the same argument, and seing the recently issues that have been described in the mozilla list during this summer, I don´t think that Startcom is doing that bad not based in numbers not in the errors itself. As said, we have implemented and improved many things, integrating cablint/X509lint in our processes, also the crt.sh in our CMS system, key validations, have all new EJBCA releases up to date, etc.

> The qualified audits include missing or unimplemented processes, and audit/monitoring failures which
> lead to uncertainty as to how well the new roots were protected.

This has been explained and don´t think is accurate. Regarding the roots, they are very well protected, we had a root key ceremony which an auditor witnessing it and with a final report in which everything was ok.

> This all shows that StartCom were not ready to start up the production PKI when they
> did. And yet Firefox today trusts tens of thousands of certificates issued by
> this PKI.

Not sure about this. We were distrusted in october 2016, the new system started to operate in april 2017, which is not related to the old one which has been switched off. None of the new certs are trusted in Mozilla Firefox and we notify our users so by messages in the web and applications.

>
> Considering all this, our proposal is to require that StartCom begin again with
> new-new roots. These roots should be generated inside an already-security-
> validated infrastructure, as part of a new WT/BR audit process, the end
> results of which are clean because they already have all the policies and
> processes in place before the roots are generated.
> They should also build and use a parallel testing hierarchy, so that major
> operations done on the production PKI are done right, first time.
> Once they have generated new-new roots and intermediates, and got clean
> audits, they can re-re-apply for inclusion.

I don´t know how to understand this requirement. We´re required to generate new roots and intermediates, get a clean audit and then re-apply. So, the only difference of what we have done, it´s just the clean audit, which I´ve already explained. Is this interpretation ok?

>
> No-one should be allowed to cross-sign this new hierarchy until, at minimum,
> Mozilla has pronounced itself satisfied that the 5 (or 6) remediation
> conditions which were imposed have been met. To permit otherwise is to
> allow the bypassing of Mozilla's requirements.

Ok, I see this is a new requirement that was not imposed last time in which you recommended and allowed us to be cross-signed as many other CAs have done in the past to be in the business.

We´ve met all the conditions, new system, new management, security audit and webtrust audit and CT logging. In those conditions, it was not mentioned that the webtrust audit should be clean but as indicated time ago, we wanted to have a clean one and hence perform a new one (we told so the auditors), but asked us to wait until we had everything in place (also said recently that only the TSA and the BCP issues were pending) , and then wait for another 2 months as the WT requirements indicate.

>
> We should add the existing Certnomis cross-signs to OneCRL to revoke all the
> existing certificates. As of 10th August (now a month ago) StartCom said they
> have 50000 outstanding SSL certs which are valid due to the Certnomis cross-
> sign.

I´ve never said this. In fact, despite having that cross-signed which were provided to us in july we have never used and provided to any of our customers to build a trusted path. So none of those 50000, or the new ones, go with the Certinomis path because none have it. But all those 50000 certs are untrusted because we´re not in the Mozilla root, not the new one, and the old one was distrusted.
In fact, recently, I asked for permission to use the Certinomis cross-signed certificates and have no response. I don´t know if this is an administrative silence which may allow me to use it but until having a clear direction we haven´t used it.

> Revoking them all by adding intermediates to OneCRL would therefore lead to non-negligible disruption. But these were issued by an org whose
> most recent audits are qualified, which is under sanction, and about whose
> issuance practices and process safety there is a reasonable amount of doubt.

Again, I don´t understand why you say this. We haven´t used the Certinomis path so no need to revoke anything. Regarding the audits, which are qualified, does this mean that only clean audits are valid? This is not a requirement in the Mozilla policy afaik.

> We may allow a grace period for customers to replace them with certs from a
> trusted provider.
>

> We are not sure what to do about StartCom's poor quality PHP code. While
> continued use of it would cause us concern, we are not really in a position to
> request particular changes to it, or a complete rewrite, in a verifiable way. On
> the other hand, a security audit is a remediation condition, and the current
> codebase can hardly be said to have passed with flying colours.

I think this has been explained. I don´t understand why you say it´s a "poor quality PHP code". As said, I admit that the first time was not good, but that´s not the reality now and it´s not when we applied for re-inclusion. Are you going to check all the code of all CAs? I don´t think so.

>
> We feel some sympathy for StartCom CEO Inigo Barreira, who has been
> placed in a difficult position since he took on the role,

Thanks, I really appreciate it.

> but we need to treat all CAs equally and fairly, according to our professional judgement. We would
> not accept this set of circumstances from other CAs, and so we feel we cannot
> accept it here.

Well, frankly, I think the requirements impossed to StartCom are not the same you require for other CAs. We´ve had additional requirements and hence you´re not treating all the CAs equally. We accepted those requirements and I think we´ve gone further than that (i.e. adquiring a new well-know PKI solution like EJBCA) but it´s true that the reasons for distrust Startcom were not for technically reasons mainly and we see (technical) issues in other CAs which are not treated so deeply. So, I think that StartCom is treated differently.

We applied in july, following all the steps required and answering to the conditions impossed, some of the issues happened before the application, most of the audit findings were fixed before the application, we applied some of the countermeasures explained in mid-july, just after the application, since the application we´ve made lots of improvements as explained and I think Mozilla didn´t started yet handling our our application because we were waiting in the queue due to the number of applications.
So, when does this process start? I mean, are you dealing with historic or past issues even before applying? I think these questions have been posted in the mozilla list and this is the case which I think we´re not treated equally.

It´s been some discussion about StartCom during this time, digging and posting things not related but to hurt us, even when were distrusted and none of our certificates were valid somehow. The number again matters, 50000, for being distrusted is not a usual number, and these have been issued with the new one, just since april, and only 40 had some issues, which as mentioned, maybe not so clear and still under discussion. But instead, the charges against us seem that we are a total disaster and not deserve any trust nor chance. If the community feels so, then would like to see how the others are treated when make some mistake. And will think again we´re not treated equally.

[Nick Lamb]

On Thursday, 14 September 2017 16:00:35 UTC+1, Inigo Barreira wrote:
> Well, finally this is a business and I don´t think none on this list is working for free. At the end everyone has his/her salary, etc. But that was not the main reason because getting included in the root programs takes time but wanted to provide our customers which gave us support for what happened with the distrust (which IMHO in the case of Startcom was very aggressive) a solution generating a new fresh and clean system.

I can't speak for other people contributing to m.d.s.policy but of course I am not paid to do this. I have a job, but it isn't this. I have never asked my employer's permission to contribute here, judging it to be entirely outside their purview. My job does include running a (small, private) CA, but then it also includes maintaining a DBMS, a web crawler and all sorts of other stuff, I'm sure it would be possible to identify some connection to my job for almost any technical contribution I could make anywhere.

There is an proverb in English, I am not sure if you're familiar with it, "More haste, less speed". What this means is that in trying to perform a task as quickly as possible you may instead cause yourself such extra trouble that in the end the task takes even longer to complete. I am sure that even if you have not come across a phrase like this before you can see its applicability to the situation for StartCom.

[Jakob Bohm]

On 14/09/2017 17:05, Inigo Barreira wrote:
> All,
>
> ...

>>
>> We should add the existing Certnomis cross-signs to OneCRL to revoke all the
>> existing certificates. As of 10th August (now a month ago) StartCom said they
>> have 50000 outstanding SSL certs which are valid due to the Certnomis cross-
>> sign.
>
> I´ve never said this. In fact, despite having that cross-signed which were provided to us in july we have never used and provided to any of our customers to build a trusted path. So none of those 50000, or the new ones, go with the Certinomis path because none have it. But all those 50000 certs are untrusted because we´re not in the Mozilla root, not the new one, and the old one was distrusted.
> In fact, recently, I asked for permission to use the Certinomis cross-signed certificates and have no response. I don´t know if this is an administrative silence which may allow me to use it but until having a clear direction we haven´t used it.
>

I can't speak for Mozilla, but the obvious point is, that as soon as
the cross certificate from Certinomis was published in CT-logs or
elsewhere, any StartCom customer could (and still can) download it and
install it on their server, thus activating the Certinomis path for
their server.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

[Inigo Barreira]

Hi Percy,

Yes, you´re right, that was on the table and also suggested by Mozilla, but
the issue was that people from 360 are used to code in PHP and the old one
was in Java and some other for which they are not so familiar and then was
decided to re-write all the code in PHP trying to keep the same
functionality. Besides, the old code had to be integrated with the new PKI
infrastructure, EJBCA, and that was not an easy task.
All in all on the table, integration with new systems, maintenance of the
old code if issues arise due to this integration or any other problem, not
good language knowledge, get used to PHP, etc. made us took that decission.
Furthermore, with this decission, we also wanted to let the community know
that this is totally a new CA system in all aspects, nothing related to the
past, everything from scratch, so new coding, new programming language, new
PKI system, infrastructure, etc. hoping this would make the community have a
better impression of the new Startcom regarding the distrust issue.

Best regards

Iñigo Barreira
CEO
StartCom CA Limited

> -----Original Message-----
> From: dev-security-policy [mailto:dev-security-policy-
> bounces+inigo=startco...@lists.mozilla.org] On Behalf Of Percy via
dev-
> security-policy
> Sent: jueves, 14 de septiembre de 2017 22:13
> To: mozilla-dev-s...@lists.mozilla.org
> Subject: Re: StartCom inclusion request: next steps
>
> "Conclusion: StartCom's attempt to restart the CA was rushed."
>

> "It was a very hard task in very few time but the people at 360 tried
> everything to get it done by that date, end of december 2016, and yes, we
> reached the date but with many failures"
>

> May I ask why StartCom choose to rush everything in PHP from the ground up
> rather than using the more secure system already in place in the old
> StartCom? From my understanding, the distrust of StartCom is more related
> to the secret acquisition by WoSign an Qihoo 360 rather than insecure
> infrastructure. So if the deadline is so imminent as you stated and
pressure is
> so high from customers, can't you use the reasonably secure old code base
> rather than rushing everything from the ground up? Then you will have more
> time transition to another system if needed with sufficient time for
secure
> processes?
> _______________________________________________
> dev-security-policy mailing list
> dev-secur...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy

[Inigo Barreira]

> On 14/09/2017 17:05, Inigo Barreira wrote:
> > All,
> >
> > ...
> >>

> >> We should add the existing Certnomis cross-signs to OneCRL to revoke
> >> all the existing certificates. As of 10th August (now a month ago)
> >> StartCom said they have 50000 outstanding SSL certs which are valid
> >> due to the Certnomis cross- sign.
> >
> > I´ve never said this. In fact, despite having that cross-signed which were
> provided to us in july we have never used and provided to any of our
> customers to build a trusted path. So none of those 50000, or the new ones,
> go with the Certinomis path because none have it. But all those 50000 certs
> are untrusted because we´re not in the Mozilla root, not the new one, and the
> old one was distrusted.
> > In fact, recently, I asked for permission to use the Certinomis cross-signed
> certificates and have no response. I don´t know if this is an administrative
> silence which may allow me to use it but until having a clear direction we
> haven´t used it.
> >
>
>

> I can't speak for Mozilla, but the obvious point is, that as soon as the cross
> certificate from Certinomis was published in CT-logs or elsewhere, any
> StartCom customer could (and still can) download it and install it on their
> server, thus activating the Certinomis path for their server.
>

AFAIK, Certinomis only disclosed in the CCADB and even we received it, we have not sent to any customer nor posted anywhere. And I know crt.sh includes it and when questioned for it Rob answered that crt.sh includes all certs that have been submitted to one or more of the monitored CT logs, and includes all trust paths that can be built.
I don´t think any StartCom customer has downloaded from wherever it could be and installed and created the trusted path. Our customers base are not very familiar with this stuff. Usually we need to provide all of them with instructions on what and how to do it.

>
> Enjoy
>
> Jakob
> --
> Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
> Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This
> public discussion message is non-binding and may contain errors.
> WiseMo - Remote Service Management for PCs, Phones and Embedded

[James Burton]

> Those tests were done to check the CT behaviour, there was any other testing of the new systems, just for the CT. Those certs were under control all the time and were lived for some minutes because were revoked inmediately after checking the certs were logged correctly in the CTs. It´s not a mis-issuance by means of we didn´t know what happened, we had to investigate, etc. It was not a good practice and I can´t excuse for that, but it was not related to the regular issuance procedure as someone suggested. We provided a report in which indicated all that happened and what we did to not happen this again, updating the EJBCA roles permissions.

1) Why didn't StartCom build a test hierarchy?
2) Why didn't StartCom use the TestTube CT log for testing CT?

[Inigo Barreira]

>
> > Those tests were done to check the CT behaviour, there was any other
> testing of the new systems, just for the CT. Those certs were under control all
> the time and were lived for some minutes because were revoked inmediately
> after checking the certs were logged correctly in the CTs. It´s not a mis-
> issuance by means of we didn´t know what happened, we had to investigate,
> etc. It was not a good practice and I can´t excuse for that, but it was not
> related to the regular issuance procedure as someone suggested. We
> provided a report in which indicated all that happened and what we did to
> not happen this again, updating the EJBCA roles permissions.
>

> 1) Why didn't StartCom build a test hierarchy?

Considering that we were distrusted, that we didn´t reapply for inclussion, that CT is only required by Chrome and it´s not included in the Mozilla policy (even we were requested that all of our certs had to be CT logged) nor required by Firefox, that those certs were under our control all the time and lived for some minutes because were revoked inmediately, at that time, when we did it, we didn´t expect this reaction for sure.

Of course if we had known it we hadn´t done it and for sure had built a test hierarchy but there´s nothing we can do now. Only wanted to state that those certs were under our control all the time, and lived for some minutes because were revoked after the test. There were not any other testing of any other nature directly in the production system

> 2) Why didn't StartCom use the TestTube CT log for testing CT?

We tried to check and test the same behaviour before going live with the CT logging, so followed the requirements to use 3 logs, one google and one non-google, for the EVs and this is what we did. We used the same settings we had before the distrust using the startcom log and the google ones (pilot and rocketeer).

[James Burton]

Hi Inigo,

I'm just trying to get everything straight, cleared up and then we can move on.

I found all of your answers very concerning in the way you've conducted testing. I thought that all CAs must have some type of test hierarchy in place to test new software, requirements and etc before going live but from the answers you've given, StartCom neglected this in favour to test as you go along and deal with the problems using a live CA hierarchy. All this feels very amateurish and doesn't give me the any confidence at all that your CA is proven itself to be a trustworthy part of this infrastructure.

I recommend to Mozilla to require StartCom to start again fresh.

1. Build a test hierarchy. Test the software and etc to industry guidelines.
2. Build a new production hierarchy (including new HSMs, keys, roots, etc.) and then re-apply for inclusion into the Mozilla root program.
3. Once approved then you can cross-sign your roots with another CA.

While this is happening. You can resell certificates from other CAs and build up trust in the industry which will benefit you in the long term I feel.

James

[James Burton]

Hi Inigo,

To add from the last post.

I know this is unwelcome news to you but I feel that with all these incidents happening right now with Symantec and the incidents before, we can't really take any more chances. Every incident is eroding trust in this system and if we want more people to take up adoption of https in the long term, I feel that we need to start operating infrastructure above board and without issues.

James

[Inigo Barreira]

> Hi Inigo,
>
> I'm just trying to get everything straight, cleared up and then we can move
> on.
>
> I found all of your answers very concerning in the way you've conducted
> testing. I thought that all CAs must have some type of test hierarchy in place
> to test new software,

Well, this is not required. Of course we have a development and a testing system, which are not "connected" into production.
As we wanted to check the same behaviour for the CT ecosystem than before the distrust, we did it directly in production, but don´t take this as we didn´t test it before in our development and testing systems, for sure we did.

> requirements and etc before going live but from the answers you've given, StartCom neglected this in favour to test as you go
> along and deal with the problems using a live CA hierarchy.

As said, this wasn´t a "live" CA hierarchy. We were distrusted, nothing issued from that "live" CA was trusted, this wasn´t any harm to any of the Mozilla users.

> All this feels very amateurish and doesn't give me the any confidence at all that your CA is
> proven itself to be a trustworthy part of this infrastructure.

I respect this opinion but can´t agree with it.

>
> I recommend to Mozilla to require StartCom to start again fresh.
>
> 1. Build a test hierarchy. Test the software and etc to industry guidelines.
> 2. Build a new production hierarchy (including new HSMs, keys, roots, etc.)
> and then re-apply for inclusion into the Mozilla root program.
> 3. Once approved then you can cross-sign your roots with another CA.
>
> While this is happening. You can resell certificates from other CAs and build
> up trust in the industry which will benefit you in the long term I feel.
>
> James

[Inigo Barreira]

> Hi Inigo,
>
> To add from the last post.
>
> I know this is unwelcome news to you but I feel that with all these incidents
> happening right now with Symantec and the incidents before, we can't really
> take any more chances. Every incident is eroding trust in this system and if we
> want more people to take up adoption of https in the long term, I feel that
> we need to start operating infrastructure above board and without issues.
>
> James

James, I share with you this desire. But we´re seeing one day and another that issues are still happening, and not all are from StartCom :-) so to avoid eroding trust what do you suggest if not give more chances? Remove them all?

[Gervase Markham]

Hi Inigo,

On 14/09/17 16:05, Inigo Barreira wrote:
> Those tests were done to check the CT behaviour, there was any other testing of the new systems, just for the CT.

Is there any reason those tests could not have been done using a
parallel testing hierarchy (other than the fact that you hadn't set one up)?

> Some of these "mis-issuances" were due to some incongruencies between the BRs and the Mozilla policy, such as the use of different curves (allowed by the BRs but not for some browsers),

But it is the job of a CA to be aware of browser policies.

> As far as I know, the current manager of Startcom has not been previously accused of deception or bad action.

No, indeed. There is no accusation that you have been deceptive towards
anyone.

> Yes, it´s not a policy violation. As explained, this was a problem in the EJBCA with the UTF8 encoding.

That was the reason for the revocation; it's not the reason for the key
reuse.

>> * The WT/BR/EV audits on StartCom's website are significantly qualified, and
>> they include lack of controls on issuance. They should have clean ones done
>> before we permit any inclusion request to proceed. The qualifications include:
>>
>> - Risk analysis process defined but not implemented
>> - Business continuity plan defined but not implemented
>> - Audit logs not guaranteed to have integrity
>> - Monitoring system cannot detect security-related changes to
>> Certificate Systems
>
> Yes, this is also true. Our webtrust audits have findings but those are not so significant according to the auditors who signed the reports, so I assume the auditors thought that the system is good enough to have the audit report in place.

I think lack of monitoring and lack of integrity of logs are serious
issues. Repairing them afterwards does not remove the uncertainty. If
you said "I left the root certificate private key on a USB stick on the
desk in my unlocked office over the weekend - but it's OK, I've
remediated the problem now, and it's back in the safe", that would not
remove the uncertainty about whether someone had done something with it
in the mean time.

> I don´t know how these you mention have applied, but I remember lots of issues regarding Google and the acquisition of the Globalsign roots and how they proceeded.

I think "lots of issues" is an overstatement. There was an issue
regarding the scope of their audits which was raised publicly, but it
was something that Google has explicitly sought our approval for in advance.

> Not sure about this. We were distrusted in october 2016, the new system started to operate in april 2017, which is not related to the old one which has been switched off. None of the new certs are trusted in Mozilla Firefox and we notify our users so by messages in the web and applications.

I may have made a mistake here. I was under the impression that you had
told me that your new hierarchy, cross-signed by Certnomis, had issued
50,000 certificates. Did I misunderstand? If so, my apologies.

> Ok, I see this is a new requirement that was not imposed last time in which you recommended and allowed us to be cross-signed as many other CAs have done in the past to be in the business.

Mozilla did not recommend that you be cross-signed. Certnomis raised the
possibility, and we said that it could happen when you had met the BRs
and completed the remediation steps. The plan we "approved" was not a
cross-signing plan. The audits show you haven't yet met the BRs, and we
have not yet signed off on you having completed the remediation steps.

> We´ve met all the conditions, new system, new management, security audit and webtrust audit and CT logging. In those conditions, it was not mentioned that the webtrust audit should be clean

Isn't that implied?

> I´ve never said this. In fact, despite having that cross-signed which were provided to us in july we have never used and provided to any of our customers to build a trusted path. So none of those 50000, or the new ones, go with the Certinomis path because none have it. But all those 50000 certs are untrusted because we´re not in the Mozilla root, not the new one, and the old one was distrusted.

So the 50,000 certs you mentioned are issued in your old hierarchy,
which is not cross-signed by Certnomis?

If you could clarify the numbers for your old and new PKI, and confirm
that the Certnomis cross-signs apply only to the new one, that would be
helpful. Again, apologies for any misunderstanding on my part.

> In fact, recently, I asked for permission to use the Certinomis cross-signed certificates and have no response. I don´t know if this is an administrative silence which may allow me to use it but until having a clear direction we haven´t used it.

Can you remind me how you asked and when?

> I think this has been explained. I don´t understand why you say it´s a "poor quality PHP code".

Because poor quality PHP code does not become good quality PHP code by
applying security fixes to it. I am talking about the development
metholodology and speed which were evident from the Cure53 report.

Gerv

[Gervase Markham]

On 15/09/17 11:01, Inigo Barreira wrote:
> Considering that we were distrusted, that we didn´t reapply for
> inclussion, that CT is only required by Chrome and it´s not included
> in the Mozilla policy (even we were requested that all of our certs
> had to be CT logged) nor required by Firefox, that those certs were
> under our control all the time and lived for some minutes because
> were revoked inmediately, at that time, when we did it, we didn´t
> expect this reaction for sure.

But surely CT testing is not the only sort of testing you've been doing?
E.g. you made some test certificates with different types of ECC curve,
which you then had to revoke some of as against browser policies. If
these had been in a testing hierarchy there would have been no problem.

CAs have been heavily criticised over the past few years for issuing
test certificates in public hierarchies (see e.g. Symantec). The danger
of doing so should be well known to all CAs by now.

Perhaps once a test has been passed and checked in a testing system, and
if the certificates concerned do not violate any policies, it could be
repeated on a production system to deal with any possible differences
between the two. But starting with the production system is not a good idea.

Gerv

[Inigo Barreira]

> On 15/09/17 11:01, Inigo Barreira wrote:

> > Considering that we were distrusted, that we didn´t reapply for
> > inclussion, that CT is only required by Chrome and it´s not included
> > in the Mozilla policy (even we were requested that all of our certs
> > had to be CT logged) nor required by Firefox, that those certs were
> > under our control all the time and lived for some minutes because were
> > revoked inmediately, at that time, when we did it, we didn´t expect
> > this reaction for sure.
>

> But surely CT testing is not the only sort of testing you've been doing?

Yes, this is the only test we did it in production

> E.g. you made some test certificates with different types of ECC curve, which

> you then had to revoke some of as against browser policies.

No, those weren´t tests. We allowed the use of curves permitted by the BRs but this issue came up in the mozilla policy (I think Arkadiusz posted) and I also asked about it in the last CABF F2F (I asked Ryan about it) and then, with that outcome and as the browsers didn´t accept them, we revoked and then not allow the issuance. I think the discussion is still active (i.e. the use of P-521).

> If these had been in a testing hierarchy there would have been no problem.
>
> CAs have been heavily criticised over the past few years for issuing test
> certificates in public hierarchies (see e.g. Symantec). The danger of doing so
> should be well known to all CAs by now.

Yes, I know. But the only testing we did in production was the one related to the CT.

>
> Perhaps once a test has been passed and checked in a testing system, and if
> the certificates concerned do not violate any policies, it could be repeated on
> a production system to deal with any possible differences between the two.
> But starting with the production system is not a good idea.

True, but it seems you´re understanding that we have only a production system in which we test everything and this is not the case. Before moving anything into production, we have tested in development and in the QA system.
>
> Gerv

[Inigo Barreira]

> > Yes, you´re right, that was on the table and also suggested by
> > Mozilla, but the issue was that people from 360 are used to code in
> > PHP and the old one was in Java and some other for which they are not
> > so familiar and then was decided to re-write all the code in PHP
> > trying to keep the same functionality.
>

> Given the quality of code produced,

I don´t think the quality of the code which is in production now is poor or of bad quality. It wasn´t good initially, that´s true, but not now.

> it might have been better in hindsight tohire Java experts to work on the old codebase.

That was also on the table.

>
> > Furthermore, with this decission, we also wanted to let the community
> > know that this is totally a new CA system in all aspects, nothing
> > related to the past, everything from scratch, so new coding, new
> > programming language, new PKI system, infrastructure, etc. hoping this
> > would make the community have a better impression of the new Startcom
> regarding the distrust issue.
>

> "We rewrote everything from scratch" is not actually something which itself
> inspires confidence.

What I meant, is that we used a new programming language and then recoded.

In the case of WoSign, it was required of them because
> their old code was clearly terrible and buggy. But the reason the result would
> have to be strongly audited (were they to
> reapply) is that new code is riskier than old, tried-and-tested code.
>
> I don't know if I ever wrote it down anywhere, but I'm fairly sure that
> switching back from the WoSign codebase to the older StartCom codebase
> (i.e. reversing the change you made after the purchase) was my suggestion for
> how StartCom should proceed after the dis-trust event.

Yes, that was your suggestion.

> That doesn't mean you are required to take my advice,

Yes, I know

> but it might have beena hint that I wouldn't consider "hey, we rewrote everything from scratch!" as
> a positive point.

Well, we thought that it could be. During the distrust issues, I think Ryan posted some old issues related to the old Startcom code or procedures (long time ago) and then recoding everything was our intent to give a positive answer. As said, the term "from scratch" maybe it´s not appropiate, but in the end this code has been audited.
>
> Gerv

[Inigo Barreira]

>
> Hi Inigo,
>
> On 14/09/17 16:05, Inigo Barreira wrote:

> > Those tests were done to check the CT behaviour, there was any other
> testing of the new systems, just for the CT.
>

> Is there any reason those tests could not have been done using a parallel
> testing hierarchy (other than the fact that you hadn't set one up)?

I think I provided the reasons. We were distrusted, not re-applied yet, those certs lived for minutes, ... So, I think these are the reasons. It´s not an excuse and we didn´t expect this maremágnum. There´s been long discussions about the harm long term certificates can make and asked CAs about short term to avoid damages. These certs, that it´s true was a mistake, only lived for minutes, so I don´t know what else I can add to this matter.

>
> > Some of these "mis-issuances" were due to some incongruencies between
> > the BRs and the Mozilla policy, such as the use of different curves
> > (allowed by the BRs but not for some browsers),
>

> But it is the job of a CA to be aware of browser policies.

Yes, you´re right. And it was my fault for not have checked in deep this particular one.

>
> > As far as I know, the current manager of Startcom has not been previously
> accused of deception or bad action.
>

> No, indeed. There is no accusation that you have been deceptive towards
> anyone.
>

> > Yes, it´s not a policy violation. As explained, this was a problem in the EJBCA
> with the UTF8 encoding.
>

> That was the reason for the revocation; it's not the reason for the key reuse.

Yes, right, but this is allowed afaik. It´s mentioned in the BRs.

>
> >> * The WT/BR/EV audits on StartCom's website are significantly
> >> qualified, and they include lack of controls on issuance. They should
> >> have clean ones done before we permit any inclusion request to proceed.
> The qualifications include:
> >>
> >> - Risk analysis process defined but not implemented
> >> - Business continuity plan defined but not implemented
> >> - Audit logs not guaranteed to have integrity
> >> - Monitoring system cannot detect security-related changes to
> >> Certificate Systems
> >
> > Yes, this is also true. Our webtrust audits have findings but those are not so
> significant according to the auditors who signed the reports, so I assume the
> auditors thought that the system is good enough to have the audit report in
> place.
>

> I think lack of monitoring and lack of integrity of logs are serious issues.

There wasn´t a lack of integrity and monitoring, of course not. All PKI logs were and are signed, it´s just the auditors wanted to add the integrity to other systems which is not so clear that should have this enabled. For example, if you want to archive database information for not managing a big one, the integrity of the logs could be a problem when trying to "move" to an archive system. I had some discussions about the "scope" of the integrity. Regarding the monitoring, well, we monitor many things, in both data centers, 24x7, etc. For this specific issue, it´s true that we didn´t have it automatically but manually, but well, and we implement a solution, but this is not a lack of monitoring. I think the audits are to correct and improve the systems and don´t think any CA at the first time had everything correct. So, for example, I thought this finding was good because made us improve.

> Repairing them afterwards does not remove the uncertainty.

Well, then any issue that you could find, even repaired or fixed, does not provide you any security and hence you should not trust anyone.

> If you said "I left the root certificate private key on a USB stick on the desk in my unlocked
> office over the weekend - but it's OK, I've remediated the problem now, and
> it's back in the safe", that would not remove the uncertainty about whether
> someone had done something with it in the mean time.

Well, I don´t think this is the same "quality" of example.

>
> > I don´t know how these you mention have applied, but I remember lots of
> issues regarding Google and the acquisition of the Globalsign roots and how
> they proceeded.
>

> I think "lots of issues" is an overstatement. There was an issue regarding the
> scope of their audits which was raised publicly, but it was something that
> Google has explicitly sought our approval for in advance.

Yes, it´s an overstatement but similar to others. I said that I didn´t know what or how they had applied but you put them as examples and just wanted to indicate that there were discussions, so maybe was not so smooth, but again, I don´t know.

>
> > Not sure about this. We were distrusted in october 2016, the new system
> started to operate in april 2017, which is not related to the old one which has
> been switched off. None of the new certs are trusted in Mozilla Firefox and
> we notify our users so by messages in the web and applications.
>

> I may have made a mistake here. I was under the impression that you had
> told me that your new hierarchy, cross-signed by Certnomis, had issued
> 50,000 certificates. Did I misunderstand? If so, my apologies.

No, or not totally. We have issued those certs but not cross-signed by Certinomis because we didn´t have the cross-signed certificates so, all of them were issued under the new startcom hierarchy

>
> > Ok, I see this is a new requirement that was not imposed last time in which
> you recommended and allowed us to be cross-signed as many other CAs have
> done in the past to be in the business.
>

> Mozilla did not recommend that you be cross-signed.

Well, maybe not Mozilla but we were talking during the F2F meeting in Seattle about options and that one was put on the table

> Certnomis raised the possibility, and we said that it could happen when you had met the BRs and
> completed the remediation steps. The plan we "approved" was not a cross-
> signing plan. The audits show you haven't yet met the BRs, and we have not
> yet signed off on you having completed the remediation steps.

This is something I don´t understand. Why do you say the audits I presented don´t meet the BRs? Because of the findings? The auditors indicate those were fixed
About the remediation steps, well, I answered the bug about it providing all the info and yes, you haven´t answered yet nor to approve nor to deny.

>
> > We´ve met all the conditions, new system, new management, security
> > audit and webtrust audit and CT logging. In those conditions, it was
> > not mentioned that the webtrust audit should be clean
>

> Isn't that implied?

I wanted a clean audit for sure, but I think this also shows our transparency for it. We had findings, those were fixed.

>
> > I´ve never said this. In fact, despite having that cross-signed which were
> provided to us in july we have never used and provided to any of our
> customers to build a trusted path. So none of those 50000, or the new ones,
> go with the Certinomis path because none have it. But all those 50000 certs
> are untrusted because we´re not in the Mozilla root, not the new one, and the
> old one was distrusted.
>

> So the 50,000 certs you mentioned are issued in your old hierarchy, which is
> not cross-signed by Certnomis?

No, it´s from the new hierarchy.

>
> If you could clarify the numbers for your old and new PKI, and confirm that
> the Certnomis cross-signs apply only to the new one, that would be helpful.
> Again, apologies for any misunderstanding on my part.

Certinomis only cross-signed the new hierarchy, in fact, the subordinate CAs not the root CAs, so does not implicit trust the old hierarchy.

>
> > In fact, recently, I asked for permission to use the Certinomis cross-signed
> certificates and have no response. I don´t know if this is an administrative
> silence which may allow me to use it but until having a clear direction we
> haven´t used it.
>

> Can you remind me how you asked and when?

It was in an email of sept 4th, titled "StartCom communication" in which at the end of the long email I asked for feedback to use the cross-signed certificates and give additional explanations

>
> > I think this has been explained. I don´t understand why you say it´s a "poor
> quality PHP code".
>

> Because poor quality PHP code does not become good quality PHP code by
> applying security fixes to it. I am talking about the development
> metholodology and speed which were evident from the Cure53 report.

I´m not agree with this. It´s true that the development methodology and speed were not good the first time but that´s not said in the second one. The methodology was changed same as framework

>
> Gerv

[m...@flanga.io]

I want to give you some words from one of the "community side" (this is a personal opinion and may vary from other opinions inside the community).

Trust is not something that you get, it is something that you earn.
StartCom was distrusted because of serious issues with their old PKI and now had the chance to restart - there are serious issues again. I don't think that the "community" wants rogue CAs on its list just because they restarted with new certificates.

- The fact that you were cross-signed by Certnomis before you had valid WebTrust Audits and the permission to issue trusted certificates again and that the only thing which prevented you from using the trust path is a PUBLIC certificate? Is the only thing that prevents me from entering your datacenter a sign which tells me not to do so and the fact that you did not tell me where your datacenter is located?

- Startcom operates/operated multiple CT Log Server itself. There is absolutely no reason to use trusted certificates for testing purposes if he does have a testing infrastructure. It would be easy for you to add one of your testing roots to your CT Logs and then test your CT behaviour. I don't think that Googles CTs are different from your own ones. Though your certificates might not have been trusted at that time, they would be now and as Gerv said, test certificates are not allowed. If you did not care about compliance at that time, why should you care about it now?

- There is a reason why Best Practices are called best practices. Why did you reuse your key in root and intermediate certificates? Because there is no money for additional HSMs? Because you don't know how to generate new keys? An explanation would be great.

- P-521 are forbidden by Mozilla. Even if there is a discussion to change this it does not allow you to take that as a permission to test it. The fact that these certificates were reported as unrevoked at the time of reporting (as far as I remember) does imply that you do not monitor your certificate issuances for policy compliance at all. What do you do to ensure that all of your certificates are compliant with all requirements at all times?

- What internal audits have you done to ensure the integrity of your systems? If something so critically as the permission to issue certificates in EJBCA is only noticed after you explicitly looked for it, what happens if someone removes all of your security mechanisms? You will find that out too after you misissued thousands of certificates? Quis custodiet ipsos custodes.

- The incidents with Diginotar should have made clear that secure, well audited and hardened code is absolutely necessary as well as reliable logs. The fact that these flaws where not found by your internal team and only discovered after an external company tested your systems is deeply concerning. What have you done now and what will you do to ensure that your systems won't be abused? How do you make sure that the code your people write in the future is safe and how do you detect security problems if you were unable to do so at the first time?

Though I would love to see StartCom up and running again, I have to agree with James that all of these issues do not enwake trust into you and instead produce more uncertainties if StartCom is really able to run a PKI itself. But as I said before, this is a personal opinion :)

[Eric Mill]

I didn't understand the original below comment by StartCom very well about
the cross-sign, but after Ryan's message I understand it better in
retrospect:

> On Thu, Sep 14, 2017 at 11:05 AM, Inigo Barreira via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
>
> I´ve never said this. In fact, despite having that cross-signed which
were provided to us in july we have never used and provided to any of our
customers to build a trusted path. So none of those 50000, or the new ones,
go with the Certinomis path because none have it. But all those 50000 certs
are untrusted because we´re not in the Mozilla root, not the new one, and
the old one was distrusted.
>
> In fact, recently, I asked for permission to use the Certinomis
cross-signed certificates and have no response. I don´t know if this is an
administrative silence which may allow me to use it but until having a
clear direction we haven´t used it.

So this appears to be saying that "all those 50000 certs are untrusted"
because StartCom didn't provide the full chain to customers, even though
such a chain could be constructed. The cross-signature wasn't published in
CT until August 2nd, but that's not any sort of guarantee that the
cross-signature wasn't discoverable by other means -- its availability
until August 2nd is a function of actions by Certinomis that are not
disclosed. The August 2nd date is also after StartCom's actions were being
publicly questioned, so it suggests the possibility that the
cross-signature would have been kept secret for longer, and was only
submitted to CT once scrutiny had increased.

Whether the cross-cert was issued before the audit report date is also a
mystery, especially if it's possible that either Certinomis or StartCom was
operating under the assumption that the cross-signature is irrelevant until
"delivered" to customers.

StartCom has remarked several times in this thread that they are being
treated unfairly, but I can think of at least one comparison to a previous
distrust event, which is that one of the more significant (in my opinion)
issues with Symantec's now-deprecated PKI is that there existed chains that
brought U.S. Federal PKI certificates into being trusted by Mozilla. Those
chains were, as far as I know, never delivered proactively to customers,
but could easily be constructed by any interested party with sufficient
knowledge of the universe of cross-signatures. For example, Qualys' SSL
Labs reports would automatically construct those chains for sites using
FPKI certs, and let users download the full chain in one click.

The threat model here is not what ordinary inexpert customers do, but what
opportunities an adversary has available to them among the universe of
trusted CAs to obtain certificates. In the Symantec/FPKI case, the problem
was that an adversary could easily use an FPKI certificate to intercept
connections made by Mozilla products, whether or not Symantec or the FPKI
ever advertised or proactively enabled this use case. What made this such a
big issue, in addition to the scope of the technical impact, is that the
issue was not noticed or elevated for years, during which multiple
"generations" of cross-signs had been issued and expired. It brought
Symantec's ability to understand their own PKI into serious question.

So I think the biggest issue here is not so much the technical impact, but
that StartCom was communicating inaccurate information to Mozilla. The
certs were publicly trusted by Certinomis, whether the cross-signature was
delivered to StartCom or to customers or to no one. While presumably this
inaccuracy was unintentional, it was enough to cause Gerv to express public
confusion and doubt about whether the certificates were part of the
cross-signed hierarchy. It also reflects a potentially dangerous difference
of perspective between StartCom and root stores in how StartCom evaluates
the trust and impact of the certificates they issue. For a CA that has been
operating for as long as StartCom has, I think it's fair to describe this
as concerning.

I also think that Certinomis, whose cross-signing practices are now being
scrutinized, should proactively post to this list with a timeline of its
own actions during this process, so that their actions can be understood in
the context of StartCom's.

-- Eric

[Inigo Barreira]

>
> I want to give you some words from one of the "community side" (this is a
> personal opinion and may vary from other opinions inside the community).
>
> Trust is not something that you get, it is something that you earn.

True

> StartCom was distrusted because of serious issues with their old PKI and now
> had the chance to restart - there are serious issues again. I don't think that
> the "community" wants rogue CAs on its list just because they restarted with
> new certificates.
>
> - The fact that you were cross-signed by Certnomis before you had valid
> WebTrust Audits and the permission to issue trusted certificates again and
> that the only thing which prevented you from using the trust path is a PUBLIC
> certificate? Is the only thing that prevents me from entering your datacenter
> a sign which tells me not to do so and the fact that you did not tell me where
> your datacenter is located?
>
> - Startcom operates/operated multiple CT Log Server itself. There is absolutely
> no reason to use trusted certificates for testing purposes if he does have a
> testing infrastructure. It would be easy for you to add one of your testing
> roots to your CT Logs and then test your CT behaviour. I don't think that
> Googles CTs are different from your own ones. Though your certificates might
> not have been trusted at that time, they would be now and as Gerv said, test
> certificates are not allowed. If you did not care about compliance at that
> time, why should you care about it now?

Those certificates were not trusted at that time and can´t be now because they were revoked within minutes.

>
> - There is a reason why Best Practices are called best practices. Why did you
> reuse your key in root and intermediate certificates?

> Because there is nomoney for additional HSMs? Because you don't know how to generate new

> keys? An explanation would be great.

A new thread has started about this. It´s not forbidden.

>
> - P-521 are forbidden by Mozilla. Even if there is a discussion to change this it
> does not allow you to take that as a permission to test it. The fact that these
> certificates were reported as unrevoked at the time of reporting (as far as I
> remember) does imply that you do not monitor your certificate issuances for
> policy compliance at all. What do you do to ensure that all of your certificates
> are compliant with all requirements at all times?

At the time of application, the certificates were revoked and countermeasures set and since then there´s no more issues. We have implemented cablint, crt.sh, ... and some other tools into our issuance process and still trying to improve much more.
I´m not trying to excuse we had issues but we corrected them.

>
> - What internal audits have you done to ensure the integrity of your systems?
> If something so critically as the permission to issue certificates in EJBCA is
> only noticed after you explicitly looked for it, what happens if someone
> removes all of your security mechanisms? You will find that out too after you
> misissued thousands of certificates? Quis custodiet ipsos custodes.

Despite all the terrible systems we have, etc. we haven´t misissued thousands of certificates, nor hundreds. The issues we had, have been fixed. Those test certs issued directly from the EJBCA was a mistake, explained many times. I have nothing to add to what I´ve already said. It was not a good decission, not a good practice, and it´s forbidden.

>
> - The incidents with Diginotar should have made clear that secure, well
> audited and hardened code is absolutely necessary as well as reliable logs.
> The fact that these flaws where not found by your internal team and only
> discovered after an external company tested your systems is deeply
> concerning. What have you done now and what will you do to ensure that
> your systems won't be abused? How do you make sure that the code your
> people write in the future is safe and how do you detect security problems if
> you were unable to do so at the first time?

This is a different example, Diginotar was attacked and the attacker was able to enter in their systems, and this is not what happened with StartCom. As said, the code that went live is not the same that was audited the first time and has been improved since then. The audits are just for that, and we will continue doing yearly security audits to improve our systems.

>
> Though I would love to see StartCom up and running again, I have to agree
> with James that all of these issues do not enwake trust into you and instead
> produce more uncertainties if StartCom is really able to run a PKI itself. But as
> I said before, this is a personal opinion :)
>
>
> Am Freitag, 15. September 2017 16:38:25 UTC+2 schrieb Inigo Barreira:

[James Burton]

Why not open-source the code on GitHub and let us be the judge of the improvements made to your systems code? Lets encrypt does this and works successfully.

[Ryan Sleevi]

On Mon, Sep 18, 2017 at 8:12 AM, Inigo Barreira <in...@startcomca.com>
wrote:
>
> We are not seeking to identify personal blame. We are seeking to
> understand what, if any, improvements have been made to address such
> issues. In reading this thread, I have difficulty finding any discussion
> about the steps that StartCom has proposed to improve its awareness of the
> expectations placed upon it as a potential participant in the Mozilla
> store. Regardless of who bears responsibility for that, the absence of a
> robust process - and, unfortunately, the absence of a deep understanding -
> does mean that the restablishing of trust can present a significant risk to
> the community.
>
>
>
> I think I´ve posted everything we did to improve our systems. I replied to
> every error posted in the crt.sh explaining what happened and what we did
> to fix it for not having the same issue again, but will try to recap here
> again.
>
>
>
> - Test certificates. We issued test certificates in production
> to test the CT behaviour. After the checking those certs were revoked,
> within minutes. This was due to an incorrect configuration in the EJBCA
> roles that was changed and updated accordingly for not allowing anyone to
> issue certs from the EJBCA directly
>
> - Use of unallowed curves. We issued certificates with P-521
> which is not allowed by Mozilla. We revoked all those certs and configure
> the system to not allow it. This remediation was put into production on
> mid-july not issuing certs with that curve.
>
> - RSA parameter not included. We issued one certificate which no
> RSA parameter included. We revoked that certificate and started an
> investigation. The EJBCA system didn´t check the keys, concretely for this
> issue. We developed a solution to check the CSR files properly before
> sending to sign
>
> - Country code not allowed. We issued one certificate with
> country code ZR for Zaire, which does not exist officially. We revoked the
> cert and checked our internal country code database with the ISO one. We
> made the correspondent changes. The cert was reissued with the right code
> representing the Democratic Republic of Congo.
>
>
>
> Furthermore, we have added x509lint and cablint to our issuance process.
> We have integrated crt.sh tool into our CMS system. We have developed a CSR
> checking tool. We have updated the EJBCA system to the latest patch,
> 6.0.9.5 which also came with a Key (RSA and ECC) validator, and we are also
> willing to integrate the zlint once is getting more stable. We have applied
> all these tools and we are not misissuing certificates.
>

Unfortunately, I am not sure how to more effectively communicate that this
pattern of issues indicates an organization failure in the review of,
application of, and implementation of the Baseline Requirements. Through
both coding practices and issuing practices, security and compliance are
not being responded to as systemic objectives, but rather as 'one offs',
giving the impression of 'whack-a-mole' and ad-hoc response.

For example, the country code failure indicates a more deeper failing - did
you misunderstand the BRs? Were they not reviewed? Was the code simply not
implemented correctly? With the RSA parameters, it similarly indicates a
lack of attention.

I greatly appreciate the use of and deployment of x509lint and cablint, but
those merely offer technical checking that, as an aspiring trusted root CA,
you should have already been implementing - whether your own or using those
available COTS.

The continued approach to issue-and-revoke rather than holistically review
the practices and take every step possible to ensure compliance -
particularly at a CA that was previously distrusted due to non-compliance -
is a particularly egregious oversight that hasn't been responded to.

In every response, it still feels as if you're suggesting these are
one-offs and coding errors, when the concern being raised is how deeply
indicative they are of systemic failures from top to bottom, from policy to
technology, from oversight to implementation. Rather than demonstrate how
beyond reproach StartCom is, it feels like an excessive emphasis is being
put on the ineffective revocation of these certificates, while ignoring the
issues that lead to them being issued in the first place - both from policy
and from code.

It´s not like disagreements, but the example was about a root certificate
> private key in a USB stick, so IMO that example starts with a very
> problematic issue, because it´s about a root private key and in a USB stick
> left in the table, while the issues Startcom did was about end entity
> certificates, and nothing related to private keys and not in the root,
> that´s what I meant with “quality”.
>

I understand what you meant. I'm suggesting the community has a perception
that the issues StartCom is presently being faced with is as egregious and
as serious. I understand you disagree it's as significant. The suggestion
is that it _is_ this significant - and the failure to respond to why it
isn't, or the responses offered so far, is not particularly inspiring for
trust.


>
>
> But, if we want to talk about the option that someone can do anything in
> the meantime when there´s a lack of security measures or procedures, or
> that you can´t check that those measures, remediations, etc. have been set
> and create uncertainty, etc. I think the best way is to check the CT, in
> which all the certs are logged. Also, in the audits, when an external party
> checks the CA. But, I mean, as any other CA. We´ve seen misissuances from
> all CAs, none is free more or less, and all act the same, remediate the
> issue for not happening again, and how do you check it, with CT, with
> crt.sh, audits, etc. But, if you need anything else, just as kit. I can
> provide whatever you may ask to avoid that uncertainty.
>

This response is deeply concerning, and continues to be.

First, you appear to be suggesting "Everyone makes mistakes, so we should
not be held accountable for our mistakes."
Second, you appear to be suggesting "Audits aren't great, so we shouldn't
put so much trust in them."

Yes, mistakes happen. When a pattern of mistakes happen, CAs get
distrusted. When a CA is distrusted, and continues a pattern of mistakes,
it is not trusted for reapplication.

To avoid the uncertainty is to _not misissue in the first place_. Once that
uncertainty has been introduced - or, in the case of a CA with a pattern of
failures, reintroduced - it is incredibly difficult to regain trust. CAs
that are previously distrusted are, in effect, needed to operate at an even
higher level of compliance and operational efficacy, to demonstrate that
the past issues are not present. Instead, we have multiple data points
suggesting worse.


> I can´t remember of any notification about this regarding Startcom
> certificates. And I don´t think any notification has not been discussed.
> For this certificate, we should start this is a warning, and we have been
> focused in crt.sh errors, to deal with warnings later. This one was due to
> a issue with the translation from unicode to punnycode and a fail in the
> comparision check with CN and SAN. We´re working on it.
>

https://bugzilla.mozilla.org/show_bug.cgi?id=1357067 was due to StartCom
acting as an RA for DNS validation for Camerfirma. Which is now (but not at
the time) forbidden.


> The fact that it was revoked an hour after issuance does not raise
> confidence - it reduces it.

>
>
>
> I don´t think so.
>

Perhaps it's a translation issue, but a better way of phrasing is not "I
don't think so" (which implies that you know whether or not it reduces
confidence), but "I don't think it should". I'm trying to be charitable
here, but it absolutely reduces confidence when a CA actively misissues a
certificate, and then claims revocation is sufficient. No. You never should
have issued such a certificate in the first place. That you have causes a
loss of confidence - revocation does not and cannot restore that by itself.

What restores confidence is how effectively you respond - and revocation
may be part of that answer, but a systemic evaluation of the process and
policies that allowed it to happen, and steps to prevent both the
_specific_ incident from happening and a holistic review of all related
practices so that all other bugs of that class are addressed can.

That is, if you detect, for example, a BR violating cert, a 'minimum'
answer is to revoke it, a 'necessary' action is to prevent that specific BR
violation, but the 'correct' action is to holistically review the BRs, from
top to bottom, and audit against the system that every technical control is
met, and every policy and procedure for non-technical controls is written
and effective.


> I can understand your assertion that these certificates are revoked. That
> does not mean they do not present risk to the ecosystem.
>
>
>
> Then I assume like many other of the certificates revoked from everyone.
>

On the basis of this reply alone, I think it demonstrates a cavalier enough
attitude towards security and compliance - and on emphasizing external
blame rather than the opportunity for internal improvement - that I would
wholly agree with Gerv's suggestion of new keys, and with an explicit
prohibition against any cross-signing from any member CA.

There's simply no room in public trust stores for such approaches to
security - it's fundamentally untrustworthy.


> Yes, it´s true. The certificate was issued in April because Certinomis had
> to do something at their facilities and took the opportunity. That
> certificate was custodied by them until we could provide a valid Webtrust
> certificate, which we did and then disclosed in the CCADB.
>

Then they misissued a CA certificate and failed to disclose it, and we
should start an incident report into it.

[Franck Leroy]

Hello
In April 2017 the mozilla policy in force (v2.4) stated:
“The CA with a certificate included in Mozilla’s CA Certificate Program MUST disclose this information before any such subordinate CA is allowed to issue certificates.”

Our understanding in April was that as long as StartCom is not allowed by Certinomis to issue EE certs, the disclosure was not mandated immediately.

This control that StartCom was not allowed to use our path was technical in place by the fact that I was the only one to have the intermediate cross signed certificates, stored (retained) in my personal safe.

As soon as Certinomis has authorized StartCom to use the path to our root, I disclosed the certificates with the audit reports in the CCADB, and send the certificates to Inigo.

May be I misunderstood the Mozilla requirements v2.4, and as I already said in previous post, I do apologize for it. But it was not my intention not to enforce the policy; I personally took care that StartCom could not be able to use the path to our root until a full BR audit assessment report was provided.

Regards
Franck Leroy

[Nick Lamb]

On Monday, 18 September 2017 15:50:16 UTC+1, Franck Leroy wrote:
> This control that StartCom was not allowed to use our path was technical in place by the fact that I was the only one to have the intermediate cross signed certificates, stored (retained) in my personal safe.

I see. Three (groups of) questions as someone who does not operate a public CA:

When the cross signature certificate was signed did this result in some sort of auditable record of the signing? A paper trial, or its electronic equivalent - so that any audit team would be aware that the certificate existed, regardless of whether they were present when it was created ?

(If so) Was this record inadequate to reproduce the certificate itself, for example just consisting of a serial number and other facts ?

Many important functions of a CA are protected by "no lone zone" type practices, but would it be possible for you to retrieve the certificate from this safe on your own, without oversight by other employees ?

I suspect all the above questions have answers that would be obvious to me if I had worked for a public CA but I hope you will humour me with answers anyway.

[Gervase Markham]

On 15/09/17 15:35, Inigo Barreira wrote:
> No, those weren´t tests. We allowed the use of curves permitted by the BRs but this issue came up in the mozilla policy (I think Arkadiusz posted) and I also asked about it in the last CABF F2F (I asked Ryan about it) and then, with that outcome and as the browsers didn´t accept them, we revoked and then not allow the issuance. I think the discussion is still active (i.e. the use of P-521).

Unless Mozilla says otherwise (which we do occasionally), you should not
be basing your practice on what we are discussing, or even on draft
versions of the policy, but on the published version - which clearly
prohibits P-521.

Gerv

[Inigo Barreira]

Hi Gerv

>
> But once the cross-signed cert is publicly available (and it is; it's in CT,
> however it got there), all of those certificates become trusted (or potentially
> trusted, if the owner reconfigures their webserver to serve the intermediate,
> or if Firefox has already encountered it in the current browsing session).
>

True

> > This is something I don´t understand. Why do you say the audits I
> > presented don´t meet the BRs? Because of the findings? The auditors
> > indicate those were fixed
>

> I don't believe there's a formal way for an auditor to bindingly say "by the
> way, the problems we found have since been fixed" in an audit report.

Not bindingly, we provided a CAP (Corrective Action Plan) for all those findings indicated what we did to fix them and provided the evidences and what we were going to do for those that couldn´t be fixed before receiving the report.


> But to help me understand: exactly what statement on what page of your audit
> report(s) are you referring to here?

It´s in a section called "other questions" in which they say "Startcom has developed a plan of corrective actions with the objective of solving the identified exceptions, having been implemented the majority of these actions".

>
> > About the remediation steps, well, I answered the bug about it providing all
> the info and yes, you haven´t answered yet nor to approve nor to deny.
>

> Right. So why are you proceeding?
>
> You might reasonably complain it's taken us a while to respond to that
> comment about the steps. Yes, it has. The Mozilla inclusion process is slow. :-(

Well, because I wanted to speed up the process if possible. We did everything what was requested and replied the bug. And also applied for the inclussion and none said nothing about it. Kathleen told me that it was going to be slow because the queue was long so I was waiting, no problem, but didn´t know that need to ask permission for applying.

>
> >>> In fact, recently, I asked for permission to use the Certinomis
> >>> cross-signed
> >> certificates and have no response. I don´t know if this is an
> >> administrative silence which may allow me to use it but until having
> >> a clear direction we haven´t used it.
> >>
> >> Can you remind me how you asked and when?
> >
> > It was in an email of sept 4th, titled "StartCom communication" in
> > which at the end of the long email I asked for feedback to use the
> > cross-signed certificates and give additional explanations
>

> I have no record of any email with that title, or any email from you between
> 15th August ("Re: Problem Reporting Mechanism") and 11th September ("Re:
> Remove old Startcom roots from NSS"). Where did you send it?

I sent it to the m.d.s.p list and got a reply from Andrew Ayer almost inmediately.

>
> Gerv

[Franck Leroy]

Hello

You are right, answers are quite obvious, but I don’t understand the purpose of your questions (may be I'm lost in the translation...)

1/ When we use our root, we produce a key ceremony report.
2/ The signature value doesn’t appears in the report so it is not possible to reproduce the certificate.
3/ My safe is in a closet which I don’t have the key, so I have to ask my manager to open it, then I can open my safe this my key.

These are standard practices, and it changes nothing on the fact that we cross signed in April and send the certificate to StartCom in August and that cannot be mathematically proven, just my declaration.


Franck

[James Burton]

Hi Franck,

Could you provide us with a copy of this key ceremony report after removing confidential security information which is not relevant?

James

[userwithuid]

On Tue, Sep 19, 2017 at 3:09 PM, Nick Lamb via dev-security-policy
<dev-secur...@lists.mozilla.org> wrote:
> I have no doubt that this was obvious to people who have worked for a public CA, but it wasn't obvious to me, so thank you for answering. I think these answers give us good reason to be confident that a cross-signed certificate in this situation would not be available to either end subscribers or StartCom unless/ until the CA which cross-signed it wanted that to happen.
>
> It might still make sense for Mozilla to clarify that this isn't a good idea, or even outright forbid it anyway, but I agree with your perspective that this seemed permissible under the rules as you understood them and wasn't obviously unreasonable.

I'm pretty sure it's already forbidden, since policy version 2.5
anyway (has effective date after the Certinomis shenanigans though):

"The CA with a certificate included in Mozilla’s root program MUST
disclose this information within a week of certificate creation, and

before any such subordinate CA is allowed to issue certificates."

2.5 added the "within a week of certificate creation" [1] . "Creation"
vs "My Safe", "Creation" wins. :-)

[1] https://github.com/mozilla/pkipolicy/commit/b7d1b6c04458114fbe73fa3f146ad401235c2a1b

[Gervase Markham]

Dear Inigo,

On 14/09/17 09:49, Gervase Markham wrote:
> The Mozilla CA Certificates team has been considering what the
> appropriate next steps are for the inclusion request from the CA
> "StartCom".[0] As readers will know, this CA has previously been removed
> from trust[1], and so a re-application obviously involves particular
> scrutiny. In addition, several questions have been raised about the way
> in which the new StartCom PKI has been operated technically[2]. This is
> a proposal for the way forward, on which comments are invited.

A month ago we posted the above re: StartCom, in what was officially a
request for comments. The ensuing discussion did not lead us to conclude
that anything we were asking for was inappropriate. So we want to
formally confirm that it is indeed Mozilla's requirement that StartCom
begin again with a new-new hierarchy and do the other things outlined in
the original post before we will further consider a root inclusion
request from StartCom.

Gerv