info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Questions regarding the qualifications and competency of TUVIT
2,253 views
Skip to first unread message
Ryan Sleevi
Oct 30, 2018, 11:20:31 AM10/30/18
to mozilla-dev-security-policy
(Writing with an individual hat)
I would like to suggest that consideration be given to rejecting future
audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
Widermann, for some period of time. I would suggest this period be at least
one year long; however, given the technical details of ETSI accreditation,
believe a period of three years may be more appropriate.
As part of an investigation into the incorrect qcStatements reported at
https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
, I've used this as an example to explore the complaint handling process
under ETSI EN 319 403.
For those not familiar with the process, ETSI EN 319 403 normatively
incorporates various portions of ISO/IEC 17065, which is an international
standard for Conformity Assessment Bodies. Under the eIDAS scheme, auditors
(henceforth called Conformity Assessment Bodies, or CAB) are assessed by
their National Accreditation Body, or NAB, as being accredited under EN 319
403 to conduct audits against the schemes in ETSI EN 319 411-1 and ETSI EN
319 411-2. When a CA (called a Trust Service Provider, or TSP, by eIDAS)
has been audited (accredited) by a CAB against the scheme in EN 319 411-1
or 411-2, the CAB is applying the principles in EN 319 403.
If there is a belief that a TSP has failed to meet the requirements of
their accreditation, EN 319 403 describes a process for which complaints
may be made to either the TSP or to the CAB. This complaint process is
further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
same process also applies when there have been mistakes by the CAB to
adhere to its scheme requirements under EN 319 403 - a complaint may be
made with either the CAB or the NAB regarding the CAB's accreditation.
Each CAB and NAB must make available their information for processing
complaints. Given that 100% of the qualified certificates issued by
T-Systems have failed to comply with the requirements of ETSI EN 319 411-2,
by virtue of failing to comply with ETSI EN 319 412-5, I lodged a complaint
regarding T-System's certification with TUVIT, which certified T-Systems.
As part of processing the complaint, the following issues regarding TUVIT's
handling of complaints and overall approach to audits were revealed.
Issue A) As part of their initial response to my complaint, TUVIT, by way
of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
very first, quick cross check with our audit evidence record, I can only
say that we did check issued certificates from within the declared period
and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
3". However, this statement was in direct conflict with the TSP's own
investigation and incident report, provided at
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
mistake was introduced during the development of support - that is, no such
properly issued certificates were issued.
In follow-up with TUVIT, I requested they provide any evidence to support
the claim that such certificates were issued, knowing the matter reported
by T-Systems means this is highly improbable.
In subsequent reply, Matthias Wiedenhorst acknowledged that their
methodology for testing compliance with ETSI EN 319 412-5 stated that the
process was "the auditors have to check the website certificates manually
for the presence of the QC-statement. In this case, the existence of the
id-etsi-qct-web was identified, but for any reason it was not realized that
the esi4-qcStatements-6 was missing. We informed the auditors about how to
check the QC-statement correctly so that we are confident, that in future
an incorrect coding will be detected. "
This implies a significant lack of core competencies within TUVIT to assess
against the criteria of ETSI EN 319 412-5, which provides a normative ASN.1
module within its specification.
Issue C) In regards to remedies, I requested a suspension of T-Systems
current ETSI EN 319 411-2 accreditation, and a subsequent assessment of a
Full audit, given concerns with respect to the audit team performing the
initial audit. TUVIT declined this, on the basis that they assess such
technical non-compliance to be a 'minor non-conformity' that, per ETSI EN
319 403, may be resolved in the next three months by T-Systems without
adversely affecting their accreditation.
Further, T-Systems and TUVIT decided that, on the basis that it is a
non-conformity, such certificates do not need to be revoked according to
the Baseline Requirements' timeline. Furthermore, the failure to revoke
according to the Baseline Requirements' timeline was a further minor
non-conformity, not adversely affecting the certification decision of
T-Systems, and may be remedied over the next 3 months.
Furthermore, in response to whether or not such matters of non-conformities
would be reported by TUVIT (the CAB), Matthis Wiedenhorst acknowledged that
they are not required to make such information public at this time.
I believe this significantly undermines the level of confidence and
assurance that relying parties should place in audits conducted by TUVIT.
The inducement to non-compliant actions, combined with the non-disclosure
of these "non-conformities" to affected parties, is at odds with the
principles and values of the respective Root Programs' goals.
In addition to these concerns with respect to the audits provided, I will
be further lodging a complaint with the German National Accreditation
Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
handling of this incident, the material misstatement as to what was
examined, combined with the failure to ensure appropriate and prompt
notification to the CAB and the Supervisory Body (as detailed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
question the compliance with Regulation (EU) N°910/2014 on behalf of both
T-Systems and TUVIT. In particular, the failure to perform a timely
notification under Article 19(2) of the "loss of integrity that has a
significant impact on the trust service provider" represents a significant
breach of trust.
With respect to what constitutes a "loss of integrity", both Article 8 and
Article 7(f) note compliance to the appropriate technical standards and the
prohibition of any "specific disproportionate technical requirements on
relying parties ..." that would "significantly impede the interoperability
of the notified electronic identification schemes".
Through their failure to assure a timely disclosure, and the failure to
revoke, TUVIT's certifications disproportionately and adversely affect
Relying Parties. Such certificates assert that they are qualified
certificates, and thus Relying Parties should be able to rely upon them for
their constitutive value. However, in the absence of the appropriate and
mandatory notice as to the qualified purpose of such certificates, as this
incident demonstrates, Relying Parties are left without an interoperable
means of determining whether or not such Qualified Certificates are indeed
qualified, and thus subject to the protections afforded by eIDAS.
Furthermore, the complaint process established under EN 319 403 refers to
that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
put forward within 7.13.5 and 7.13.6 that the complaint resolution process
be independent from those from those involved in the audit or those who
have consulted with or been employed by the subject of the audit. Matthias
Wiedenhorst's involvement in, and resolution of, this complaint, calls into
question whether TUVIT has acted according to this, given that they are
represented as Lead Auditor for T-Systems' audit. At this point, the only
objective means of resolving whether or not the process was followed is to
lodge a complaint such that the NAB may themselves investigate the handling
of this complaint.
Given that the Supervisory Body and National Accreditation bodies exist to
protect the legal value of this scheme, the failure by TUVIT to uphold the
safety and security of the eIDAS regime represents an ongoing threat to the
ecosystem.
I would like to suggest that consideration be given to rejecting future
audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
Widermann, for some period of time. I would suggest this period be at least
one year long; however, given the technical details of ETSI accreditation,
believe a period of three years may be more appropriate.
As part of an investigation into the incorrect qcStatements reported at
https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
, I've used this as an example to explore the complaint handling process
under ETSI EN 319 403.
For those not familiar with the process, ETSI EN 319 403 normatively
incorporates various portions of ISO/IEC 17065, which is an international
standard for Conformity Assessment Bodies. Under the eIDAS scheme, auditors
(henceforth called Conformity Assessment Bodies, or CAB) are assessed by
their National Accreditation Body, or NAB, as being accredited under EN 319
403 to conduct audits against the schemes in ETSI EN 319 411-1 and ETSI EN
319 411-2. When a CA (called a Trust Service Provider, or TSP, by eIDAS)
has been audited (accredited) by a CAB against the scheme in EN 319 411-1
or 411-2, the CAB is applying the principles in EN 319 403.
If there is a belief that a TSP has failed to meet the requirements of
their accreditation, EN 319 403 describes a process for which complaints
may be made to either the TSP or to the CAB. This complaint process is
further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
same process also applies when there have been mistakes by the CAB to
adhere to its scheme requirements under EN 319 403 - a complaint may be
made with either the CAB or the NAB regarding the CAB's accreditation.
Each CAB and NAB must make available their information for processing
complaints. Given that 100% of the qualified certificates issued by
T-Systems have failed to comply with the requirements of ETSI EN 319 411-2,
by virtue of failing to comply with ETSI EN 319 412-5, I lodged a complaint
regarding T-System's certification with TUVIT, which certified T-Systems.
As part of processing the complaint, the following issues regarding TUVIT's
handling of complaints and overall approach to audits were revealed.
Issue A) As part of their initial response to my complaint, TUVIT, by way
of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
very first, quick cross check with our audit evidence record, I can only
say that we did check issued certificates from within the declared period
and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
3". However, this statement was in direct conflict with the TSP's own
investigation and incident report, provided at
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
mistake was introduced during the development of support - that is, no such
properly issued certificates were issued.
In follow-up with TUVIT, I requested they provide any evidence to support
the claim that such certificates were issued, knowing the matter reported
by T-Systems means this is highly improbable.
In subsequent reply, Matthias Wiedenhorst acknowledged that their
methodology for testing compliance with ETSI EN 319 412-5 stated that the
process was "the auditors have to check the website certificates manually
for the presence of the QC-statement. In this case, the existence of the
id-etsi-qct-web was identified, but for any reason it was not realized that
the esi4-qcStatements-6 was missing. We informed the auditors about how to
check the QC-statement correctly so that we are confident, that in future
an incorrect coding will be detected. "
This implies a significant lack of core competencies within TUVIT to assess
against the criteria of ETSI EN 319 412-5, which provides a normative ASN.1
module within its specification.
Issue C) In regards to remedies, I requested a suspension of T-Systems
current ETSI EN 319 411-2 accreditation, and a subsequent assessment of a
Full audit, given concerns with respect to the audit team performing the
initial audit. TUVIT declined this, on the basis that they assess such
technical non-compliance to be a 'minor non-conformity' that, per ETSI EN
319 403, may be resolved in the next three months by T-Systems without
adversely affecting their accreditation.
Further, T-Systems and TUVIT decided that, on the basis that it is a
non-conformity, such certificates do not need to be revoked according to
the Baseline Requirements' timeline. Furthermore, the failure to revoke
according to the Baseline Requirements' timeline was a further minor
non-conformity, not adversely affecting the certification decision of
T-Systems, and may be remedied over the next 3 months.
Furthermore, in response to whether or not such matters of non-conformities
would be reported by TUVIT (the CAB), Matthis Wiedenhorst acknowledged that
they are not required to make such information public at this time.
I believe this significantly undermines the level of confidence and
assurance that relying parties should place in audits conducted by TUVIT.
The inducement to non-compliant actions, combined with the non-disclosure
of these "non-conformities" to affected parties, is at odds with the
principles and values of the respective Root Programs' goals.
In addition to these concerns with respect to the audits provided, I will
be further lodging a complaint with the German National Accreditation
Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
handling of this incident, the material misstatement as to what was
examined, combined with the failure to ensure appropriate and prompt
notification to the CAB and the Supervisory Body (as detailed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
question the compliance with Regulation (EU) N°910/2014 on behalf of both
T-Systems and TUVIT. In particular, the failure to perform a timely
notification under Article 19(2) of the "loss of integrity that has a
significant impact on the trust service provider" represents a significant
breach of trust.
With respect to what constitutes a "loss of integrity", both Article 8 and
Article 7(f) note compliance to the appropriate technical standards and the
prohibition of any "specific disproportionate technical requirements on
relying parties ..." that would "significantly impede the interoperability
of the notified electronic identification schemes".
Through their failure to assure a timely disclosure, and the failure to
revoke, TUVIT's certifications disproportionately and adversely affect
Relying Parties. Such certificates assert that they are qualified
certificates, and thus Relying Parties should be able to rely upon them for
their constitutive value. However, in the absence of the appropriate and
mandatory notice as to the qualified purpose of such certificates, as this
incident demonstrates, Relying Parties are left without an interoperable
means of determining whether or not such Qualified Certificates are indeed
qualified, and thus subject to the protections afforded by eIDAS.
Furthermore, the complaint process established under EN 319 403 refers to
that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
put forward within 7.13.5 and 7.13.6 that the complaint resolution process
be independent from those from those involved in the audit or those who
have consulted with or been employed by the subject of the audit. Matthias
Wiedenhorst's involvement in, and resolution of, this complaint, calls into
question whether TUVIT has acted according to this, given that they are
represented as Lead Auditor for T-Systems' audit. At this point, the only
objective means of resolving whether or not the process was followed is to
lodge a complaint such that the NAB may themselves investigate the handling
of this complaint.
Given that the Supervisory Body and National Accreditation bodies exist to
protect the legal value of this scheme, the failure by TUVIT to uphold the
safety and security of the eIDAS regime represents an ongoing threat to the
ecosystem.
Kurt Roeckx
Oct 30, 2018, 11:59:31 AM10/30/18
to mozilla-dev-s...@lists.mozilla.org
On 2018-10-30 16:20, Ryan Sleevi wrote:
> Given that the Supervisory Body and National Accreditation bodies exist to
> protect the legal value of this scheme, the failure by TUVIT to uphold the
> safety and security of the eIDAS regime represents an ongoing threat to the
> ecosystem.
Do we have a way of verifying the accreditation, and do we verify that
> Given that the Supervisory Body and National Accreditation bodies exist to
> protect the legal value of this scheme, the failure by TUVIT to uphold the
> safety and security of the eIDAS regime represents an ongoing threat to the
> ecosystem.
they have a valid accreditation? Should it be enough for us to check the
accreditation, and just follow the process you are already doing?
Kurt
Ryan Sleevi
Oct 30, 2018, 12:29:14 PM10/30/18
to Kurt Roeckx, mozilla-dev-security-policy
On Tue, Oct 30, 2018 at 11:59 AM Kurt Roeckx via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> On 2018-10-30 16:20, Ryan Sleevi wrote:
dev-secur...@lists.mozilla.org> wrote:
> On 2018-10-30 16:20, Ryan Sleevi wrote:
> > Given that the Supervisory Body and National Accreditation bodies exist
> to
> > protect the legal value of this scheme, the failure by TUVIT to uphold
> the
> > safety and security of the eIDAS regime represents an ongoing threat to
> the
> > ecosystem.
>
> to
> > protect the legal value of this scheme, the failure by TUVIT to uphold
> the
> > safety and security of the eIDAS regime represents an ongoing threat to
> the
> > ecosystem.
>
> Do we have a way of verifying the accreditation, and do we verify that
> they have a valid accreditation? Should it be enough for us to check the
> accreditation, and just follow the process you are already doing?
>
Yes. You can either begin with a 'top-down' approach or a 'bottom-up'
> they have a valid accreditation? Should it be enough for us to check the
> accreditation, and just follow the process you are already doing?
>
approach, depending on the information you have at hand. Conceptually, it's
very similar to Revocation Checking - and just as conceptually broken.
To begin with a 'bottom-up' approach, we start with the CA being assessed.
We'll use https://crt.sh/?id=3726125 as an example.
>From there, we then look at the audit, which leads to
https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072004_Audit_Attestation_E_T-TeleSec-GlobalRoot-Class-3_20180723_s.pdf
>From this audit, we learn that TÜV Informationstechnik GmbH is accredited
by DAkkS with certificate D-ZE-12022-01 under ETSI EN 319 403 v2.2.2
In this case, TUVIT has included a direct link to their certificate in the
footnotes, but you could otherwise look up with DAkkS directly. In either
event,
https://www.dakks.de/en/content/accredited-bodies-dakks?Regnr=D-ZE-12022-01-01
takes you to the certification
You can then view the certificate itself at
https://www.dakks.de/as/ast/d/D-ZE-12022-01-01.pdf
>From a top-down approach, you'd start with identifying who the NABs are
under the eIDAS scheme. As EU Regulation No. 910/2014 builds upon EU
Regulation No 765/2008 with respect for the establishment of NABs, your
starting point is with http://www.european-accreditation.org/
>From there, you look for Members or MLA/BLA Signatories (with respect to
ISO 17065 and/or EN 319 403), and you can determine that the NAB for
Germany is DAkkS ( http://www.european-accreditation.org/ea-members )
>From DAkkS, you can then examine the Directory of accredited bodies (
https://www.dakks.de/en/content/directory-accredited-bodies-0 ) and search
for the relevant Conformity Assessment Bodies certifications
Both approaches lead you to the certification of TUVIT. If your question
was with respect to T-Systems' certification, you follow roughly that same
process, with the top-down approach also involving looking through TUVIT's
directory of accredited TSPs to determine if T-Systems is accredited.
This establishes who the CAB is and who the NAB is. As the scheme used in
eIDAS for CABs is ETSI EN 319 403, the CAB must perform their assessments
in concordance with this scheme, and the NAB is tasked with assessing their
qualification and certification under any local legislation (if
appropriate) or, lacking such, under the framework for the NAB applying the
principles of ISO/IEC 17065 in evaluating the CAB against EN 319 403. The
NAB is the singular national entity recognized for issuing certifications
against ISO/IEC 17065 through the MLA/BLA and the EU Regulation No 765/2008
(as appropriate), which is then recognized trans-nationally.
As the framework utilizes ISO/IEC 17065, the complaints process and
certification process for both TSPs and CABs bears strong similarity, which
is why I wanted to explore how this process works in function.
Note that if either the TSP is suspended of their certification or
withdrawn, no notification will be made to relying parties. The closest
that it comes is that if they're accredited according to EN 319 411-2
(Qualified Certificates), the suspension/withdrawing will be reported to
the Supervisory Body, which will them update the Qualified Trust List for
that country and that will flow into the EU Qualified Trust List. If
they're accredited against EN 319 411-1, the Supervisory Body will be
informed by the CAB (in theory, although note my complaint about TSP
informing the CAB was not followed, and the same can exist with CAB to SB),
but no further notification may be made. Furthermore, if certification is
later reissued, after a full audit, the certification history will not
reflect that there was a period of 'failed' certification. This similarly
exists with respect to CABs - if a CAB has their accreditation suspended,
on the advice of or decision of the NAB based on feedback from the SB - the
community will not necessarily be informed. In theory, because
certification is 'forward' looking rather than 'past' looking, a suspension
or withdraw of a CAB by a NAB may not affect its past certification of
TSPs; this is an area of process that has not been well-specified or
determined.
Erwann Abalea
Oct 30, 2018, 1:10:02 PM10/30/18
to mozilla-dev-s...@lists.mozilla.org
Bonjour,
Le mardi 30 octobre 2018 16:20:31 UTC+1, Ryan Sleevi a écrit :
> (Writing with an individual hat)
>
> I would like to suggest that consideration be given to rejecting future
> audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
> Widermann, for some period of time. I would suggest this period be at least
> one year long; however, given the technical details of ETSI accreditation,
> believe a period of three years may be more appropriate.
>
> As part of an investigation into the incorrect qcStatements reported at
> https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
> , I've used this as an example to explore the complaint handling process
> under ETSI EN 319 403.
[...]
> In addition to these concerns with respect to the audits provided, I will
> be further lodging a complaint with the German National Accreditation
> Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
> under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
> handling of this incident, the material misstatement as to what was
> examined, combined with the failure to ensure appropriate and prompt
> notification to the CAB and the Supervisory Body (as detailed in
> https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
> question the compliance with Regulation (EU) N°910/2014 on behalf of both
> T-Systems and TUVIT. In particular, the failure to perform a timely
> notification under Article 19(2) of the "loss of integrity that has a
> significant impact on the trust service provider" represents a significant
> breach of trust.
>
> With respect to what constitutes a "loss of integrity", both Article 8 and
> Article 7(f) note compliance to the appropriate technical standards and the
> prohibition of any "specific disproportionate technical requirements on
> relying parties ..." that would "significantly impede the interoperability
> of the notified electronic identification schemes".
Articles 7 and 8 have nothing to do with that kind of notification (for security breaches), they're related to notification of identification schemes to the European Commission.
> Through their failure to assure a timely disclosure, and the failure to
> revoke, TUVIT's certifications disproportionately and adversely affect
> Relying Parties. Such certificates assert that they are qualified
> certificates, and thus Relying Parties should be able to rely upon them for
> their constitutive value. However, in the absence of the appropriate and
> mandatory notice as to the qualified purpose of such certificates, as this
> incident demonstrates, Relying Parties are left without an interoperable
> means of determining whether or not such Qualified Certificates are indeed
> qualified, and thus subject to the protections afforded by eIDAS.
In fact, for the Relying Party, these certificates are definitely considered as Qualified certificates for website authentication, regardless of the content of the QCStatement extension.
Grab the German TL, find the T-Systems TSP, this specific service, you'll see it's declared as a CA/QC type, status granted, with a Sie equal to ForWebSiteAuthentication. There is no ambiguity (yet).
Are we going to also revoke all the certificates which contain encoded DEFAULT values (in the ASN.1 sense), invalid PrintableString attributes, invalid hostnames, and reject audits performed by auditors who missed such certificates?
This esi4-qcStatement-6 QCStatement is a recent addition, has been really poorly designed (a SEQUENCE OF that shall contain only 1 element, what a great idea), and has seen several changes during the draft. It's an easy statement, I agree, and a decent TSP shouldn't make any mistake in encoding it.
But on the control side, there's not that much available tool to decode a QCStatements extension (and no, "openssl asn1parse" and "dumpasn1" don't count).
Le mardi 30 octobre 2018 16:20:31 UTC+1, Ryan Sleevi a écrit :
> (Writing with an individual hat)
>
> I would like to suggest that consideration be given to rejecting future
> audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
> Widermann, for some period of time. I would suggest this period be at least
> one year long; however, given the technical details of ETSI accreditation,
> believe a period of three years may be more appropriate.
>
> As part of an investigation into the incorrect qcStatements reported at
> https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
> , I've used this as an example to explore the complaint handling process
> under ETSI EN 319 403.
> In addition to these concerns with respect to the audits provided, I will
> be further lodging a complaint with the German National Accreditation
> Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
> under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
> handling of this incident, the material misstatement as to what was
> examined, combined with the failure to ensure appropriate and prompt
> notification to the CAB and the Supervisory Body (as detailed in
> https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
> question the compliance with Regulation (EU) N°910/2014 on behalf of both
> T-Systems and TUVIT. In particular, the failure to perform a timely
> notification under Article 19(2) of the "loss of integrity that has a
> significant impact on the trust service provider" represents a significant
> breach of trust.
>
> With respect to what constitutes a "loss of integrity", both Article 8 and
> Article 7(f) note compliance to the appropriate technical standards and the
> prohibition of any "specific disproportionate technical requirements on
> relying parties ..." that would "significantly impede the interoperability
> of the notified electronic identification schemes".
> Through their failure to assure a timely disclosure, and the failure to
> revoke, TUVIT's certifications disproportionately and adversely affect
> Relying Parties. Such certificates assert that they are qualified
> certificates, and thus Relying Parties should be able to rely upon them for
> their constitutive value. However, in the absence of the appropriate and
> mandatory notice as to the qualified purpose of such certificates, as this
> incident demonstrates, Relying Parties are left without an interoperable
> means of determining whether or not such Qualified Certificates are indeed
> qualified, and thus subject to the protections afforded by eIDAS.
Grab the German TL, find the T-Systems TSP, this specific service, you'll see it's declared as a CA/QC type, status granted, with a Sie equal to ForWebSiteAuthentication. There is no ambiguity (yet).
Are we going to also revoke all the certificates which contain encoded DEFAULT values (in the ASN.1 sense), invalid PrintableString attributes, invalid hostnames, and reject audits performed by auditors who missed such certificates?
This esi4-qcStatement-6 QCStatement is a recent addition, has been really poorly designed (a SEQUENCE OF that shall contain only 1 element, what a great idea), and has seen several changes during the draft. It's an easy statement, I agree, and a decent TSP shouldn't make any mistake in encoding it.
But on the control side, there's not that much available tool to decode a QCStatements extension (and no, "openssl asn1parse" and "dumpasn1" don't count).
Erwann Abalea
Oct 30, 2018, 1:20:06 PM10/30/18
to mozilla-dev-s...@lists.mozilla.org
Le mardi 30 octobre 2018 17:29:14 UTC+1, Ryan Sleevi a écrit :
[...]
[...]
> Note that if either the TSP is suspended of their certification or
> withdrawn, no notification will be made to relying parties. The closest
> that it comes is that if they're accredited according to EN 319 411-2
> (Qualified Certificates), the suspension/withdrawing will be reported to
> the Supervisory Body, which will them update the Qualified Trust List for
> that country and that will flow into the EU Qualified Trust List.
Quick correction here: this certification suspension/withdrawal does not automatically imply a qualification suspension/withdrawal by the SB. The SB is the sole responsible of the TL content, and can ignore the certification suspension (or certification success, failure, absence, or whatever).
> withdrawn, no notification will be made to relying parties. The closest
> that it comes is that if they're accredited according to EN 319 411-2
> (Qualified Certificates), the suspension/withdrawing will be reported to
> the Supervisory Body, which will them update the Qualified Trust List for
> that country and that will flow into the EU Qualified Trust List.
Ryan Sleevi
Oct 30, 2018, 1:28:50 PM10/30/18
to eab...@gmail.com, mozilla-dev-security-policy
On Tue, Oct 30, 2018 at 1:10 PM Erwann Abalea via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> In fact, for the Relying Party, these certificates are definitely
> considered as Qualified certificates for website authentication, regardless
> of the content of the QCStatement extension.
> Grab the German TL, find the T-Systems TSP, this specific service, you'll
> see it's declared as a CA/QC type, status granted, with a Sie equal to
> ForWebSiteAuthentication. There is no ambiguity (yet).
>
On what basis do you believe this claim is to be made? By virtue of
dev-secur...@lists.mozilla.org> wrote:
> In fact, for the Relying Party, these certificates are definitely
> considered as Qualified certificates for website authentication, regardless
> of the content of the QCStatement extension.
> Grab the German TL, find the T-Systems TSP, this specific service, you'll
> see it's declared as a CA/QC type, status granted, with a Sie equal to
> ForWebSiteAuthentication. There is no ambiguity (yet).
>
asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
was absent, do you believe the same?
I'm not sure the ambiguity can be as easily resolved as you suggest, given
the description within EN 319 412-5
> Are we going to also revoke all the certificates which contain encoded
> DEFAULT values (in the ASN.1 sense), invalid PrintableString attributes,
> invalid hostnames,
through both https://wiki.mozilla.org/CA/Incident_Dashboard and the
https://wiki.mozilla.org/CA/Closed_Incidents
It is interesting that you chose those examples, as several are explicitly
called out in the Root Policy at
https://github.com/mozilla/pkipolicy/blob/master/rootstore/policy.md#52-forbidden-and-required-practices
> and reject audits performed by auditors who missed such certificates?
>
the competency of the auditor, as has the failure to disclose such issues.
Both are relevant here, combined with the approach to revocation and
overall testing methodology. Further, as detailed, the misleading remarks
regarding what was examined are equally reason to question the competency
of the auditor.
This is no different than the rejection of audits from some
WebTrust-accredited practioners due to significant oversights about ongoing
and persistent misissuance that is critical within the scope of the audit
scheme being used. The failure to detect such issues fundamentally calls
into question the validity of the current audit, as well as those audits
performed for other CAs. The response of the auditor to such issues equally
bears calling into question the competencies of the auditor.
> This esi4-qcStatement-6 QCStatement is a recent addition, has been really
> poorly designed (a SEQUENCE OF that shall contain only 1 element, what a
> great idea), and has seen several changes during the draft. It's an easy
> statement, I agree, and a decent TSP shouldn't make any mistake in encoding
> it.
> But on the control side, there's not that much available tool to decode a
> QCStatements extension (and no, "openssl asn1parse" and "dumpasn1" don't
> count).
>
as a normative inclusion (Annex B), I find this profoundly non-compelling.
See
https://www.etsi.org/deliver/etsi_en/319400_319499/31941205/02.01.01_60/en_31941205v020101p.pdf
.
As you dig through these versions, the adopted versions do not share the
ambiguity issues. You're correct that 2.2.0 formalized the corrigenda
against 2.2.1 to include, textually, the normative requirement of "one and
exactly one" method, but in either event, such encodings violate it
entirely.
I also fail to understand how one can argue that the CAB is following
industry best practice, considering that industry best practice has
included within it the use of tools such as certlint (which can ingest
ASN.1 modules) or zlint (for which compliance support can be included). In
any event, the testing procedure of visual inspection without actually
conforming against the grammar is a fundamental approach to audit
methodologies that does not stand up to scrutiny, and seriously calls into
question the core competencies.
Moudrick M. Dadashov
Oct 30, 2018, 1:30:11 PM10/30/18
to ry...@sleevi.com, Kurt Roeckx, mozilla-dev-security-policy
Thanks for good overview.
I'd like to add some more.
Actually the most questionalble part of the chain is so called Supervisory bodies.
Of course, root programs do not rely on SB assessment, but under eIDAS they are authorised to audit TSPs and then publish National trust lists (as Scheme operators under the Commission implementing decision 2015/1505). So anyone relying on the list without sufficient care, should assume the adequate risk.
Thanks,M.D.
Sent from my Samsung device
Note that if either the TSP is suspended of their certification or
withdrawn, no notification will be made to relying parties. The closest
that it comes is that if they're accredited according to EN 319 411-2
(Qualified Certificates), the suspension/withdrawing will be reported to
the Supervisory Body, which will them update the Qualified Trust List for
that country and that will flow into the EU Qualified Trust List. If
they're accredited against EN 319 411-1, the Supervisory Body will be
informed by the CAB (in theory, although note my complaint about TSP
informing the CAB was not followed, and the same can exist with CAB to SB),
but no further notification may be made. Furthermore, if certification is
later reissued, after a full audit, the certification history will not
reflect that there was a period of 'failed' certification. This similarly
exists with respect to CABs - if a CAB has their accreditation suspended,
on the advice of or decision of the NAB based on feedback from the SB - the
community will not necessarily be informed. In theory, because
certification is 'forward' looking rather than 'past' looking, a suspension
or withdraw of a CAB by a NAB may not affect its past certification of
TSPs; this is an area of process that has not been well-specified or
determined.
_______________________________________________
dev-security-policy mailing list
dev-secur...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy
Erwann Abalea
Oct 30, 2018, 4:37:31 PM10/30/18
to mozilla-dev-s...@lists.mozilla.org
Le mardi 30 octobre 2018 18:28:50 UTC+1, Ryan Sleevi a écrit :
> On Tue, Oct 30, 2018 at 1:10 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > In fact, for the Relying Party, these certificates are definitely
> > considered as Qualified certificates for website authentication, regardless
> > of the content of the QCStatement extension.
> > Grab the German TL, find the T-Systems TSP, this specific service, you'll
> > see it's declared as a CA/QC type, status granted, with a Sie equal to
> > ForWebSiteAuthentication. There is no ambiguity (yet).
Sorry, I was reading the TL too fast; this certificate is Qualified, but not a QWAC.
> On Tue, Oct 30, 2018 at 1:10 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > In fact, for the Relying Party, these certificates are definitely
> > considered as Qualified certificates for website authentication, regardless
> > of the content of the QCStatement extension.
> > Grab the German TL, find the T-Systems TSP, this specific service, you'll
> > see it's declared as a CA/QC type, status granted, with a Sie equal to
> > ForWebSiteAuthentication. There is no ambiguity (yet).
> On what basis do you believe this claim is to be made? By virtue of
> asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
> was absent, do you believe the same?
2015/1505 can be found at https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32015D1505
My mistake was that I looked at the Sie and didn't check if there was a QualificationsExtension node.
> I'm not sure the ambiguity can be as easily resolved as you suggest, given
> the description within EN 319 412-5
> > Are we going to also revoke all the certificates which contain encoded
> > DEFAULT values (in the ASN.1 sense), invalid PrintableString attributes,
> > invalid hostnames,
>
> Yes. This has already been the process now for several years, as shown
> through both https://wiki.mozilla.org/CA/Incident_Dashboard and the
> https://wiki.mozilla.org/CA/Closed_Incidents
>
> It is interesting that you chose those examples, as several are explicitly
> called out in the Root Policy at
> https://github.com/mozilla/pkipolicy/blob/master/rootstore/policy.md#52-forbidden-and-required-practices
> As you dig through these versions, the adopted versions do not share the
> ambiguity issues. You're correct that 2.2.0 formalized the corrigenda
> against 2.2.1 to include, textually, the normative requirement of "one and
> exactly one" method, but in either event, such encodings violate it
> entirely.
Erwann Abalea
Oct 30, 2018, 4:43:01 PM10/30/18
to mozilla-dev-s...@lists.mozilla.org
Le mardi 30 octobre 2018 18:30:11 UTC+1, Moudrick M. Dadashov a écrit :
> Thanks for good overview.
> I'd like to add some more.
> Actually the most questionalble part of the chain is so called Supervisory bodies.
> Of course, root programs do not rely on SB assessment, but under eIDAS they are authorised to audit TSPs and then publish National trust lists (as Scheme operators under the Commission implementing decision 2015/1505). So anyone relying on the list without sufficient care, should assume the adequate risk.
More precisely, SB have the duty to supervise Qualified TSPs, and maintain the TL (i.e. they're not just "authorized to" do that).
> Thanks for good overview.
> I'd like to add some more.
> Actually the most questionalble part of the chain is so called Supervisory bodies.
> Of course, root programs do not rely on SB assessment, but under eIDAS they are authorised to audit TSPs and then publish National trust lists (as Scheme operators under the Commission implementing decision 2015/1505). So anyone relying on the list without sufficient care, should assume the adequate risk.
And by law, whatever is contained in the EUMS-TL shall be trusted and accepted. If a SB publishes a TL where Honest Ahmed is a QTSP, European Relying Parties have no choice but accept that.
Erwann Abalea
Oct 30, 2018, 5:09:04 PM10/30/18
to ryan....@gmail.com, mozilla-dev-s...@lists.mozilla.org
Not seeing this on Google Groups :/
Le mar. 30 oct. 2018 à 18:28, Ryan Sleevi <ryan....@gmail.com> a écrit :
>
>
> On Tue, Oct 30, 2018 at 1:20 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> Le mardi 30 octobre 2018 17:29:14 UTC+1, Ryan Sleevi a écrit :
>> [...]
>
Other that the eIDAS regulation? No.
What you wrote would mean that the CAB is finally responsible of the
Qualified status of a TSP. And this is wrong.
--
Erwann.
Le mar. 30 oct. 2018 à 18:28, Ryan Sleevi <ryan....@gmail.com> a écrit :
>
>
> On Tue, Oct 30, 2018 at 1:20 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> Le mardi 30 octobre 2018 17:29:14 UTC+1, Ryan Sleevi a écrit :
>> [...]
>> > Note that if either the TSP is suspended of their certification or
>> > withdrawn, no notification will be made to relying parties. The closest
>> > that it comes is that if they're accredited according to EN 319 411-2
>> > (Qualified Certificates), the suspension/withdrawing will be reported to
>> > the Supervisory Body, which will them update the Qualified Trust List
>> for
>> > that country and that will flow into the EU Qualified Trust List.
>>
>> > withdrawn, no notification will be made to relying parties. The closest
>> > that it comes is that if they're accredited according to EN 319 411-2
>> > (Qualified Certificates), the suspension/withdrawing will be reported to
>> > the Supervisory Body, which will them update the Qualified Trust List
>> for
>> > that country and that will flow into the EU Qualified Trust List.
>>
>> Quick correction here: this certification suspension/withdrawal does not
>> automatically imply a qualification suspension/withdrawal by the SB. The SB
>> is the sole responsible of the TL content, and can ignore the certification
>> suspension (or certification success, failure, absence, or whatever).
>>
>
> Got a citation?
>> automatically imply a qualification suspension/withdrawal by the SB. The SB
>> is the sole responsible of the TL content, and can ignore the certification
>> suspension (or certification success, failure, absence, or whatever).
>>
>
>
Other that the eIDAS regulation? No.
What you wrote would mean that the CAB is finally responsible of the
Qualified status of a TSP. And this is wrong.
--
Erwann.
Ryan Sleevi
Oct 30, 2018, 5:23:10 PM10/30/18
to eab...@gmail.com, mozilla-dev-security-policy
On Tue, Oct 30, 2018 at 4:37 PM Erwann Abalea via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> > On what basis do you believe this claim is to be made? By virtue of
> > asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
> > was absent, do you believe the same?
>
dev-secur...@lists.mozilla.org> wrote:
> > On what basis do you believe this claim is to be made? By virtue of
> > asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
> > was absent, do you believe the same?
>
> qcStatement-1 is not mandatory, by law, if the policyId is QCP or QCP+, or
> if there's a matching Qualification stating that the certificate is
> Qualified. Implementing decision 2015/1505 defines the common EU rules, and
> I haven't found the specific German rules (they're asserted in the German
> TL).
> 2015/1505 can be found at
> https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32015D1505
>
> My mistake was that I looked at the Sie and didn't check if there was a
> QualificationsExtension node.
>
Ah hah! Thanks for that context! That means I should re-examine the case of
> if there's a matching Qualification stating that the certificate is
> Qualified. Implementing decision 2015/1505 defines the common EU rules, and
> I haven't found the specific German rules (they're asserted in the German
> TL).
> 2015/1505 can be found at
> https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32015D1505
>
> My mistake was that I looked at the Sie and didn't check if there was a
> QualificationsExtension node.
>
Certinomis in the context of
https://groups.google.com/d/msg/mozilla.dev.security.policy/x3s3CSKdIlY/R1C4JAvOBQAJ
, since I was downplaying the significance based upon the lack of asserting
id-etsi-qcs-QcCompliance in the QCStatements, without consideration of the
Certificate Policies.
> I'm not sure the ambiguity can be as easily resolved as you suggest, given
> > the description within EN 319 412-5
>
> The weight taken by EN 319412-5 is important for the EN 319412-2
> certification, but not for the Qualified status and usage of the
> certificate (because that's a legal issue).
>
I'm not sure the issues are so easily disentangled, given the other
> certification, but not for the Qualified status and usage of the
> certificate (because that's a legal issue).
>
QCStatements supported. For example, the constitutive value of an
id-etsi-qcs-QcLimitValue. Or is the view that such issues are to be
addressed at the national level in accordance with TL maintenance?
> > Considering that even prior versions (e.g. 2.0.12) included an ASN.1
> module
> > as a normative inclusion (Annex B), I find this profoundly
> non-compelling.
> > See
> >
> https://www.etsi.org/deliver/etsi_en/319400_319499/31941205/02.01.01_60/en_31941205v020101p.pdf
> > .
>
> I was talking about draft versions. The QcType definition was SEQUENCE {
> qcType OBJECT IDENTIFIER } just before that.
>
Oh, for sure, that was in
> qcType OBJECT IDENTIFIER } just before that.
>
https://www.etsi.org/deliver/etsi_en/319400_319499/31941205/02.00.12_20/en_31941205v020012a.pdf
- but even still, OIDs never aligned
> > As you dig through these versions, the adopted versions do not share the
> > ambiguity issues. You're correct that 2.2.0 formalized the corrigenda
> > against 2.2.1 to include, textually, the normative requirement of "one
> and
> > exactly one" method, but in either event, such encodings violate it
> > entirely.
>
> I agree, having the id-etsi-qct-web OID used for the statementId is a
> clear violation. I'm just pointing that this specific QCStatement was
> really stupidly defined from the start.
>
Sure. It's also unlikely that will stop anytime soon though (c.f. PSD2 in
> clear violation. I'm just pointing that this specific QCStatement was
> really stupidly defined from the start.
>
TS 119 495, although that may be withdrawn now?
https://portal.etsi.org/webapp/workProgram/Report_Schedule.asp?WKI_ID=53961
)
Ryan Sleevi
Oct 30, 2018, 5:30:25 PM10/30/18
to eab...@gmail.com, mozilla-dev-security-policy
On Tue, Oct 30, 2018 at 5:08 PM Erwann Abalea <eab...@gmail.com> wrote:
> Not seeing this on Google Groups :/
>
> Le mar. 30 oct. 2018 à 18:28, Ryan Sleevi <ryan....@gmail.com> a
> écrit :
>
>>
>>
> Not seeing this on Google Groups :/
>
> Le mar. 30 oct. 2018 à 18:28, Ryan Sleevi <ryan....@gmail.com> a
> écrit :
>
>>
>>
>> On Tue, Oct 30, 2018 at 1:20 PM Erwann Abalea via dev-security-policy <
>> dev-secur...@lists.mozilla.org> wrote:
>>
>>> Le mardi 30 octobre 2018 17:29:14 UTC+1, Ryan Sleevi a écrit :
>>> [...]
>> dev-secur...@lists.mozilla.org> wrote:
>>
>>> Le mardi 30 octobre 2018 17:29:14 UTC+1, Ryan Sleevi a écrit :
>>> [...]
>>> > Note that if either the TSP is suspended of their certification or
>>> > withdrawn, no notification will be made to relying parties. The closest
>>> > that it comes is that if they're accredited according to EN 319 411-2
>>> > (Qualified Certificates), the suspension/withdrawing will be reported
>>> to
>>> > the Supervisory Body, which will them update the Qualified Trust List
>>> for
>>> > that country and that will flow into the EU Qualified Trust List.
>>>
>>> > withdrawn, no notification will be made to relying parties. The closest
>>> > that it comes is that if they're accredited according to EN 319 411-2
>>> > (Qualified Certificates), the suspension/withdrawing will be reported
>>> to
>>> > the Supervisory Body, which will them update the Qualified Trust List
>>> for
>>> > that country and that will flow into the EU Qualified Trust List.
>>>
>>> Quick correction here: this certification suspension/withdrawal does not
>>> automatically imply a qualification suspension/withdrawal by the SB. The SB
>>> is the sole responsible of the TL content, and can ignore the certification
>>> suspension (or certification success, failure, absence, or whatever).
>>>
>>
>> Got a citation?
>>
>
> Other that the eIDAS regulation? No.
> What you wrote would mean that the CAB is finally responsible of the
> Qualified status of a TSP. And this is wrong.
>
Perhaps it was poorly stated, but I think we're in agreement that the
>>> automatically imply a qualification suspension/withdrawal by the SB. The SB
>>> is the sole responsible of the TL content, and can ignore the certification
>>> suspension (or certification success, failure, absence, or whatever).
>>>
>>
>> Got a citation?
>>
>
> Other that the eIDAS regulation? No.
> What you wrote would mean that the CAB is finally responsible of the
> Qualified status of a TSP. And this is wrong.
>
Supervisory Body ultimately makes the decision regarding both the addition
to and removal from the qualified trust list within that country. That
said, in re-examining Article 20(3) and Article 17, I agree, it's clear
that the suspension of accreditation does not itself trigger an obligation
to suspend certified status.
Erwann Abalea
Oct 30, 2018, 6:45:04 PM10/30/18
to mozilla-dev-s...@lists.mozilla.org
Le mardi 30 octobre 2018 22:23:10 UTC+1, Ryan Sleevi a écrit :
> On Tue, Oct 30, 2018 at 4:37 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > > On what basis do you believe this claim is to be made? By virtue of
> > > asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
> > > was absent, do you believe the same?
> >
> > qcStatement-1 is not mandatory, by law, if the policyId is QCP or QCP+, or
> > if there's a matching Qualification stating that the certificate is
> > Qualified. Implementing decision 2015/1505 defines the common EU rules, and
> > I haven't found the specific German rules (they're asserted in the German
> > TL).
> > 2015/1505 can be found at
> > https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32015D1505
> >
> > My mistake was that I looked at the Sie and didn't check if there was a
> > QualificationsExtension node.
> >
>
> Ah hah! Thanks for that context! That means I should re-examine the case of
> Certinomis in the context of
> https://groups.google.com/d/msg/mozilla.dev.security.policy/x3s3CSKdIlY/R1C4JAvOBQAJ
> , since I was downplaying the significance based upon the lack of asserting
> id-etsi-qcs-QcCompliance in the QCStatements, without consideration of the
> Certificate Policies.
Ok. Then in this context, the Certinomis' certificate is just not claimed to be qualified. It's permitted.
> On Tue, Oct 30, 2018 at 4:37 PM Erwann Abalea via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
> > > On what basis do you believe this claim is to be made? By virtue of
> > > asserting qcStatement-1? If qcStatement was mis-encoded, or qcStatement-1
> > > was absent, do you believe the same?
> >
> > qcStatement-1 is not mandatory, by law, if the policyId is QCP or QCP+, or
> > if there's a matching Qualification stating that the certificate is
> > Qualified. Implementing decision 2015/1505 defines the common EU rules, and
> > I haven't found the specific German rules (they're asserted in the German
> > TL).
> > 2015/1505 can be found at
> > https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32015D1505
> >
> > My mistake was that I looked at the Sie and didn't check if there was a
> > QualificationsExtension node.
> >
>
> Ah hah! Thanks for that context! That means I should re-examine the case of
> Certinomis in the context of
> https://groups.google.com/d/msg/mozilla.dev.security.policy/x3s3CSKdIlY/R1C4JAvOBQAJ
> , since I was downplaying the significance based upon the lack of asserting
> id-etsi-qcs-QcCompliance in the QCStatements, without consideration of the
> Certificate Policies.
The problem regarding 0.4.0.1.6 remains, though.
> > I'm not sure the ambiguity can be as easily resolved as you suggest, given
> > > the description within EN 319 412-5
> >
> > The weight taken by EN 319412-5 is important for the EN 319412-2
> > certification, but not for the Qualified status and usage of the
> > certificate (because that's a legal issue).
>
> I'm not sure the issues are so easily disentangled, given the other
> QCStatements supported. For example, the constitutive value of an
> id-etsi-qcs-QcLimitValue. Or is the view that such issues are to be
> addressed at the national level in accordance with TL maintenance?
So, from an eIDAS point of view (Qualified status, signature/seal/...), compliance to EN 319412-5 does not matter.
For the LimitValue statement, the note is even clearer in that it was introduced to support the 1999 directive, and no equivalent text is present in the eIDAS regulation.
[...]
> > > As you dig through these versions, the adopted versions do not share the
> > > ambiguity issues. You're correct that 2.2.0 formalized the corrigenda
> > > against 2.2.1 to include, textually, the normative requirement of "one
> > and
> > > exactly one" method, but in either event, such encodings violate it
> > > entirely.
> >
> > I agree, having the id-etsi-qct-web OID used for the statementId is a
> > clear violation. I'm just pointing that this specific QCStatement was
> > really stupidly defined from the start.
> >
>
> Sure. It's also unlikely that will stop anytime soon though (c.f. PSD2 in
> TS 119 495, although that may be withdrawn now?
> https://portal.etsi.org/webapp/workProgram/Report_Schedule.asp?WKI_ID=53961
> )
It certainly won't be withdrawn (PSD2 is another European Directive which consumes eIDAS services, this TS provides technical support for that). It's also badly designed (IMHO). I'll let payment service providers come in and do their part of the job, or accept the resulting beast (they're late, but it seems some are discovering it and mourn, that's amusing).
> > > ambiguity issues. You're correct that 2.2.0 formalized the corrigenda
> > > against 2.2.1 to include, textually, the normative requirement of "one
> > and
> > > exactly one" method, but in either event, such encodings violate it
> > > entirely.
> >
> > I agree, having the id-etsi-qct-web OID used for the statementId is a
> > clear violation. I'm just pointing that this specific QCStatement was
> > really stupidly defined from the start.
> >
>
> Sure. It's also unlikely that will stop anytime soon though (c.f. PSD2 in
> TS 119 495, although that may be withdrawn now?
> https://portal.etsi.org/webapp/workProgram/Report_Schedule.asp?WKI_ID=53961
> )
The withdrawal you're pointing at is for the WI (work item). Once the deliverable is produced (and considered mature enough), the WI is shut down. A new one can later be re-open in order to work on a revision.
Dimitris Zacharopoulos
Oct 31, 2018, 7:11:07 AM10/31/18
to ry...@sleevi.com, mozilla-dev-s...@lists.mozilla.org
On 30/10/2018 6:28 μμ, Ryan Sleevi via dev-security-policy wrote:
> This establishes who the CAB is and who the NAB is. As the scheme used in
> eIDAS for CABs is ETSI EN 319 403, the CAB must perform their assessments
> in concordance with this scheme, and the NAB is tasked with assessing their
> qualification and certification under any local legislation (if
> appropriate) or, lacking such, under the framework for the NAB applying the
> principles of ISO/IEC 17065 in evaluating the CAB against EN 319 403. The
> NAB is the singular national entity recognized for issuing certifications
> against ISO/IEC 17065 through the MLA/BLA and the EU Regulation No 765/2008
> (as appropriate), which is then recognized trans-nationally.
Some clarifications/corrections because I saw some wrong usage of terms
> This establishes who the CAB is and who the NAB is. As the scheme used in
> eIDAS for CABs is ETSI EN 319 403, the CAB must perform their assessments
> in concordance with this scheme, and the NAB is tasked with assessing their
> qualification and certification under any local legislation (if
> appropriate) or, lacking such, under the framework for the NAB applying the
> principles of ISO/IEC 17065 in evaluating the CAB against EN 319 403. The
> NAB is the singular national entity recognized for issuing certifications
> against ISO/IEC 17065 through the MLA/BLA and the EU Regulation No 765/2008
> (as appropriate), which is then recognized trans-nationally.
being repeated.
A CAB MUST perform their assessments applying ISO/IEC 17065 AND ETSI EN
319 403 AND any applicable legislation (for EU CABs this includes
European and National legislation).
Also, a NAB issues "Accreditations" to CABs and not "Certifications".
Also, a CAB issues "Certifications" to TSPs and not "Accredidations".
So, T-Systems is "Certified", not "Accredited".
>
> As the framework utilizes ISO/IEC 17065, the complaints process and
> certification process for both TSPs and CABs bears strong similarity, which
> is why I wanted to explore how this process works in function.
>
> Note that if either the TSP is suspended of their certification or
> withdrawn, no notification will be made to relying parties.
This depends on applicable legislation and the implementation of ISO
> withdrawn, no notification will be made to relying parties.
17065 sections 4.6, 7.11.3 by each CAB. Some CABs have a public
repository where RPs can query the validity of TSP Certifications so if
a Certification is Suspended or Revoked, it will be displayed
accordingly. I don't think WT has a notification scheme for RPs either.
If the TSP publishes the seal URL or the CAB's URL to the TSP
Certificate (which is not mandatory), RPs can manually check the
validity of the TSP Certification.
> The closest
> that it comes is that if they're accredited according to EN 319 411-2
> (Qualified Certificates), the suspension/withdrawing will be reported to
> the Supervisory Body, which will them update the Qualified Trust List for
> that country and that will flow into the EU Qualified Trust List. If
> they're accredited against EN 319 411-1, the Supervisory Body will be
> informed by the CAB (in theory, although note my complaint about TSP
> informing the CAB was not followed, and the same can exist with CAB to SB),
> but no further notification may be made. Furthermore, if certification is
> later reissued, after a full audit, the certification history will not
> reflect that there was a period of 'failed' certification. This similarly
> exists with respect to CABs - if a CAB has their accreditation suspended,
> on the advice of or decision of the NAB based on feedback from the SB - the
> community will not necessarily be informed. In theory, because
> certification is 'forward' looking rather than 'past' looking, a suspension
> or withdraw of a CAB by a NAB may not affect its past certification of
> TSPs; this is an area of process that has not been well-specified or
> determined.
Note that Supervisory Bodies (only related to eIDAS) have no authority
> they're accredited against EN 319 411-1, the Supervisory Body will be
> informed by the CAB (in theory, although note my complaint about TSP
> informing the CAB was not followed, and the same can exist with CAB to SB),
> but no further notification may be made. Furthermore, if certification is
> later reissued, after a full audit, the certification history will not
> reflect that there was a period of 'failed' certification. This similarly
> exists with respect to CABs - if a CAB has their accreditation suspended,
> on the advice of or decision of the NAB based on feedback from the SB - the
> community will not necessarily be informed. In theory, because
> certification is 'forward' looking rather than 'past' looking, a suspension
> or withdraw of a CAB by a NAB may not affect its past certification of
> TSPs; this is an area of process that has not been well-specified or
> determined.
for TSP Certifications under ETSI EN 319 411-1, but only ETSI EN 319
411-2. In all cases of Certification (ETSI EN 319 411-1 or ETSI EN 319
411-2), the NAB is assessing the CAB. In most EU countries, the NAB IS
NOT the Supervisory Body.
Similarly with TSPs losing their Certification, if a CAB loses their
Accreditation it will be displayed on the NAB's web site.
I also consider the "WT seal" and "ETSI certification" very similar. A
WT seal is similar to an ETSI certificate because they state (emphasis
mine):
"An unqualified opinion from the practitioner indicates that such
principles *are being followed* in conformity with the WebTrust for
Certification Authorities Criteria. These principles and criteria
reflect fundamental standards for *the establishment and on-going
operation* of a Certification Authority organization or function."
So, if I check a WT seal today Oct 31, 2018, even though the CA has not
been audited between their last audit and today, the WT seal represents
that it is still valid and not withdrawn. They are both "forward
looking" in the eyes of Relying Parties.
As far as the non-disclosure of compliance certificate
suspension/withdrawals is concerned, CABs are only allowed to follow
their practices as described in ISO 17065 section 7.11.3. Root Programs
could possibly require that CAs MUST disclose any possible Certification
suspension or revocation that occurred during their audit period.
Hope this helps.
Dimitris.
Ryan Sleevi
Oct 31, 2018, 10:48:12 AM10/31/18
to Dimitris Zacharopoulos, Ryan Sleevi, mozilla-dev-security-policy
There's a lot of nitpicking in this, and I feel that if you want to
continue this discussion, it would be better off in a separate thread on
terminology. I disagree with some of the claims you've made, so have
corrected them for the discussion.
I would much rather keep this focused on the discussion of TUVIT as
auditors; if you feel that the nitpicking is relevant to that discussion
(which I don't believe anything you've said rises to that level), we should
certainly hash it out here. This is why I haven't forked this thread yet -
to make sure I've not misread your concern. However, if there's more
broadly a disagreement, but without impact to this discussion, we should
spin that out.
On Wed, Oct 31, 2018 at 7:11 AM Dimitris Zacharopoulos <ji...@it.auth.gr>
wrote:
> On 30/10/2018 6:28 μμ, Ryan Sleevi via dev-security-policy wrote:
> > This establishes who the CAB is and who the NAB is. As the scheme used in
> > eIDAS for CABs is ETSI EN 319 403, the CAB must perform their assessments
> > in concordance with this scheme, and the NAB is tasked with assessing
> their
> > qualification and certification under any local legislation (if
> > appropriate) or, lacking such, under the framework for the NAB applying
> the
> > principles of ISO/IEC 17065 in evaluating the CAB against EN 319 403. The
> > NAB is the singular national entity recognized for issuing certifications
> > against ISO/IEC 17065 through the MLA/BLA and the EU Regulation No
> 765/2008
> > (as appropriate), which is then recognized trans-nationally.
>
> Some clarifications/corrections because I saw some wrong usage of terms
> being repeated.
>
> A CAB MUST perform their assessments applying ISO/IEC 17065 AND ETSI EN
> 319 403 AND any applicable legislation (for EU CABs this includes
> European and National legislation).
>
Dimitris, I'm sorry, but I don't believe this is a correct correction.
EN 319 403 incorporates ISO/IEC 17065; much like the discussion about EN
319 411-2 incorporating, but being separate from, EN 319 411-1, the
structure of EN 319 403 is that it incorporates normatively the structure
of ISO/IEC 17065, and, at some places, extends.
Your description of the system is logically incompatible, given the
incompatibilities in 319 403 and 17065.
You're correct that any applicable national legislation applies, with
respect to the context of eIDAS. However, one can be assessed solely
against the scheme of EN 319 403 and 319 411-1, without going for qualified.
> Also, a NAB issues "Accreditations" to CABs and not "Certifications".
> Also, a CAB issues "Certifications" to TSPs and not "Accredidations".
> So, T-Systems is "Certified", not "Accredited".
>
Fair. If you replace these words, does it change the semantic meaning of
the message at all? I don't believe so.
> > As the framework utilizes ISO/IEC 17065, the complaints process and
> > certification process for both TSPs and CABs bears strong similarity,
> which
> > is why I wanted to explore how this process works in function.
> >
> > Note that if either the TSP is suspended of their certification or
> > withdrawn, no notification will be made to relying parties.
>
> This depends on applicable legislation and the implementation of ISO
> 17065 sections 4.6, 7.11.3 by each CAB. Some CABs have a public
> repository where RPs can query the validity of TSP Certifications so if
> a Certification is Suspended or Revoked, it will be displayed
> accordingly. I don't think WT has a notification scheme for RPs either.
> If the TSP publishes the seal URL or the CAB's URL to the TSP
> Certificate (which is not mandatory), RPs can manually check the
> validity of the TSP Certification.
>
I don't think this is a valid criticism, particularly in the context of the
specific case we're speaking about. I'm speaking about what's required -
you're speaking about what's possible. Many things are possible, but what
matters for expectations is what is required. 7.11.3 simply defers to the
scheme to specify, which EN 319 403 does not as it relates to this
discussion.
> Note that Supervisory Bodies (only related to eIDAS) have no authority
> for TSP Certifications under ETSI EN 319 411-1, but only ETSI EN 319
> 411-2. In all cases of Certification (ETSI EN 319 411-1 or ETSI EN 319
> 411-2), the NAB is assessing the CAB. In most EU countries, the NAB IS
> NOT the Supervisory Body.
>
The NAB is still responsible for the oversight of the CAB's execution of EN
319 403, and the investigation therein. The SB suspends qualified status,
but the NAB ensures that the CAB is meeting the requirements of the
certification scheme (EN 319 403) as part of supervising the accreditation
of that CAB.
> Similarly with TSPs losing their Certification, if a CAB loses their
> Accreditation it will be displayed on the NAB's web site.
>
More concretely, the absence will be.
> I also consider the "WT seal" and "ETSI certification" very similar. A
> WT seal is similar to an ETSI certificate because they state (emphasis
> mine):
>
No, they are not at all similar. I don't think this is the thread to have
that discussion, however.
> "An unqualified opinion from the practitioner indicates that such
> principles *are being followed* in conformity with the WebTrust for
> Certification Authorities Criteria. These principles and criteria
> reflect fundamental standards for *the establishment and on-going
> operation* of a Certification Authority organization or function."
>
> So, if I check a WT seal today Oct 31, 2018, even though the CA has not
> been audited between their last audit and today, the WT seal represents
> that it is still valid and not withdrawn. They are both "forward
> looking" in the eyes of Relying Parties.
>
This fundamentally misunderstands what WT audits are.
> As far as the non-disclosure of compliance certificate
> suspension/withdrawals is concerned, CABs are only allowed to follow
> their practices as described in ISO 17065 section 7.11.3. Root Programs
> could possibly require that CAs MUST disclose any possible Certification
> suspension or revocation that occurred during their audit period.
>
Yes.
continue this discussion, it would be better off in a separate thread on
terminology. I disagree with some of the claims you've made, so have
corrected them for the discussion.
I would much rather keep this focused on the discussion of TUVIT as
auditors; if you feel that the nitpicking is relevant to that discussion
(which I don't believe anything you've said rises to that level), we should
certainly hash it out here. This is why I haven't forked this thread yet -
to make sure I've not misread your concern. However, if there's more
broadly a disagreement, but without impact to this discussion, we should
spin that out.
On Wed, Oct 31, 2018 at 7:11 AM Dimitris Zacharopoulos <ji...@it.auth.gr>
wrote:
> On 30/10/2018 6:28 μμ, Ryan Sleevi via dev-security-policy wrote:
> > This establishes who the CAB is and who the NAB is. As the scheme used in
> > eIDAS for CABs is ETSI EN 319 403, the CAB must perform their assessments
> > in concordance with this scheme, and the NAB is tasked with assessing
> their
> > qualification and certification under any local legislation (if
> > appropriate) or, lacking such, under the framework for the NAB applying
> the
> > principles of ISO/IEC 17065 in evaluating the CAB against EN 319 403. The
> > NAB is the singular national entity recognized for issuing certifications
> > against ISO/IEC 17065 through the MLA/BLA and the EU Regulation No
> 765/2008
> > (as appropriate), which is then recognized trans-nationally.
>
> Some clarifications/corrections because I saw some wrong usage of terms
> being repeated.
>
> A CAB MUST perform their assessments applying ISO/IEC 17065 AND ETSI EN
> 319 403 AND any applicable legislation (for EU CABs this includes
> European and National legislation).
>
EN 319 403 incorporates ISO/IEC 17065; much like the discussion about EN
319 411-2 incorporating, but being separate from, EN 319 411-1, the
structure of EN 319 403 is that it incorporates normatively the structure
of ISO/IEC 17065, and, at some places, extends.
Your description of the system is logically incompatible, given the
incompatibilities in 319 403 and 17065.
You're correct that any applicable national legislation applies, with
respect to the context of eIDAS. However, one can be assessed solely
against the scheme of EN 319 403 and 319 411-1, without going for qualified.
> Also, a NAB issues "Accreditations" to CABs and not "Certifications".
> Also, a CAB issues "Certifications" to TSPs and not "Accredidations".
> So, T-Systems is "Certified", not "Accredited".
>
the message at all? I don't believe so.
> > As the framework utilizes ISO/IEC 17065, the complaints process and
> > certification process for both TSPs and CABs bears strong similarity,
> which
> > is why I wanted to explore how this process works in function.
> >
> > Note that if either the TSP is suspended of their certification or
> > withdrawn, no notification will be made to relying parties.
>
> This depends on applicable legislation and the implementation of ISO
> 17065 sections 4.6, 7.11.3 by each CAB. Some CABs have a public
> repository where RPs can query the validity of TSP Certifications so if
> a Certification is Suspended or Revoked, it will be displayed
> accordingly. I don't think WT has a notification scheme for RPs either.
> If the TSP publishes the seal URL or the CAB's URL to the TSP
> Certificate (which is not mandatory), RPs can manually check the
> validity of the TSP Certification.
>
specific case we're speaking about. I'm speaking about what's required -
you're speaking about what's possible. Many things are possible, but what
matters for expectations is what is required. 7.11.3 simply defers to the
scheme to specify, which EN 319 403 does not as it relates to this
discussion.
> Note that Supervisory Bodies (only related to eIDAS) have no authority
> for TSP Certifications under ETSI EN 319 411-1, but only ETSI EN 319
> 411-2. In all cases of Certification (ETSI EN 319 411-1 or ETSI EN 319
> 411-2), the NAB is assessing the CAB. In most EU countries, the NAB IS
> NOT the Supervisory Body.
>
319 403, and the investigation therein. The SB suspends qualified status,
but the NAB ensures that the CAB is meeting the requirements of the
certification scheme (EN 319 403) as part of supervising the accreditation
of that CAB.
> Similarly with TSPs losing their Certification, if a CAB loses their
> Accreditation it will be displayed on the NAB's web site.
>
> I also consider the "WT seal" and "ETSI certification" very similar. A
> WT seal is similar to an ETSI certificate because they state (emphasis
> mine):
>
that discussion, however.
> "An unqualified opinion from the practitioner indicates that such
> principles *are being followed* in conformity with the WebTrust for
> Certification Authorities Criteria. These principles and criteria
> reflect fundamental standards for *the establishment and on-going
> operation* of a Certification Authority organization or function."
>
> So, if I check a WT seal today Oct 31, 2018, even though the CA has not
> been audited between their last audit and today, the WT seal represents
> that it is still valid and not withdrawn. They are both "forward
> looking" in the eyes of Relying Parties.
>
> As far as the non-disclosure of compliance certificate
> suspension/withdrawals is concerned, CABs are only allowed to follow
> their practices as described in ISO 17065 section 7.11.3. Root Programs
> could possibly require that CAs MUST disclose any possible Certification
> suspension or revocation that occurred during their audit period.
>
Wiedenhorst, Matthias
Oct 31, 2018, 11:43:14 AM10/31/18
to mozilla-dev-s...@lists.mozilla.org
Dear all,
I am posting the following statement on behalf of TÜViT.
We strongly refuse the suggestion of (the individual) Ryan due to basing it, at least partly, on incorrectness on the audit practice of TUVIT and misunderstandings of technical / legal norms like ISO/IEC 17065 a and eIDAS and therefore coming to exaggerated conclusions. Before coming to the details, let me summarize relevant facts out of Ryan’s contribution:
· Since January 2018, T-Systems issued EV certificates with an incorrect qcStatement. T-Systems was made aware of the problem in October 2018, i.e. for about 9 month the error was not detected/reported
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
T-Systems fixed the error in a timely manner so that certificates now contain the correct qcStatement.
· TUVIT performed an audit of T-Systems according to ETSI policies EVCP and QCP-w in the beginning of 2018. During the audit the incorrect coding of the qcStatement was not detected.
· On October 15, Ryan lodged a formal complaint to the certification body with respect to the certification of T-Systems International, assessment letter AA2018072001, available at https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf
· In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
For non-critical non-conformities, our certification requirements foresee a maximum period of 3 month for remediation before the certification shall be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the classification as minor, we do not see a necessity for revocation.
That’s about the relevant facts.
Let me now reply in detail to Ryans private contribution:
>I would like to suggest that consideration be given to rejecting future
>audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
>Widermann, for some period of time. I would suggest this period be at least
>one year long; however, given the technical details of ETSI accreditation,
>believe a period of three years may be more appropriate.
Dr. Anja Wiedemann (please mind the correct spelling) was not part of the audit team. We do not understand why her name is mentioned here.
One / three years exclusion from audit sounds like a punishment. We do not understand where this time frame comes from and why such a time frame is needed.
>As part of an investigation into the incorrect qcStatements reported at
>https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
>, I've used this as an example to explore the complaint handling process
>under ETSI EN 319 403.
>
>Each CAB and NAB must make available their information for processing
>complaints. Given that 100% of the qualified certificates issued by
>T-Systems have failed to comply with the requirements of ETSI EN 319 411-2,
>by virtue of failing to comply with ETSI EN 319 412-5, I lodged a complaint
>regarding T-System's certification with TUVIT, which certified T-Systems.
>As part of processing the complaint, the following issues regarding TUVIT's
>handling of complaints and overall approach to audits were revealed.
>Issue A) As part of their initial response to my complaint, TUVIT, by way
>of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
>very first, quick cross check with our audit evidence record, I can only
>say that we did check issued certificates from within the declared period
>and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
>3". However, this statement was in direct conflict with the TSP's own
>investigation and incident report, provided at
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
>mistake was introduced during the development of support - that is, no such
>properly issued certificates were issued.
We do not understand why the important fact that Matthias was not in office and replied what he remembered from an audit that was month ago is not mentioned here. In addition, he replied that he would verify and come back as soon as possible (when he is in office again). That actually happened, see below.
>In follow-up with TUVIT, I requested they provide any evidence to support
>the claim that such certificates were issued, knowing the matter reported
>by T-Systems means this is highly improbable.
>In subsequent reply, Matthias Wiedenhorst acknowledged that their
>methodology for testing compliance with ETSI EN 319 412-5 stated that the
>process was "the auditors have to check the website certificates manually
>for the presence of the QC-statement. In this case, the existence of the
>id-etsi-qct-web was identified, but for any reason it was not realized that
>the esi4-qcStatements-6 was missing. We informed the auditors about how to
>check the QC-statement correctly so that we are confident, that in future
>an incorrect coding will be detected. "
>
>This implies a significant lack of core competencies within TUVIT to assess
>against the criteria of ETSI EN 319 412-5, which provides a normative ASN.1
>module within its specification.
We do not agree at all that core competencies are missing. As you described, back in his office Matthias accessed the audit logs, did the manual check and found the wrong encoding. IT is obvious, that he has the competency to read ASN.1. In addition, it is not correct, that TUVIT does not use any tools for certificate checking but the currently available lint tools do not support the checking of the qcStatement.
Best regards
Matthias Wiedenhorst
______________________________________________________________________________________________________________________
Sitz der Gesellschaft/Headquarter: TÜV Informationstechnik GmbH * Langemarckstr. 20 * 45141 Essen, Germany
Registergericht/Register Court: Amtsgericht/Local Court Essen * HRB 11687 * USt.-IdNr./VAT No.: DE 176132277 * Steuer-Nr./Tax No.: 111/57062251
Geschäftsführung/Management Board: Dirk Kretzschmar
TÜV NORD GROUP
Expertise for your Success
Please visit our website: www.tuv-nord.com<http://www.tuv-nord.com>
Besuchen Sie unseren Internetauftritt: www.tuev-nord.de<http://www.tuev-nord.de>
I am posting the following statement on behalf of TÜViT.
We strongly refuse the suggestion of (the individual) Ryan due to basing it, at least partly, on incorrectness on the audit practice of TUVIT and misunderstandings of technical / legal norms like ISO/IEC 17065 a and eIDAS and therefore coming to exaggerated conclusions. Before coming to the details, let me summarize relevant facts out of Ryan’s contribution:
· Since January 2018, T-Systems issued EV certificates with an incorrect qcStatement. T-Systems was made aware of the problem in October 2018, i.e. for about 9 month the error was not detected/reported
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
T-Systems fixed the error in a timely manner so that certificates now contain the correct qcStatement.
· TUVIT performed an audit of T-Systems according to ETSI policies EVCP and QCP-w in the beginning of 2018. During the audit the incorrect coding of the qcStatement was not detected.
· On October 15, Ryan lodged a formal complaint to the certification body with respect to the certification of T-Systems International, assessment letter AA2018072001, available at https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf
· In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
For non-critical non-conformities, our certification requirements foresee a maximum period of 3 month for remediation before the certification shall be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the classification as minor, we do not see a necessity for revocation.
That’s about the relevant facts.
Let me now reply in detail to Ryans private contribution:
>I would like to suggest that consideration be given to rejecting future
>audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
>Widermann, for some period of time. I would suggest this period be at least
>one year long; however, given the technical details of ETSI accreditation,
>believe a period of three years may be more appropriate.
One / three years exclusion from audit sounds like a punishment. We do not understand where this time frame comes from and why such a time frame is needed.
>As part of an investigation into the incorrect qcStatements reported at
>https://groups.google.com/forum/#!topic/mozilla.dev.security.policy/x3s3CSKdIlY
>, I've used this as an example to explore the complaint handling process
>under ETSI EN 319 403.
>
>For those not familiar with the process, ETSI EN 319 403 normatively
>incorporates various portions of ISO/IEC 17065, which is an international
>standard for Conformity Assessment Bodies. Under the eIDAS scheme, auditors
> (henceforth called Conformity Assessment Bodies, or CAB) are assessed by
>their National Accreditation Body, or NAB, as being accredited under EN 319
>403 to conduct audits against the schemes in ETSI EN 319 411-1 and ETSI EN
>319 411-2. When a CA (called a Trust Service Provider, or TSP, by eIDAS)
>has been audited (accredited) by a CAB against the scheme in EN 319 411-1
or 411-2, the CAB is applying the principles in EN 319 403.
ETSI EN 319 411-1 and ETSI EN 319 411-2 are no schemes but standards describing policy requirements for TSPs.
>incorporates various portions of ISO/IEC 17065, which is an international
>standard for Conformity Assessment Bodies. Under the eIDAS scheme, auditors
> (henceforth called Conformity Assessment Bodies, or CAB) are assessed by
>their National Accreditation Body, or NAB, as being accredited under EN 319
>403 to conduct audits against the schemes in ETSI EN 319 411-1 and ETSI EN
>319 411-2. When a CA (called a Trust Service Provider, or TSP, by eIDAS)
>has been audited (accredited) by a CAB against the scheme in EN 319 411-1
or 411-2, the CAB is applying the principles in EN 319 403.
>
>If there is a belief that a TSP has failed to meet the requirements of
>their accreditation, EN 319 403 describes a process for which complaints
>may be made to either the TSP or to the CAB. This complaint process is
>further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
>same process also applies when there have been mistakes by the CAB to
>adhere to its scheme requirements under EN 319 403 - a complaint may be
>made with either the CAB or the NAB regarding the CAB's accreditation.
TSPs are not accredited but certified, ETSI EN 319 403 §7.13 does not make any additional requirements on complaint procedures but just reference ISO/IEC 17065. (The requirements from ISO/IEC 17065 [1], clause 7.13 shall apply.) In particular, no procedures for complaints to TSPs or NABs are defined (only to CABs).
>If there is a belief that a TSP has failed to meet the requirements of
>their accreditation, EN 319 403 describes a process for which complaints
>may be made to either the TSP or to the CAB. This complaint process is
>further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
>same process also applies when there have been mistakes by the CAB to
>adhere to its scheme requirements under EN 319 403 - a complaint may be
>made with either the CAB or the NAB regarding the CAB's accreditation.
>Each CAB and NAB must make available their information for processing
>complaints. Given that 100% of the qualified certificates issued by
>T-Systems have failed to comply with the requirements of ETSI EN 319 411-2,
>by virtue of failing to comply with ETSI EN 319 412-5, I lodged a complaint
>regarding T-System's certification with TUVIT, which certified T-Systems.
>As part of processing the complaint, the following issues regarding TUVIT's
>handling of complaints and overall approach to audits were revealed.
>Issue A) As part of their initial response to my complaint, TUVIT, by way
>of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
>very first, quick cross check with our audit evidence record, I can only
>say that we did check issued certificates from within the declared period
>and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
>3". However, this statement was in direct conflict with the TSP's own
>investigation and incident report, provided at
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
>mistake was introduced during the development of support - that is, no such
>properly issued certificates were issued.
>In follow-up with TUVIT, I requested they provide any evidence to support
>the claim that such certificates were issued, knowing the matter reported
>by T-Systems means this is highly improbable.
>In subsequent reply, Matthias Wiedenhorst acknowledged that their
>methodology for testing compliance with ETSI EN 319 412-5 stated that the
>process was "the auditors have to check the website certificates manually
>for the presence of the QC-statement. In this case, the existence of the
>id-etsi-qct-web was identified, but for any reason it was not realized that
>the esi4-qcStatements-6 was missing. We informed the auditors about how to
>check the QC-statement correctly so that we are confident, that in future
>an incorrect coding will be detected. "
>
>This implies a significant lack of core competencies within TUVIT to assess
>against the criteria of ETSI EN 319 412-5, which provides a normative ASN.1
>module within its specification.
>
>Issue C) In regards to remedies, I requested a suspension of T-Systems
>current ETSI EN 319 411-2 accreditation, and a subsequent assessment of a
>Full audit, given concerns with respect to the audit team performing the
>initial audit. TUVIT declined this, on the basis that they assess such
>technical non-compliance to be a 'minor non-conformity' that, per ETSI EN
>319 403, may be resolved in the next three months by T-Systems without
>adversely affecting their accreditation.
>
>Further, T-Systems and TUVIT decided that, on the basis that it is a
>non-conformity, such certificates do not need to be revoked according to
>the Baseline Requirements' timeline. Furthermore, the failure to revoke
>according to the Baseline Requirements' timeline was a further minor
>non-conformity, not adversely affecting the certification decision of
>T-Systems, and may be remedied over the next 3 months.
>
>Furthermore, in response to whether or not such matters of non-conformities
>would be reported by TUVIT (the CAB), Matthis Wiedenhorst acknowledged that
>they are not required to make such information public at this time.
Actually, we offered to support the establishment of rules with in the CAB/Forum for making such kind of incidents public in an adequate form. Currently, we do not see such a requirement and per default (see e. g. ISO/IEC 17065 §4.5) we are obliged to keep things confidential.
>Issue C) In regards to remedies, I requested a suspension of T-Systems
>current ETSI EN 319 411-2 accreditation, and a subsequent assessment of a
>Full audit, given concerns with respect to the audit team performing the
>initial audit. TUVIT declined this, on the basis that they assess such
>technical non-compliance to be a 'minor non-conformity' that, per ETSI EN
>319 403, may be resolved in the next three months by T-Systems without
>adversely affecting their accreditation.
>
>Further, T-Systems and TUVIT decided that, on the basis that it is a
>non-conformity, such certificates do not need to be revoked according to
>the Baseline Requirements' timeline. Furthermore, the failure to revoke
>according to the Baseline Requirements' timeline was a further minor
>non-conformity, not adversely affecting the certification decision of
>T-Systems, and may be remedied over the next 3 months.
>
>Furthermore, in response to whether or not such matters of non-conformities
>would be reported by TUVIT (the CAB), Matthis Wiedenhorst acknowledged that
>they are not required to make such information public at this time.
>
>I believe this significantly undermines the level of confidence and
>assurance that relying parties should place in audits conducted by TUVIT.
>The inducement to non-compliant actions, combined with the non-disclosure
>of these "non-conformities" to affected parties, is at odds with the
>principles and values of the respective Root Programs' goals.
>
This personal view (a CAB is making confidential information public) is in striking contradiction to the confidentiality requirement of the international norm ISO/IEC 17065.
>I believe this significantly undermines the level of confidence and
>assurance that relying parties should place in audits conducted by TUVIT.
>The inducement to non-compliant actions, combined with the non-disclosure
>of these "non-conformities" to affected parties, is at odds with the
>principles and values of the respective Root Programs' goals.
>
>
>In addition to these concerns with respect to the audits provided, I will
>be further lodging a complaint with the German National Accreditation
>Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
>under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
>handling of this incident, the material misstatement as to what was
>examined, combined with the failure to ensure appropriate and prompt
>notification to the CAB and the Supervisory Body (as detailed in
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
>question the compliance with Regulation (EU) N°910/2014 on behalf of both
>T-Systems and TUVIT. In particular, the failure to perform a timely
>notification under Article 19(2) of the "loss of integrity that has a
>significant impact on the trust service provider" represents a significant
>breach of trust.
This statement is partly wrong as to our knowledge the supervisory body was informed.
>In addition to these concerns with respect to the audits provided, I will
>be further lodging a complaint with the German National Accreditation
>Body, Deutsche Akkreditierungsstelle GmbH, regarding TUVIT's accreditation
>under EN 319 403 to asses TSPs against EN 319 411-1 and EN 319 411-2. The
>handling of this incident, the material misstatement as to what was
>examined, combined with the failure to ensure appropriate and prompt
>notification to the CAB and the Supervisory Body (as detailed in
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 ) call into serious
>question the compliance with Regulation (EU) N°910/2014 on behalf of both
>T-Systems and TUVIT. In particular, the failure to perform a timely
>notification under Article 19(2) of the "loss of integrity that has a
>significant impact on the trust service provider" represents a significant
>breach of trust.
>
>With respect to what constitutes a "loss of integrity", both Article 8 and
>Article 7(f) note compliance to the appropriate technical standards and the
>prohibition of any "specific disproportionate technical requirements on
>relying parties ..." that would "significantly impede the interoperability
>of the notified electronic identification schemes". >
Articles 7 and 8 of eIDAS are not about TSPs. Sorry, but wrong eIDAS section.
>With respect to what constitutes a "loss of integrity", both Article 8 and
>Article 7(f) note compliance to the appropriate technical standards and the
>prohibition of any "specific disproportionate technical requirements on
>relying parties ..." that would "significantly impede the interoperability
>of the notified electronic identification schemes". >
>
>Through their failure to assure a timely disclosure, and the failure to
>revoke, TUVIT's certifications disproportionately and adversely affect
>Relying Parties. Such certificates assert that they are qualified
>certificates, and thus Relying Parties should be able to rely upon them for
>their constitutive value. However, in the absence of the appropriate and
>mandatory notice as to the qualified purpose of such certificates, as this
>incident demonstrates, Relying Parties are left without an interoperable
>means of determining whether or not such Qualified Certificates are indeed
>qualified, and thus subject to the protections afforded by eIDAS.
We do not agree with this whole statement. It seems that technical and legal things got mixed up. A qualified certificate is a purely legal construct. (see eIDAS definition)
>Through their failure to assure a timely disclosure, and the failure to
>revoke, TUVIT's certifications disproportionately and adversely affect
>Relying Parties. Such certificates assert that they are qualified
>certificates, and thus Relying Parties should be able to rely upon them for
>their constitutive value. However, in the absence of the appropriate and
>mandatory notice as to the qualified purpose of such certificates, as this
>incident demonstrates, Relying Parties are left without an interoperable
>means of determining whether or not such Qualified Certificates are indeed
>qualified, and thus subject to the protections afforded by eIDAS.
>
>Furthermore, the complaint process established under EN 319 403 refers to
>that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
>put forward within 7.13.5 and 7.13.6 that the complaint resolution process
>be independent from those from those involved in the audit or those who
>have consulted with or been employed by the subject of the audit. Matthias
>Wiedenhorst's involvement in, and resolution of, this complaint, calls into
>question whether TUVIT has acted according to this, given that they are
>represented as Lead Auditor for T-Systems' audit. At this point, the only
>objective means of resolving whether or not the process was followed is to
>lodge a complaint such that the NAB may themselves investigate the handling
>of this complaint.
Wrong again. Please have another look at 7.13.5, it talks about the <<decision>> to resolve the dispute and not of the process of resolving itself. The decision shall be made by an independent person. It is more than natural that for resolution process itself, concerned auditors needs to be involved.
>Furthermore, the complaint process established under EN 319 403 refers to
>that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
>put forward within 7.13.5 and 7.13.6 that the complaint resolution process
>be independent from those from those involved in the audit or those who
>have consulted with or been employed by the subject of the audit. Matthias
>Wiedenhorst's involvement in, and resolution of, this complaint, calls into
>question whether TUVIT has acted according to this, given that they are
>represented as Lead Auditor for T-Systems' audit. At this point, the only
>objective means of resolving whether or not the process was followed is to
>lodge a complaint such that the NAB may themselves investigate the handling
>of this complaint.
>
>Given that the Supervisory Body and National Accreditation bodies exist to
>protect the legal value of this scheme, the failure by TUVIT to uphold the
>safety and security of the eIDAS regime represents an ongoing threat to the
>ecosystem.
A personal opinion, with no founded base, that we do not share.
>Given that the Supervisory Body and National Accreditation bodies exist to
>protect the legal value of this scheme, the failure by TUVIT to uphold the
>safety and security of the eIDAS regime represents an ongoing threat to the
>ecosystem.
Best regards
Matthias Wiedenhorst
______________________________________________________________________________________________________________________
Sitz der Gesellschaft/Headquarter: TÜV Informationstechnik GmbH * Langemarckstr. 20 * 45141 Essen, Germany
Registergericht/Register Court: Amtsgericht/Local Court Essen * HRB 11687 * USt.-IdNr./VAT No.: DE 176132277 * Steuer-Nr./Tax No.: 111/57062251
Geschäftsführung/Management Board: Dirk Kretzschmar
TÜV NORD GROUP
Expertise for your Success
Please visit our website: www.tuv-nord.com<http://www.tuv-nord.com>
Besuchen Sie unseren Internetauftritt: www.tuev-nord.de<http://www.tuev-nord.de>
Kurt Roeckx
Oct 31, 2018, 12:13:33 PM10/31/18
to mozilla-dev-s...@lists.mozilla.org
On 2018-10-31 16:42, Wiedenhorst, Matthias wrote:
> In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
I think we never consider encoding errors as a minor error.
> In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
Kurt
Ryan Sleevi
Oct 31, 2018, 12:35:24 PM10/31/18
to M.Wied...@tuvit.de, mozilla-dev-security-policy
On Wed, Oct 31, 2018 at 11:43 AM Wiedenhorst, Matthias via
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> · Since January 2018, T-Systems issued EV certificates with an
> incorrect qcStatement. T-Systems was made aware of the problem in October
> 2018, i.e. for about 9 month the error was not detected/reported
> https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
> T-Systems fixed the error in a timely manner so that certificates now
> contain the correct qcStatement.
>
T-Systems identified a deficiency within their systems, made a change on
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> · Since January 2018, T-Systems issued EV certificates with an
> incorrect qcStatement. T-Systems was made aware of the problem in October
> 2018, i.e. for about 9 month the error was not detected/reported
> https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
> T-Systems fixed the error in a timely manner so that certificates now
> contain the correct qcStatement.
>
October 5, but did not notify their CAB and SB until October 16.
Under the requirements provided by EN 319 401, 7.9, that does not seem
consistent with meeting those requirements.
> · TUVIT performed an audit of T-Systems according to ETSI policies
> EVCP and QCP-w in the beginning of 2018. During the audit the incorrect
> coding of the qcStatement was not detected.
>
> · In several emails, we answered to his complaint, explained our
> procedures and justified the classification of the encoding error as minor
> (non-critical) non-conformity.
> (non-critical) non-conformity.
> For non-critical non-conformities, our certification requirements foresee
> a maximum period of 3 month for remediation before the certification shall
> be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the
> classification as minor, we do not see a necessity for revocation.
> That’s about the relevant facts.
> Let me now reply in detail to Ryans private contribution:
>
> >I would like to suggest that consideration be given to rejecting future
> >audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
> >Widermann, for some period of time. I would suggest this period be at
> least
> >one year long; however, given the technical details of ETSI accreditation,
> >believe a period of three years may be more appropriate.
>
> Dr. Anja Wiedemann (please mind the correct spelling) was not part of the
> audit team. We do not understand why her name is mentioned here.
> One / three years exclusion from audit sounds like a punishment. We do not
> understand where this time frame comes from and why such a time frame is
> needed.
>
Auditor and Reviewer, as stated on
> a maximum period of 3 month for remediation before the certification shall
> be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the
> classification as minor, we do not see a necessity for revocation.
> That’s about the relevant facts.
> Let me now reply in detail to Ryans private contribution:
>
> >I would like to suggest that consideration be given to rejecting future
> >audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
> >Widermann, for some period of time. I would suggest this period be at
> least
> >one year long; however, given the technical details of ETSI accreditation,
> >believe a period of three years may be more appropriate.
>
> Dr. Anja Wiedemann (please mind the correct spelling) was not part of the
> audit team. We do not understand why her name is mentioned here.
> One / three years exclusion from audit sounds like a punishment. We do not
> understand where this time frame comes from and why such a time frame is
> needed.
>
https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf
-
the parties tasked with ensuring that the audit is meaningfully able to
ensure the criteria were met and the testing procedures were able to meet
those requirements.
The time frame selected is one that has been consistently used in the past
regarding questions about audits. Unfortunately, there lacks suitable means
to objectively determine whether or not the auditor is sufficiently
competent in remediation of problematic audits. Past procedures have
resulted in indefinite suspensions for some auditors, or temporary
suspensions of their recognition. The choice of three years, rather than
one year, is based on the fact that we have now seen auditors who were not
accredited perform audits against the frameworks, later become accredited,
and retroactively issue reports covering their activities prior to
accreditation. This does not instill confidence in the ETSI approach to
auditor supervision, and thus the longer period is to ensure that no
in-process audits are retroactively certified upon the expiration of the
period. Three years thus aligns with both the 1 year (CA/B Forum) and 2
year (eIDAS) time periods in ensuring that such a possibility is not
technically achievable.
> >If there is a belief that a TSP has failed to meet the requirements of
> >their accreditation, EN 319 403 describes a process for which complaints
> >may be made to either the TSP or to the CAB. This complaint process is
> >further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
> >same process also applies when there have been mistakes by the CAB to
> >adhere to its scheme requirements under EN 319 403 - a complaint may be
> >made with either the CAB or the NAB regarding the CAB's accreditation.
>
> TSPs are not accredited but certified, ETSI EN 319 403 §7.13 does not make
> any additional requirements on complaint procedures but just reference
> ISO/IEC 17065. (The requirements from ISO/IEC 17065 [1], clause 7.13 shall
> apply.) In particular, no procedures for complaints to TSPs or NABs are
> defined (only to CABs).
>
complaints made known to it. You're correct that procedures for complaints
directly to the TSP are not normatively specified by ISO/IEC 17065 or that
of EN 319 403; however, the countenance is made that a client (the TSP) may
have knowledge of and records of complaints outside the scope and purview
of the CAB's complaints.
With respect to the NAB process,
https://www.dakks.de/sites/default/files/71_sd_0_009_e_beschwerdeverfahren_20111118_v1.0_0.pdf
and
in the context of ISO/IEC 17011
https://www.dakks.de/en/content/dakks-successfully-evaluated-ea
>
> >Issue A) As part of their initial response to my complaint, TUVIT, by way
> >of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
> >very first, quick cross check with our audit evidence record, I can only
> >say that we did check issued certificates from within the declared period
> >and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
> >3". However, this statement was in direct conflict with the TSP's own
> >investigation and incident report, provided at
> >https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states
> the
> >mistake was introduced during the development of support - that is, no
> such
> >properly issued certificates were issued.
>
> We do not understand why the important fact that Matthias was not in
> office and replied what he remembered from an audit that was month ago is
> not mentioned here. In addition, he replied that he would verify and come
> back as soon as possible (when he is in office again). That actually
> happened, see below.
>
questioning and a request for proof or evidence of the claim - has been
used to disqualify CAs in the past. This statement was made after the TSP
had themselves already investigated and confirmed this was not possible.
The same standard being applied to CA incident reports is being proposed
here - that incomplete and improper investigations raise serious questions.
> We do not agree at all that core competencies are missing. As you
> described, back in his office Matthias accessed the audit logs, did the
> manual check and found the wrong encoding. IT is obvious, that he has the
> competency to read ASN.1. In addition, it is not correct, that TUVIT does
> not use any tools for certificate checking but the currently available lint
> tools do not support the checking of the qcStatement.
>
been highlighted and identified by external members of the community.
I also disagree that an acceptable response is that the tools did not
support checking the qcStatement; it is expected that the auditor should be
checking for conformity as part of their core competencies. To that extent,
the extension of such tools to check was something the auditor could have
performed. Alternatively, the auditor could have devised testing procedures
to evaluate compliance against the specified module. At no point was the
auditor prevented from examining nor absolved of the expectations simply
because someone else had not written the tool for them yet.
Considering the significance of misencoding of profiles - which has lead to
critical misissuance and security risk (see, for example, Turktrust) - the
question about whether or not TUVIT's testing procedures are adequate to
ensure compliance are extremely relevant and appropriate. TUVIT could, for
example, attempt to resolve these concerns by providing documentation about
their approach to assessing compliance with the relevant certificate
profiles, including methodologies for sampling (not normatively specified
by the relevant documents), their testing and training procedures, and
other evidence that would feel suitable to highlight that this was, indeed,
a 'one off' approach.
This is no different than the "Incident Reports" requested of CAs that fail
to adhere to the requirements - understanding how 100% total and complete
misissuance could be failed to be detected by the auditor IS a very real,
and very pressing, question. It does not have the ease of explaining away
via, say, statistical sampling issues, as this was 100%. It does not have
the ease of explaining away as confusion, given that precise ASN.1 modules
were given. It does not have the ease of explaining away as not
significant, given this was a mandatory field with respects to the profile
asserted. I reject the premise that this is 'minor' for exactly that
reason, and there does not seem a reasonable explanation that has been
offered, other than:
1) TUVIT's assessment methodologies are fundamentally flawed and do not
meet the level of assurance expected of the community
2) TUVIT's and T-Systems' incident handling approach does not meet the
level of expectations of the community. This includes both the oversight of
revocation, but also the approach to auditing for when such a critical
non-conformity has been found. A resolution of "They promise to fix it"
does not provide a reasonable level of assurance, given that at fundamental
question is what other issues TUVIT failed to consider or assess. The lack
of introspection, by TUVIT, as to its auditing process and procedures is
concerning.
With respect to confidentiality and the disclosure of non-conformities,
we've seen auditors make such disclosures - e.g.
https://bug1425805.bmoattachments.org/attachment.cgi?id=9013271
>Furthermore, the complaint process established under EN 319 403 refers to
> >that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
> >put forward within 7.13.5 and 7.13.6 that the complaint resolution process
> >be independent from those from those involved in the audit or those who
> >have consulted with or been employed by the subject of the audit. Matthias
> >Wiedenhorst's involvement in, and resolution of, this complaint, calls
> into
> >question whether TUVIT has acted according to this, given that they are
> >represented as Lead Auditor for T-Systems' audit. At this point, the only
> >objective means of resolving whether or not the process was followed is to
> >lodge a complaint such that the NAB may themselves investigate the
> handling
> >of this complaint.
>
> Wrong again. Please have another look at 7.13.5, it talks about the
> <<decision>> to resolve the dispute and not of the process of resolving
> itself. The decision shall be made by an independent person. It is more
> than natural that for resolution process itself, concerned auditors needs
> to be involved.
>
Lead Auditor. Whether or not this complaint was formally resolved by
TUVIT's management - which I intentionally CC'd in such messages - has not
been forthcoming to date. As a consequence, the only way to determine
objectively is to request an independent evaluation of the complaint
resolution process to ensure that independence has been maintained.
Wiedenhorst, Matthias
Nov 2, 2018, 10:24:57 AM11/2/18
to mozilla-dev-s...@lists.mozilla.org
Dear all,
Still posting on behalf of TÜViT.
On Wed, Oct 31, 2018 at 11:43 AM Wiedenhorst, Matthias via dev-security-policy <dev-secur...@lists.mozilla.org<mailto:dev-secur...@lists.mozilla.org>> wrote:
· Since January 2018, T-Systems issued EV certificates with an incorrect qcStatement. T-Systems was made aware of the problem in October 2018, i.e. for about 9 month the error was not detected/reported
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
T-Systems fixed the error in a timely manner so that certificates now contain the correct qcStatement.
T-Systems identified a deficiency within their systems, made a change on October 5, but did not notify their CAB and SB until October 16.
Under the requirements provided by EN 319 401, 7.9, that does not seem consistent with meeting those requirements.
· TUVIT performed an audit of T-Systems according to ETSI policies EVCP and QCP-w in the beginning of 2018. During the audit the incorrect coding of the qcStatement was not detected.
Yes. I believe this is a significant issue, given the assessment report.
· In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
For non-critical non-conformities, our certification requirements foresee a maximum period of 3 month for remediation before the certification shall be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the classification as minor, we do not see a necessity for revocation.
That’s about the relevant facts.
Let me now reply in detail to Ryans private contribution:
>I would like to suggest that consideration be given to rejecting future
>audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
>Widermann, for some period of time. I would suggest this period be at least
>one year long; however, given the technical details of ETSI accreditation,
>believe a period of three years may be more appropriate.
Dr. Anja Wiedemann (please mind the correct spelling) was not part of the audit team. We do not understand why her name is mentioned here.
One / three years exclusion from audit sounds like a punishment. We do not understand where this time frame comes from and why such a time frame is needed.
Auditor and Reviewer, as stated on https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf - the parties tasked with ensuring that the audit is meaningfully able to ensure the criteria were met and the testing procedures were able to meet those requirements.
Auditors and reviewers need to be distinguished: ISO/IEC 17065 §7.5.1 forbids that the person(s) performing the review is involved in the audit process.
The time frame selected is one that has been consistently used in the past regarding questions about audits. Unfortunately, there lacks suitable means to objectively determine whether or not the auditor is sufficiently competent in remediation of problematic audits. Past procedures have resulted in indefinite suspensions for some auditors, or temporary suspensions of their recognition. The choice of three years, rather than one year, is based on the fact that we have now seen auditors who were not accredited perform audits against the frameworks, later become accredited, and retroactively issue reports covering their activities prior to accreditation. This does not instill confidence in the ETSI approach to auditor supervision, and thus the longer period is to ensure that no in-process audits are retroactively certified upon the expiration of the period. Three years thus aligns with both the 1 year (CA/B Forum) and 2 year (eIDAS) time periods in ensuring that such a possibility is not technically achievable.
>If there is a belief that a TSP has failed to meet the requirements of
>their accreditation, EN 319 403 describes a process for which complaints
>may be made to either the TSP or to the CAB. This complaint process is
>further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
>same process also applies when there have been mistakes by the CAB to
>adhere to its scheme requirements under EN 319 403 - a complaint may be
>made with either the CAB or the NAB regarding the CAB's accreditation.
TSPs are not accredited but certified, ETSI EN 319 403 §7.13 does not make any additional requirements on complaint procedures but just reference ISO/IEC 17065. (The requirements from ISO/IEC 17065 [1], clause 7.13 shall apply.) In particular, no procedures for complaints to TSPs or NABs are defined (only to CABs).
4.1.2.2 (j) provides for the client (the TSP) to inform the CAB any complaints made known to it. You're correct that procedures for complaints directly to the TSP are not normatively specified by ISO/IEC 17065 or that of EN 319 403; however, the countenance is made that a client (the TSP) may have knowledge of and records of complaints outside the scope and purview of the CAB's complaints.
With respect to the NAB process, https://www.dakks.de/sites/default/files/71_sd_0_009_e_beschwerdeverfahren_20111118_v1.0_0.pdf and in the context of ISO/IEC 17011 https://www.dakks.de/en/content/dakks-successfully-evaluated-ea
>Issue A) As part of their initial response to my complaint, TUVIT, by way
>of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
>very first, quick cross check with our audit evidence record, I can only
>say that we did check issued certificates from within the declared period
>and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
>3". However, this statement was in direct conflict with the TSP's own
>investigation and incident report, provided at
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
>mistake was introduced during the development of support - that is, no such
>properly issued certificates were issued.
We do not understand why the important fact that Matthias was not in office and replied what he remembered from an audit that was month ago is not mentioned here. In addition, he replied that he would verify and come back as soon as possible (when he is in office again). That actually happened, see below.
Wrong or misleading information - which was only corrected upon specific questioning and a request for proof or evidence of the claim - has been used to disqualify CAs in the past. This statement was made after the TSP had themselves already investigated and confirmed this was not possible.
The same standard being applied to CA incident reports is being proposed here - that incomplete and improper investigations raise serious questions.
Let’s stay with present case (not with the past): In his email, Matthias said that he is not in his office and will begin to investigate the situation as soon as possible. We think that it is clear, that further information contained in the email cannot be based on the result of the investigation. Back in his office after looking into the audit logs and verifying the qcStatement he gave the proper answer.
We do not agree at all that core competencies are missing. As you described, back in his office Matthias accessed the audit logs, did the manual check and found the wrong encoding. IT is obvious, that he has the competency to read ASN.1. In addition, it is not correct, that TUVIT does not use any tools for certificate checking but the currently available lint tools do not support the checking of the qcStatement.
I disagree with this claim as to core competency - the problem had already been highlighted and identified by external members of the community.
I also disagree that an acceptable response is that the tools did not support checking the qcStatement; it is expected that the auditor should be checking for conformity as part of their core competencies. To that extent, the extension of such tools to check was something the auditor could have performed. Alternatively, the auditor could have devised testing procedures to evaluate compliance against the specified module. At no point was the auditor prevented from examining nor absolved of the expectations simply because someone else had not written the tool for them yet.
As said before, we are using tools in the audit process. The sentence about lint tools should be seen as additional information and nothing else.
Considering the significance of misencoding of profiles - which has lead to critical misissuance and security risk (see, for example, Turktrust) - the question about whether or not TUVIT's testing procedures are adequate to ensure compliance are extremely relevant and appropriate. TUVIT could, for example, attempt to resolve these concerns by providing documentation about their approach to assessing compliance with the relevant certificate profiles, including methodologies for sampling (not normatively specified by the relevant documents), their testing and training procedures, and other evidence that would feel suitable to highlight that this was, indeed, a 'one off' approach.
Let’s stay with the present case: We have not seen any security breach due to the miscoding of the qcStatement and have not seen any arguments how certificates with wrong qcStatement could be used to breach the security in future. That is the reason why believe the non-conformity is a non-critical non-conformity with respect to the performed assessment. In addition, a correct qcStatement is not required by the Baseline Requirements (but by ETSI EN 319 412-5). Having this in mind, we expect that Browser Ecosystem should not be directly affected.
This is no different than the "Incident Reports" requested of CAs that fail to adhere to the requirements - understanding how 100% total and complete misissuance could be failed to be detected by the auditor IS a very real, and very pressing, question. It does not have the ease of explaining away via, say, statistical sampling issues, as this was 100%. It does not have the ease of explaining away as confusion, given that precise ASN.1 modules were given. It does not have the ease of explaining away as not significant, given this was a mandatory field with respects to the profile asserted. I reject the premise that this is 'minor' for exactly that reason, and there does not seem a reasonable explanation that has been offered, other than:
1) TUVIT's assessment methodologies are fundamentally flawed and do not meet the level of assurance expected of the community
This is an exaggerated conclusion based on the only fact that a wrong coded qcStatement was not detected.
2) TUVIT's and T-Systems' incident handling approach does not meet the level of expectations of the community. This includes both the oversight of revocation, but also the approach to auditing for when such a critical non-conformity has been found. A resolution of "They promise to fix it" does not provide a reasonable level of assurance, given that at fundamental question is what other issues TUVIT failed to consider or assess. The lack of introspection, by TUVIT, as to its auditing process and procedures is concerning.
How one single person (with his private hat on) can speak for the community? We explained before why we believe that the wrong qcStatement coding is a non-critical nonconformity (the security of ecosystems is not affected). Where did you copy "They promise to fix it" from? We have not used this expression.
With respect to confidentiality and the disclosure of non-conformities, we've seen auditors make such disclosures - e.g. https://bug1425805.bmoattachments.org/attachment.cgi?id=9013271
Currently, there is no requirement for disclosing such kind of information. If there is no requirement, the confidentiality requirement from ISO/IEC 17065 is binding for us.
As said before, we offer to support the establishment of rules with in the CAB/Forum for making such kind of incidents public in an adequate form.
>Furthermore, the complaint process established under EN 319 403 refers to
>that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
>put forward within 7.13.5 and 7.13.6 that the complaint resolution process
>be independent from those from those involved in the audit or those who
>have consulted with or been employed by the subject of the audit. Matthias
>Wiedenhorst's involvement in, and resolution of, this complaint, calls into
>question whether TUVIT has acted according to this, given that they are
>represented as Lead Auditor for T-Systems' audit. At this point, the only
>objective means of resolving whether or not the process was followed is to
>lodge a complaint such that the NAB may themselves investigate the handling
>of this complaint.
Wrong again. Please have another look at 7.13.5, it talks about the <<decision>> to resolve the dispute and not of the process of resolving itself. The decision shall be made by an independent person. It is more than natural that for resolution process itself, concerned auditors needs to be involved.
As I stated - the only information that has been provided has been by the Lead Auditor. Whether or not this complaint was formally resolved by TUVIT's management - which I intentionally CC'd in such messages - has not been forthcoming to date. As a consequence, the only way to determine objectively is to request an independent evaluation of the complaint resolution process to ensure that independence has been maintained.
Why not asking TUVIT, if the complaint is formally resolved? We never said anything like that in our statements.
Still posting on behalf of TÜViT.
On Wed, Oct 31, 2018 at 11:43 AM Wiedenhorst, Matthias via dev-security-policy <dev-secur...@lists.mozilla.org<mailto:dev-secur...@lists.mozilla.org>> wrote:
· Since January 2018, T-Systems issued EV certificates with an incorrect qcStatement. T-Systems was made aware of the problem in October 2018, i.e. for about 9 month the error was not detected/reported
https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3
T-Systems fixed the error in a timely manner so that certificates now contain the correct qcStatement.
T-Systems identified a deficiency within their systems, made a change on October 5, but did not notify their CAB and SB until October 16.
Under the requirements provided by EN 319 401, 7.9, that does not seem consistent with meeting those requirements.
· TUVIT performed an audit of T-Systems according to ETSI policies EVCP and QCP-w in the beginning of 2018. During the audit the incorrect coding of the qcStatement was not detected.
Yes. I believe this is a significant issue, given the assessment report.
· In several emails, we answered to his complaint, explained our procedures and justified the classification of the encoding error as minor (non-critical) non-conformity.
For non-critical non-conformities, our certification requirements foresee a maximum period of 3 month for remediation before the certification shall be withdrawn. (see also ETSI EN 319 403, section 7.6 b) Based on the classification as minor, we do not see a necessity for revocation.
That’s about the relevant facts.
Let me now reply in detail to Ryans private contribution:
>I would like to suggest that consideration be given to rejecting future
>audits from TUVIT and from that of Matthias Wiedenhorst and Dr. Anja
>Widermann, for some period of time. I would suggest this period be at least
>one year long; however, given the technical details of ETSI accreditation,
>believe a period of three years may be more appropriate.
Dr. Anja Wiedemann (please mind the correct spelling) was not part of the audit team. We do not understand why her name is mentioned here.
One / three years exclusion from audit sounds like a punishment. We do not understand where this time frame comes from and why such a time frame is needed.
Auditor and Reviewer, as stated on https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf - the parties tasked with ensuring that the audit is meaningfully able to ensure the criteria were met and the testing procedures were able to meet those requirements.
The time frame selected is one that has been consistently used in the past regarding questions about audits. Unfortunately, there lacks suitable means to objectively determine whether or not the auditor is sufficiently competent in remediation of problematic audits. Past procedures have resulted in indefinite suspensions for some auditors, or temporary suspensions of their recognition. The choice of three years, rather than one year, is based on the fact that we have now seen auditors who were not accredited perform audits against the frameworks, later become accredited, and retroactively issue reports covering their activities prior to accreditation. This does not instill confidence in the ETSI approach to auditor supervision, and thus the longer period is to ensure that no in-process audits are retroactively certified upon the expiration of the period. Three years thus aligns with both the 1 year (CA/B Forum) and 2 year (eIDAS) time periods in ensuring that such a possibility is not technically achievable.
>If there is a belief that a TSP has failed to meet the requirements of
>their accreditation, EN 319 403 describes a process for which complaints
>may be made to either the TSP or to the CAB. This complaint process is
>further expanded upon in ISO/IEC 17065, which 319 403 incorporates. This
>same process also applies when there have been mistakes by the CAB to
>adhere to its scheme requirements under EN 319 403 - a complaint may be
>made with either the CAB or the NAB regarding the CAB's accreditation.
TSPs are not accredited but certified, ETSI EN 319 403 §7.13 does not make any additional requirements on complaint procedures but just reference ISO/IEC 17065. (The requirements from ISO/IEC 17065 [1], clause 7.13 shall apply.) In particular, no procedures for complaints to TSPs or NABs are defined (only to CABs).
4.1.2.2 (j) provides for the client (the TSP) to inform the CAB any complaints made known to it. You're correct that procedures for complaints directly to the TSP are not normatively specified by ISO/IEC 17065 or that of EN 319 403; however, the countenance is made that a client (the TSP) may have knowledge of and records of complaints outside the scope and purview of the CAB's complaints.
With respect to the NAB process, https://www.dakks.de/sites/default/files/71_sd_0_009_e_beschwerdeverfahren_20111118_v1.0_0.pdf and in the context of ISO/IEC 17011 https://www.dakks.de/en/content/dakks-successfully-evaluated-ea
>Issue A) As part of their initial response to my complaint, TUVIT, by way
>of Matthias Wiedenhorst (Head of Certification Division TSP) stated "As a
>very first, quick cross check with our audit evidence record, I can only
>say that we did check issued certificates from within the declared period
>and that they did contain the proper qcStatement-6.3 / id-etsi-qcs-QcType
>3". However, this statement was in direct conflict with the TSP's own
>investigation and incident report, provided at
>https://bugzilla.mozilla.org/show_bug.cgi?id=1498463#c3 , which states the
>mistake was introduced during the development of support - that is, no such
>properly issued certificates were issued.
We do not understand why the important fact that Matthias was not in office and replied what he remembered from an audit that was month ago is not mentioned here. In addition, he replied that he would verify and come back as soon as possible (when he is in office again). That actually happened, see below.
Wrong or misleading information - which was only corrected upon specific questioning and a request for proof or evidence of the claim - has been used to disqualify CAs in the past. This statement was made after the TSP had themselves already investigated and confirmed this was not possible.
The same standard being applied to CA incident reports is being proposed here - that incomplete and improper investigations raise serious questions.
We do not agree at all that core competencies are missing. As you described, back in his office Matthias accessed the audit logs, did the manual check and found the wrong encoding. IT is obvious, that he has the competency to read ASN.1. In addition, it is not correct, that TUVIT does not use any tools for certificate checking but the currently available lint tools do not support the checking of the qcStatement.
I disagree with this claim as to core competency - the problem had already been highlighted and identified by external members of the community.
I also disagree that an acceptable response is that the tools did not support checking the qcStatement; it is expected that the auditor should be checking for conformity as part of their core competencies. To that extent, the extension of such tools to check was something the auditor could have performed. Alternatively, the auditor could have devised testing procedures to evaluate compliance against the specified module. At no point was the auditor prevented from examining nor absolved of the expectations simply because someone else had not written the tool for them yet.
Considering the significance of misencoding of profiles - which has lead to critical misissuance and security risk (see, for example, Turktrust) - the question about whether or not TUVIT's testing procedures are adequate to ensure compliance are extremely relevant and appropriate. TUVIT could, for example, attempt to resolve these concerns by providing documentation about their approach to assessing compliance with the relevant certificate profiles, including methodologies for sampling (not normatively specified by the relevant documents), their testing and training procedures, and other evidence that would feel suitable to highlight that this was, indeed, a 'one off' approach.
This is no different than the "Incident Reports" requested of CAs that fail to adhere to the requirements - understanding how 100% total and complete misissuance could be failed to be detected by the auditor IS a very real, and very pressing, question. It does not have the ease of explaining away via, say, statistical sampling issues, as this was 100%. It does not have the ease of explaining away as confusion, given that precise ASN.1 modules were given. It does not have the ease of explaining away as not significant, given this was a mandatory field with respects to the profile asserted. I reject the premise that this is 'minor' for exactly that reason, and there does not seem a reasonable explanation that has been offered, other than:
1) TUVIT's assessment methodologies are fundamentally flawed and do not meet the level of assurance expected of the community
2) TUVIT's and T-Systems' incident handling approach does not meet the level of expectations of the community. This includes both the oversight of revocation, but also the approach to auditing for when such a critical non-conformity has been found. A resolution of "They promise to fix it" does not provide a reasonable level of assurance, given that at fundamental question is what other issues TUVIT failed to consider or assess. The lack of introspection, by TUVIT, as to its auditing process and procedures is concerning.
With respect to confidentiality and the disclosure of non-conformities, we've seen auditors make such disclosures - e.g. https://bug1425805.bmoattachments.org/attachment.cgi?id=9013271
As said before, we offer to support the establishment of rules with in the CAB/Forum for making such kind of incidents public in an adequate form.
>Furthermore, the complaint process established under EN 319 403 refers to
>that of ISO/IEC 17065:2012. Under ISO/IEC 17065:2012, the requirement is
>put forward within 7.13.5 and 7.13.6 that the complaint resolution process
>be independent from those from those involved in the audit or those who
>have consulted with or been employed by the subject of the audit. Matthias
>Wiedenhorst's involvement in, and resolution of, this complaint, calls into
>question whether TUVIT has acted according to this, given that they are
>represented as Lead Auditor for T-Systems' audit. At this point, the only
>objective means of resolving whether or not the process was followed is to
>lodge a complaint such that the NAB may themselves investigate the handling
>of this complaint.
Wrong again. Please have another look at 7.13.5, it talks about the <<decision>> to resolve the dispute and not of the process of resolving itself. The decision shall be made by an independent person. It is more than natural that for resolution process itself, concerned auditors needs to be involved.
As I stated - the only information that has been provided has been by the Lead Auditor. Whether or not this complaint was formally resolved by TUVIT's management - which I intentionally CC'd in such messages - has not been forthcoming to date. As a consequence, the only way to determine objectively is to request an independent evaluation of the complaint resolution process to ensure that independence has been maintained.
Ryan Sleevi
Nov 2, 2018, 11:25:35 AM11/2/18
to M.Wied...@tuvit.de, mozilla-dev-security-policy
On Fri, Nov 2, 2018 at 10:24 AM Wiedenhorst, Matthias via
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> Auditor and Reviewer, as stated on
> https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf
> - the parties tasked with ensuring that the audit is meaningfully able to
> ensure the criteria were met and the testing procedures were able to meet
> those requirements.
>
> Auditors and reviewers need to be distinguished: ISO/IEC 17065 §7.5.1
> forbids that the person(s) performing the review is involved in the audit
> process.
>
Indeed - and yet do you agree that the Reviewer is tasked with reviewing
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> Auditor and Reviewer, as stated on
> https://www.tuvit.de/fileadmin/Content/TUV_IT/zertifikate/en/AA2018072001_Audit_Attestation_E_Deutsche-Telekom-Root-CA-2_20180718_s.pdf
> - the parties tasked with ensuring that the audit is meaningfully able to
> ensure the criteria were met and the testing procedures were able to meet
> those requirements.
>
> Auditors and reviewers need to be distinguished: ISO/IEC 17065 §7.5.1
> forbids that the person(s) performing the review is involved in the audit
> process.
>
the audit methodology and artifacts to ensure it appropriately meets the
objectives expected and required? A multi-party failure to assure the
necessary assurance is just that - a multi-party failure.
an attempt to deflect or ignore the ample set of past precedent and
expectations. As such, I reject the premise that this should be considered
without regard to past failures by CAs or other auditors - as an auditor
being entrusted to report truthfully and faithfully to the community about
a CAs compliance with its own CP, CPS, and the appropriate supervisory
framework, auditors are expected to consider the best practices and
precedents in their activities and actions.
With respect to your suggestion that the information can not be relied
upon, I'm more than happy to provide the full e-mail chain if there's some
consideration given to misinterpretation. However, I do not believe the
reply at all indicates that the information contained was not reliable or
may be counter-factual and to be corrected later. Yes, Matthias stated they
were out of office - but then immediately began with a remark that "As a
very first, quick cross check with our audit evidence record, I can only
say that we did check issued certificates".
I can understand the argument being made here that, on a more detailed
examination of your audit evidence record, you discovered that it was not,
in fact, checked. However, that raises significant concerns that a quick
check can lead to a completely opposite conclusion, particularly for a
supposedly-skilled practitioner.
> As said before, we are using tools in the audit process. The sentence
> about lint tools should be seen as additional information and nothing else.
>
readily available off the shelf. Further, in your description of the
methodology used, it was clear that human visual inspection without regard
to the actual specification was performed. While you may have used a tool
to, say, dump the DER-encoded contents into a structural representation,
the procedures for examining that structural representation against the
profile are clearly and significantly deficient.
> Considering the significance of misencoding of profiles - which has lead
> to critical misissuance and security risk (see, for example, Turktrust) -
> the question about whether or not TUVIT's testing procedures are adequate
> to ensure compliance are extremely relevant and appropriate. TUVIT could,
> for example, attempt to resolve these concerns by providing documentation
> about their approach to assessing compliance with the relevant certificate
> profiles, including methodologies for sampling (not normatively specified
> by the relevant documents), their testing and training procedures, and
> other evidence that would feel suitable to highlight that this was, indeed,
> a 'one off' approach.
>
> Let’s stay with the present case: We have not seen any security breach due
> to the miscoding of the qcStatement and have not seen any arguments how
> certificates with wrong qcStatement could be used to breach the security in
> future. That is the reason why believe the non-conformity is a non-critical
> non-conformity with respect to the performed assessment. In addition, a
> correct qcStatement is not required by the Baseline Requirements (but by
> ETSI EN 319 412-5). Having this in mind, we expect that Browser Ecosystem
> should not be directly affected.
>
no longer be recognized as a suitable auditor.
The community has, for the better part of a decade now, recognized that the
failure to appropriately encode certificates minimally represents a
significant barrier to interoperability. The need to support and
interoperate with such broken certificates has resulted in multiple
security vulnerabilities being introduced into the ecosystem, such as the
processing of public key encodings that were improperly DER encoded or the
improper encoding of negative serial numbers.
Similarly, the failure to adhere to the specified profile has resulted in a
number of high-profile CA security events. For example, the inclusion of a
"cA:True" within the basicConstraints of a certificate intended for
subscribers, or the misuse of the EV certificate profile for testing, have
lead to the dissolution of CA services due to the critical blows to trust
in those providers.
One of the most core, and fundamental, expectations of auditors is that
they have the necessary technical skill to review the configured profiles,
software, and systems to ensure the correct production of certificates.
That these produced certificates align with the stated CP and CPS, and any
relevant specifications incorporated therein. That they have the necessary
technical judgement to recognize the critical nature of these functions,
and its fundamental impact on the trust ecosystem.
One would expect the degree of auditor competency to review the produced
certificates on an aggressive basis, to ensure that the level of confidence
in the certification is appropriate. Any sampling methodology used - even
to the degree of just a single certificate - would and should have revealed
this critical misconfiguration. That no such detection or discovery was
made fundamentally calls into question the level of supervision and
oversight by the CAB, which is a critical function to ensure those relying
on the certification do not face security or compatibility risks.
Furthermore, that within the framework of the nominally more rigorous, more
critical trust objectives of qualified certificates, one would expect that
the level of scrutiny and supervision is beyond reproach, so as not to call
into disrepute the entire concept and framework.
TUVIT has not met these expectations, and in doing so, fundamentally calls
into question the necessary competencies of the auditors and the
methodology employed. This is fundamentally no different than the rejection
of those WebTrust auditors who offer letters ascribing high confidence that
the principles and criteria are met, while readily available public
information actively contradicts this, and even simple sampling would have
revealed these flaws. This is not singling out ETSI-using auditors; this is
recognizing an auditor that is failing to meet the requirements of the
community.
> 1) TUVIT's assessment methodologies are fundamentally flawed and do not
> meet the level of assurance expected of the community
>
> This is an exaggerated conclusion based on the only fact that a wrong
> coded qcStatement was not detected.
>
T-Systems for some time, as demonstrated through
https://www.t-systems.com/blob/776674/ec9fa6212c7a1c5537cfb57ad33af6bb/DL_Trust-Center_ETSI_Audit.pdf
During those same periods, T-Systems had multiple issues of material
non-compliance, as demonstrated through
https://bug1391074.bmoattachments.org/attachment.cgi?id=8915934
This is not an exaggerated conclusion by any means, but rather, a
systematic failure to apply sound audit methodology and best practices.
> 2) TUVIT's and T-Systems' incident handling approach does not meet the
> level of expectations of the community. This includes both the oversight of
> revocation, but also the approach to auditing for when such a critical
> non-conformity has been found. A resolution of "They promise to fix it"
> does not provide a reasonable level of assurance, given that at fundamental
> question is what other issues TUVIT failed to consider or assess. The lack
> of introspection, by TUVIT, as to its auditing process and procedures is
> concerning.
>
> How one single person (with his private hat on) can speak for the
> community?
than happy to detail amply the community expectations, captured in bugs and
in discussions, about the handling of revocation and errors. In this
regard, consider this a summary of the past discussions setting forth the
expectations.
> We explained before why we believe that the wrong qcStatement coding is a
> non-critical nonconformity (the security of ecosystems is not affected).
> Where did you copy "They promise to fix it" from? We have not used this
> expression.
>
"We apply this requirement in the context of a certificate decision (for
the issuance of a ETSI certificate) also for this case of a non-conformity
found for an issued ETSI certificate: Correction of the bug and revocation
of all concerned website certificates till end of 2018 will fulfill the
requirement."
and
"Failure to comply with the Baseline Requirements due to the non-revocation
of the concerned website certificates within the required time line is
analogously regarded as minor non-conformity, that shall be fixed within
three month."
Both of these assert that remediation within three months time is suitable
to satisfy the needs. While that is certainly the minimum required, that
does not align with the expectations for resolution and/or revocation.
Wayne Thayer
Nov 2, 2018, 5:00:16 PM11/2/18
to Ryan Sleevi, Wiedenhorst, Matthias, mozilla-dev-security-policy
I am particularly disturbed by three points made by TUVIT during this
discussion:
1. A malformed qcStatement extension is a minor non-conformity because
there is no known security risk - This argument is incredibly dangerous and
harmful. It implies that all sorts of well-defined requirements can be
ignored if the CA and/or auditor decide that there is no risk in doing so.
2. Citing ISO 17065 as requiring non-conformities be kept confidential -
adding on Ryan's comments, we're seeing increasing disclosure of audit
findings (another example:
https://netlock.hu/download/etsi-certification-netlock/?wpdmdl=57340),
leading me to believe that this is TUVIT's own interpretation of the
confidentiality requirements. I don't believe that TUVIT needs "the
3. The argument that T-Systems has 3-months to revoke these certificates -
while I understand that under ETSI TSPs have 3 months to correct minor
non-conformities, using that as an excuse to ignore CAB Forum revocation
requirements is unacceptable, and perhaps explains why we see such poor
compliance with this requirement. If this is indeed the accepted
interpretation (please confirm), then I will look for ways to fix this via
Mozilla policy.
- Wayne
discussion:
1. A malformed qcStatement extension is a minor non-conformity because
there is no known security risk - This argument is incredibly dangerous and
harmful. It implies that all sorts of well-defined requirements can be
ignored if the CA and/or auditor decide that there is no risk in doing so.
2. Citing ISO 17065 as requiring non-conformities be kept confidential -
adding on Ryan's comments, we're seeing increasing disclosure of audit
findings (another example:
https://netlock.hu/download/etsi-certification-netlock/?wpdmdl=57340),
leading me to believe that this is TUVIT's own interpretation of the
confidentiality requirements. I don't believe that TUVIT needs "the
establishment of rules with in the CAB/Forum for making such kind of
incidents public" in order to begin making these disclosures.
3. The argument that T-Systems has 3-months to revoke these certificates -
while I understand that under ETSI TSPs have 3 months to correct minor
non-conformities, using that as an excuse to ignore CAB Forum revocation
requirements is unacceptable, and perhaps explains why we see such poor
compliance with this requirement. If this is indeed the accepted
interpretation (please confirm), then I will look for ways to fix this via
Mozilla policy.
- Wayne
Nick Pope
Nov 5, 2018, 3:28:44 PM11/5/18
to mozilla-dev-s...@lists.mozilla.org
It is very unfortunate that at this time the owners of root store programs openly criticise one of the main auditors working on improvements to European based audits. After a number of years of audits of European CAs based on ETSI EN 319 403 being recognised as meeting the requirements of publicly trusted certificates, ETSI is working and with European auditors on further updates to improve the acceptability of European audits to root store programs. It seems to be going against this initiative to suggest draconian measures of excluding TUVIT audit from the root programs whose impact are totally out of proportion the possible impact of the issues raised.
I suggest that the providers of root stores to return to the negotiations for further improving European based audits that I understood had started at the recent CA/Browser forum. The current approach of making public criticisms against those who are trying to make improvements to the European CA audits is making the current direct discussions with root store providers difficult to progress. So unless it is the objective to deliberately exclude European CAs from their root programs, which I believe is not the case, I suggest that we return to the direct discussions with the providers of root store on how to further improve European audits so that can better take into account the root program requirements.
Nick Pope, Vice-Chair ETSI TC on Electronic Signatures and [Trust] Infrastructures
I suggest that the providers of root stores to return to the negotiations for further improving European based audits that I understood had started at the recent CA/Browser forum. The current approach of making public criticisms against those who are trying to make improvements to the European CA audits is making the current direct discussions with root store providers difficult to progress. So unless it is the objective to deliberately exclude European CAs from their root programs, which I believe is not the case, I suggest that we return to the direct discussions with the providers of root store on how to further improve European audits so that can better take into account the root program requirements.
Nick Pope, Vice-Chair ETSI TC on Electronic Signatures and [Trust] Infrastructures
Ryan Sleevi
Nov 5, 2018, 3:55:54 PM11/5/18
to nhp...@gmail.com, mozilla-dev-security-policy
On Mon, Nov 5, 2018 at 3:28 PM Nick Pope via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> It is very unfortunate that at this time the owners of root store programs
> openly criticise one of the main auditors working on improvements to
> European based audits. After a number of years of audits of European CAs
> based on ETSI EN 319 403 being recognised as meeting the requirements of
> publicly trusted certificates, ETSI is working and with European auditors
> on further updates to improve the acceptability of European audits to root
> store programs. It seems to be going against this initiative to suggest
> draconian measures of excluding TUVIT audit from the root programs whose
> impact are totally out of proportion the possible impact of the issues
> raised.
>
> I suggest that the providers of root stores to return to the negotiations
> for further improving European based audits that I understood had started
> at the recent CA/Browser forum. The current approach of making public
> criticisms against those who are trying to make improvements to the
> European CA audits is making the current direct discussions with root store
> providers difficult to progress. So unless it is the objective to
> deliberately exclude European CAs from their root programs, which I believe
> is not the case, I suggest that we return to the direct discussions with
> the providers of root store on how to further improve European audits so
> that can better take into account the root program requirements.
>
> Nick Pope, Vice-Chair ETSI TC on Electronic Signatures and [Trust]
> Infrastructures
>
Respectfully, comments like this unfortunately bring even greater concern
dev-secur...@lists.mozilla.org> wrote:
> It is very unfortunate that at this time the owners of root store programs
> openly criticise one of the main auditors working on improvements to
> European based audits. After a number of years of audits of European CAs
> based on ETSI EN 319 403 being recognised as meeting the requirements of
> publicly trusted certificates, ETSI is working and with European auditors
> on further updates to improve the acceptability of European audits to root
> store programs. It seems to be going against this initiative to suggest
> draconian measures of excluding TUVIT audit from the root programs whose
> impact are totally out of proportion the possible impact of the issues
> raised.
>
> I suggest that the providers of root stores to return to the negotiations
> for further improving European based audits that I understood had started
> at the recent CA/Browser forum. The current approach of making public
> criticisms against those who are trying to make improvements to the
> European CA audits is making the current direct discussions with root store
> providers difficult to progress. So unless it is the objective to
> deliberately exclude European CAs from their root programs, which I believe
> is not the case, I suggest that we return to the direct discussions with
> the providers of root store on how to further improve European audits so
> that can better take into account the root program requirements.
>
> Nick Pope, Vice-Chair ETSI TC on Electronic Signatures and [Trust]
> Infrastructures
>
with respect to the ETSI process.
A significant number of improvements have been made to the ecosystem by
recognizing when mistakes are made and taking steps to improve. It has now
seen both TUVIT and the Vice-Chair of the ETSI TC on ESI instead suggest
these are not mistakes and downplay their significance. This prevents
meaningful improvements, because it fails to recognize that there exist
fundamental issues.
I am all in favor of ensuring that all accepted audit schemes meet the
necessary level of robustness for the community. Much work has been done
with WebTrust, through their active engagement with Browsers to ensure that
the needs of the consumers are being met. ETSI has only recently begun to
recognize these issues, and while we are indeed seeing the beginnings of
fruitful engagement, we should not suggest that such seeds are a reasonable
justification to ignore gross negligence in security-critical functions OR
the deeply concerning dismissiveness of those concerns.
I'm sure you can understand it would be deeply offensive if, on the basis
of such collaborations with WebTrust, it be suggested that no WebTrust
auditor be disqualified. Similarly, I'm sure you can understand it would be
deeply offensive to the purpose, values, and goals to suggest that because
CAs participate in m.d.s.p., they should be excluded from accountability.
At the end of the day, browsers are accountable to ensuring their users are
secure, and regardless of how productive our conversations may be, if the
level of security is not met, it's entirely appropriate and necessary to
take steps to protect users.
I hope that, as Vice-Chair of the ETSI TC on ESI, and on behalf of
auditors, careful introspection is performed in comparing how these
statements sound when compared with CAs that have been distrusted due to
gross negligence and misissuance. Failures to acknowledge or recognize the
problem, failures to have implemented reasonable steps to resolve such
issues, repeated failures to achieve the necessary level of security, do
more to harm the brand of that organization and its products than
statements suggesting distrust.
Wayne Thayer
Nov 5, 2018, 4:50:53 PM11/5/18
to Ryan Sleevi, nhp...@gmail.com, mozilla-dev-security-policy
In addition, I take exception to the statement that open criticism is a bad
approach and the implication that private discussions are the best way to
make improvements. This is clearly not Mozilla's philosophy.
I do believe that we all need to be careful to follow Mozilla forum
etiquette [1] and community participation guidelines [2], particularly the
section titled "Be Direct but Professional". However, transparent
discussions are a core Mozilla Principle [3]:
"Transparent community-based processes promote participation,
accountability and trust."
I look forward to more open and constructive discussions aimed at improving
the quality and transparency of CA audits, regardless of the audit scheme.
- Wayne
[1] https://www.mozilla.org/en-US/about/forums/etiquette/
[2] https://www.mozilla.org/en-US/about/governance/policies/participation/
[3] https://www.mozilla.org/en-US/about/manifesto/
approach and the implication that private discussions are the best way to
make improvements. This is clearly not Mozilla's philosophy.
I do believe that we all need to be careful to follow Mozilla forum
etiquette [1] and community participation guidelines [2], particularly the
section titled "Be Direct but Professional". However, transparent
discussions are a core Mozilla Principle [3]:
"Transparent community-based processes promote participation,
accountability and trust."
I look forward to more open and constructive discussions aimed at improving
the quality and transparency of CA audits, regardless of the audit scheme.
- Wayne
[1] https://www.mozilla.org/en-US/about/forums/etiquette/
[2] https://www.mozilla.org/en-US/about/governance/policies/participation/
[3] https://www.mozilla.org/en-US/about/manifesto/
Wiedenhorst, Matthias
Nov 6, 2018, 4:48:51 AM11/6/18
to wth...@mozilla.com, mozilla-dev-s...@lists.mozilla.org
Thanks for focussing the discussion on the substance and thus allowing the community moving towards a common understanding and towards defining adequate requirements for treating such kind of incidents. We are very much interested in improving the existing requirements as we have done in the past by participating actively in drafting of ETSI standards, including EN 319 403.
I am particularly disturbed by three points made by TUVIT during this discussion:
1. A malformed qcStatement extension is a minor non-conformity because there is no known security risk - This argument is incredibly dangerous and harmful. It implies that all sorts of well-defined requirements can be ignored if the CA and/or auditor decide that there is no risk in doing so.
We agree that neither CAs nor Auditors may ignore a failure against well-defined requirements independent of their classification as critical or non-critical.
This is already currently not the case as both are non-conformities.
CAs and auditors have to address every non-conformity and cannot ignore them. The classification (just) refers to the timeframe for remediation. Every major non-conformity has to be remediated before ETSI certification is possible. In case of existing certificates, the certificate is withdrawn. With minor non-conformities, an auditor may issue an ETSI certificate under the condition that every minor non-conformity is remediated within timeframe of 3 month (maximum) after conclusion of the audit. Existing certificates may stay valid but must be withdrawn if the CA fails to remediate within the deadline.
2. Citing ISO 17065 as requiring non-conformities be kept confidential - adding on Ryan's comments, we're seeing increasing disclosure of audit findings (another example: https://netlock.hu/download/etsi-certification-netlock/?wpdmdl=57340), leading me to believe that this is TUVIT's own interpretation of the confidentiality requirements. I don't believe that TUVIT needs "the establishment of rules with in the CAB/Forum for making such kind of incidents public" in order to begin making these disclosures.
Section 4.5 of ISO/IEC 17065 states that in general all non-public information shall be regarded as confidential. However, that section also allows that CAB and TSP can agree between each other about information not to be regarded as confidential.
Our interpretation (which we think is aligned with current interpretation of general data protection legislation informally stated as “everything which is not explicitly required/allowed is forbidden”), indeed follows a minimum principle. So you are right, with consent of the TSP it is possible and we are willing to request such consent in future.
We suggest the establishment of general rules / requirements valid for all auditors instead of individual / different commitments. These rules could be on the content of public audit reports and on the roles of audits during security incidents including reporting and should allow browsers and the interested community to obtain the necessary information to get a good picture on the incident and the assessment of the auditor.
3. The argument that T-Systems has 3-months to revoke these certificates - while I understand that under ETSI TSPs have 3 months to correct minor non-conformities, using that as an excuse to ignore CAB Forum revocation requirements is unacceptable, and perhaps explains why we see such poor compliance with this requirement. If this is indeed the accepted interpretation (please confirm), then I will look for ways to fix this via Mozilla policy.
- Wayne
>From the ETSI certification point of view, this is the interpretation. Failure to revoke within the required timeframe is clearly a non-conformity. Nevertheless, if the non-conformity has been rated as minor non-conformity (due to the individual circumstances), there would be a period of 3 month before the corresponding ETSI certification would be withdrawn.
However, we do see your concern and it is a very reasonable one. Using this construct to deliberately delay revocation is not at all desired. How could we deal with it?
One possibility would be for the CAB to mandatorily require the TSP to publish the failure to adhere to the certificate revocation timeline requirement as bug in the Bugzilla (as already required from the TSP by Mozilla Policy) before the rating as minor non-conformity is possible. Without publication, it has to be rated as major non-conformity and hence an existing ETSI certificate will be withdrawn. This would facilitate the interest of transparency and would allow Mozilla, if regarded necessary, to take further action regardless of a still existing ETSI certificate. In the past, the ETSI certificate was not regarded as the primary audit deliverable by root store operators; this is the audit attestation letter. Combined with number 2 above, in such the case the next audit attestation letter would also state the failed revocation deadline as non-conformity.
Best regards
Matthias
I am particularly disturbed by three points made by TUVIT during this discussion:
1. A malformed qcStatement extension is a minor non-conformity because there is no known security risk - This argument is incredibly dangerous and harmful. It implies that all sorts of well-defined requirements can be ignored if the CA and/or auditor decide that there is no risk in doing so.
This is already currently not the case as both are non-conformities.
CAs and auditors have to address every non-conformity and cannot ignore them. The classification (just) refers to the timeframe for remediation. Every major non-conformity has to be remediated before ETSI certification is possible. In case of existing certificates, the certificate is withdrawn. With minor non-conformities, an auditor may issue an ETSI certificate under the condition that every minor non-conformity is remediated within timeframe of 3 month (maximum) after conclusion of the audit. Existing certificates may stay valid but must be withdrawn if the CA fails to remediate within the deadline.
2. Citing ISO 17065 as requiring non-conformities be kept confidential - adding on Ryan's comments, we're seeing increasing disclosure of audit findings (another example: https://netlock.hu/download/etsi-certification-netlock/?wpdmdl=57340), leading me to believe that this is TUVIT's own interpretation of the confidentiality requirements. I don't believe that TUVIT needs "the establishment of rules with in the CAB/Forum for making such kind of incidents public" in order to begin making these disclosures.
Our interpretation (which we think is aligned with current interpretation of general data protection legislation informally stated as “everything which is not explicitly required/allowed is forbidden”), indeed follows a minimum principle. So you are right, with consent of the TSP it is possible and we are willing to request such consent in future.
We suggest the establishment of general rules / requirements valid for all auditors instead of individual / different commitments. These rules could be on the content of public audit reports and on the roles of audits during security incidents including reporting and should allow browsers and the interested community to obtain the necessary information to get a good picture on the incident and the assessment of the auditor.
3. The argument that T-Systems has 3-months to revoke these certificates - while I understand that under ETSI TSPs have 3 months to correct minor non-conformities, using that as an excuse to ignore CAB Forum revocation requirements is unacceptable, and perhaps explains why we see such poor compliance with this requirement. If this is indeed the accepted interpretation (please confirm), then I will look for ways to fix this via Mozilla policy.
- Wayne
However, we do see your concern and it is a very reasonable one. Using this construct to deliberately delay revocation is not at all desired. How could we deal with it?
One possibility would be for the CAB to mandatorily require the TSP to publish the failure to adhere to the certificate revocation timeline requirement as bug in the Bugzilla (as already required from the TSP by Mozilla Policy) before the rating as minor non-conformity is possible. Without publication, it has to be rated as major non-conformity and hence an existing ETSI certificate will be withdrawn. This would facilitate the interest of transparency and would allow Mozilla, if regarded necessary, to take further action regardless of a still existing ETSI certificate. In the past, the ETSI certificate was not regarded as the primary audit deliverable by root store operators; this is the audit attestation letter. Combined with number 2 above, in such the case the next audit attestation letter would also state the failed revocation deadline as non-conformity.
Best regards
Matthias
Ryan Sleevi
Nov 6, 2018, 12:05:42 PM11/6/18
to Wiedenhorst, Matthias, Wayne Thayer, mozilla-dev-security-policy
On Tue, Nov 6, 2018 at 4:48 AM Wiedenhorst, Matthias via
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> Section 4.5 of ISO/IEC 17065 states that in general all non-public
> information shall be regarded as confidential. However, that section also
> allows that CAB and TSP can agree between each other about information not
> to be regarded as confidential.
> Our interpretation (which we think is aligned with current interpretation
> of general data protection legislation informally stated as “everything
> which is not explicitly required/allowed is forbidden”), indeed follows a
> minimum principle. So you are right, with consent of the TSP it is possible
> and we are willing to request such consent in future.
> We suggest the establishment of general rules / requirements valid for all
> auditors instead of individual / different commitments. These rules could
> be on the content of public audit reports and on the roles of audits during
> security incidents including reporting and should allow browsers and the
> interested community to obtain the necessary information to get a good
> picture on the incident and the assessment of the auditor.
>
I fundamentally disagree with this approach, and believe that rather than
dev-security-policy <dev-secur...@lists.mozilla.org> wrote:
> Section 4.5 of ISO/IEC 17065 states that in general all non-public
> information shall be regarded as confidential. However, that section also
> allows that CAB and TSP can agree between each other about information not
> to be regarded as confidential.
> Our interpretation (which we think is aligned with current interpretation
> of general data protection legislation informally stated as “everything
> which is not explicitly required/allowed is forbidden”), indeed follows a
> minimum principle. So you are right, with consent of the TSP it is possible
> and we are willing to request such consent in future.
> We suggest the establishment of general rules / requirements valid for all
> auditors instead of individual / different commitments. These rules could
> be on the content of public audit reports and on the roles of audits during
> security incidents including reporting and should allow browsers and the
> interested community to obtain the necessary information to get a good
> picture on the incident and the assessment of the auditor.
>
creating a common baseline, this would lower the bars for security and
reliability. This is because auditors, such as TUVIT is doing so here,
would argue "We were only doing what we required" - without recognizing
fundamentally that things evolve, and auditors need to evolve with them.
Consider the examples given of other auditors that have found ways to
disclose more information to relying parties and browsers. They've shown
that there's the ability and necessity to step up to meet community
expectations. All auditors should be encouraged to do so - to constantly
improve. To the extent we specify a common baseline, it will forever be
that - the lowest bar, but not reflective of expectation or need.
A CAB wishing to provide high assurance to the users of its reports, and to
the TSP-using ecosystem, would constantly be looking for ways to improve
the assurance, and to publicize those best practices, so that other CABs
may learn and integrate such practices.
> 3. The argument that T-Systems has 3-months to revoke these certificates -
> while I understand that under ETSI TSPs have 3 months to correct minor
> non-conformities, using that as an excuse to ignore CAB Forum revocation
> requirements is unacceptable, and perhaps explains why we see such poor
> compliance with this requirement. If this is indeed the accepted
> interpretation (please confirm), then I will look for ways to fix this via
> Mozilla policy.
>
> - Wayne
>
> From the ETSI certification point of view, this is the interpretation.
> Failure to revoke within the required timeframe is clearly a
> non-conformity. Nevertheless, if the non-conformity has been rated as minor
> non-conformity (due to the individual circumstances), there would be a
> period of 3 month before the corresponding ETSI certification would be
> withdrawn.
> However, we do see your concern and it is a very reasonable one. Using
> this construct to deliberately delay revocation is not at all desired. How
> could we deal with it?
>
certainly does not align with industry best practice - as reflected through
the Root Program agreements that exist, so how do you, as the CAB, defend
those classifications?
Another is to recognize that the CAB (and/or SB, depending) must be
notified of anything the TSP changes that may affect the conformity, and
there is a public interest in making that information available. Having the
CAB notify the program of any non-conformities found, both those that
affect certification and those that do not ("minor" non-conformities),
would help ensure the necessary public confidence in ETSI, and be a step
above what WebTrust provides.
> One possibility would be for the CAB to mandatorily require the TSP to
> publish the failure to adhere to the certificate revocation timeline
> requirement as bug in the Bugzilla (as already required from the TSP by
> Mozilla Policy) before the rating as minor non-conformity is possible.
> Without publication, it has to be rated as major non-conformity and hence
> an existing ETSI certificate will be withdrawn. This would facilitate the
> interest of transparency and would allow Mozilla, if regarded necessary, to
> take further action regardless of a still existing ETSI certificate. In the
> past, the ETSI certificate was not regarded as the primary audit
> deliverable by root store operators; this is the audit attestation letter.
> Combined with number 2 above, in such the case the next audit attestation
> letter would also state the failed revocation deadline as non-conformity.
>
CABs and audited CAs that the value is in the certification, and the
certification has consistently been what has been provided to Root Store
Operators. As Root Store Operators have begun pointing out the
certification alone is deficient in the information, we've seen CABs say
start using 'this' report instead - even though the whole assurance of the
audit is based on the certification and the auditor's accreditation thereof.
Nick Pope
Nov 8, 2018, 9:24:42 AM11/8/18
to mozilla-dev-s...@lists.mozilla.org
Following on from Waynes earlier positive statement:
"I look forward to more open and constructive discussions aimed at improving
the quality and transparency of CA audits, regardless of the audit scheme."
I believe centring discussion on one particular auditor is not progressing things with regards generally improving audits.
I understood from my EU colleagues that Ryan and Wayne had undertaken to produce a "wish list" covering requirements that they had on audits. We can then we can then discuss this with the European stakeholders and see how we could best answer the wish list. This wish list would be most helpful if it builds on the measures already proposed in TS 119 403-2 and its parent standards which provide specific requirements on all European audits for PTC. I understand also that we undertook to meet with WebTrust in December to get an understand of each other schemes which could lead to resolution of any alignment issues.
I kindly request that we progress the earlier plan by identifying your full list of requirements related to the current European audit scheme.
Nick
"I look forward to more open and constructive discussions aimed at improving
the quality and transparency of CA audits, regardless of the audit scheme."
I understood from my EU colleagues that Ryan and Wayne had undertaken to produce a "wish list" covering requirements that they had on audits. We can then we can then discuss this with the European stakeholders and see how we could best answer the wish list. This wish list would be most helpful if it builds on the measures already proposed in TS 119 403-2 and its parent standards which provide specific requirements on all European audits for PTC. I understand also that we undertook to meet with WebTrust in December to get an understand of each other schemes which could lead to resolution of any alignment issues.
I kindly request that we progress the earlier plan by identifying your full list of requirements related to the current European audit scheme.
Nick
Ryan Sleevi
Nov 8, 2018, 10:34:27 AM11/8/18
to Nick Pope, mozilla-dev-s...@lists.mozilla.org
On Thu, Nov 8, 2018 at 6:24 AM Nick Pope via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> Following on from Waynes earlier positive statement:
>
> "I look forward to more open and constructive discussions aimed at
> improving
> the quality and transparency of CA audits, regardless of the audit scheme."
>
> I believe centring discussion on one particular auditor is not progressing
> things with regards generally improving audits.
That sounds very much like you don’t believe in either accountability or in
dev-secur...@lists.mozilla.org> wrote:
> Following on from Waynes earlier positive statement:
>
> "I look forward to more open and constructive discussions aimed at
> improving
> the quality and transparency of CA audits, regardless of the audit scheme."
>
> I believe centring discussion on one particular auditor is not progressing
> things with regards generally improving audits.
trustworthiness being necessary for auditors. Statements like this, which
actively promote overlooking fundamentally defective application of the
existing requirements, calls the ETSI model itself into disrepute. I
realize the opposite is your goal, but I hope you can understand how such
an approach is fundamentally and deeply offensive to the trust ecosystem.
Perhaps put differently: Do you believe that the audit criteria under ETSI
are sufficiently clear to set forward an expectation that certificates
conform to a profile?
If no, we should not use or accept ETSI audits until such a time as the
issues are resolved.
If yes, then it is absolutely appropriate and necessary to discuss why
specific auditors are failing to deliver on that.
There is no middle ground, and this is not about wishlists. This is about
fundamentally not meeting base level expectations.
>
> I understood from my EU colleagues that Ryan and Wayne had undertaken to
> produce a "wish list" covering requirements that they had on audits. We
> can then we can then discuss this with the European stakeholders and see
> how we could best answer the wish list. This wish list would be most
> helpful if it builds on the measures already proposed in TS 119 403-2 and
> its parent standards which provide specific requirements on all European
> audits for PTC. I understand also that we undertook to meet with WebTrust
> in December to get an understand of each other schemes which could lead to
> resolution of any alignment issues.
should and must improve overall. But with regards to the current
requirements and auditors such as TUVIT failing to appropriately apply
them, that’s an issue that needs discussion and resolution now, and in
public. I am glad the ESI TC recognizes there is room for improvement, just
as there is room for improvement with WebTrust, but it is inaccurate to
conflate that room for improvement with current failures in the
application. This is not about not having things that are wanted - this is
about not having the basics that are already required.
Nick Pope
Nov 9, 2018, 10:05:38 AM11/9/18
to mozilla-dev-s...@lists.mozilla.org
I am asking that we get a clear statement of what you would like to see from EU audits based on ETSI standards and so that we (European Auditors and ETSI) can come back with a considered response on how we can meet you concerns. Rather than saying what a particular individual person thinks, we would like to understand what your concerns are in as much detail as possible against what is specified as the current requirements for EU audits. We can then make a considered joint response to your concerns to ensure that ETSI audits meet your needs in a way works for the existing European environment.
I note your concerns about transparency and ensuring that the requirements certificate profile are met. If you can put these concerns down in detail, along with any other issue you have, as a joint document from the root stores, we can provide a coordinated response on how we can address your concerns.
If you see this as "basics that are already required" rather than "wish list" fine, again just provide us with a clear set requirements so that we can properly respond.
Nick
I note your concerns about transparency and ensuring that the requirements certificate profile are met. If you can put these concerns down in detail, along with any other issue you have, as a joint document from the root stores, we can provide a coordinated response on how we can address your concerns.
If you see this as "basics that are already required" rather than "wish list" fine, again just provide us with a clear set requirements so that we can properly respond.
Nick
Ryan Sleevi
Nov 9, 2018, 4:18:40 PM11/9/18
to Nick Pope, mozilla-dev-s...@lists.mozilla.org
On Fri, Nov 9, 2018 at 7:05 AM Nick Pope via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> I am asking that we get a clear statement of what you would like to see
> from EU audits based on ETSI standards and so that we (European Auditors
> and ETSI) can come back with a considered response on how we can meet you
> concerns. Rather than saying what a particular individual person thinks,
> we would like to understand what your concerns are in as much detail as
> possible against what is specified as the current requirements for EU
> audits. We can then make a considered joint response to your concerns to
> ensure that ETSI audits meet your needs in a way works for the existing
> European environment.
>
> I note your concerns about transparency and ensuring that the requirements
> certificate profile are met. If you can put these concerns down in detail,
> along with any other issue you have, as a joint document from the root
> stores, we can provide a coordinated response on how we can address your
> concerns.
>
> If you see this as "basics that are already required" rather than "wish
> list" fine, again just provide us with a clear set requirements so that we
> can properly respond.
I really don’t see how this is a productive response. It really is rather
dev-secur...@lists.mozilla.org> wrote:
> I am asking that we get a clear statement of what you would like to see
> from EU audits based on ETSI standards and so that we (European Auditors
> and ETSI) can come back with a considered response on how we can meet you
> concerns. Rather than saying what a particular individual person thinks,
> we would like to understand what your concerns are in as much detail as
> possible against what is specified as the current requirements for EU
> audits. We can then make a considered joint response to your concerns to
> ensure that ETSI audits meet your needs in a way works for the existing
> European environment.
>
> I note your concerns about transparency and ensuring that the requirements
> certificate profile are met. If you can put these concerns down in detail,
> along with any other issue you have, as a joint document from the root
> stores, we can provide a coordinated response on how we can address your
> concerns.
>
> If you see this as "basics that are already required" rather than "wish
> list" fine, again just provide us with a clear set requirements so that we
> can properly respond.
simple - do you believe auditors should be assessing compliance with EN 319
412-* under the existing standards?
If yes, TUVIT has demonstrated a pattern of failing to do so, and it’s
appropriate to discuss what next steps are appropriate to minimize the risk
from such repeated failures - such as no longer accepting.
If not, then ETSI audits are quite literally missing one of the most basic
expectations, and their acceptance should be immediately stopped until such
a time as they do.
I fail to see how there’s any other possible response there; it really is
cut and dry like that.
Nick Pope
Nov 12, 2018, 11:00:14 AM11/12/18
to mozilla-dev-s...@lists.mozilla.org
Ryan,
The difference in opinion seems to be which approach is most productive. Targeting particular concerns at an individual auditor or clearly stating all your concerns on European based audits for PTC so that we can come back come back with a common decision how, through ETSI standards and improvements in auditor practices, we can address all those concerns for all EU audits.
You don't seem to be happy that you are achieving what you want with the current direction and so I suggest the most productive is to adopt an alternative approach as I suggest.
Nick
The difference in opinion seems to be which approach is most productive. Targeting particular concerns at an individual auditor or clearly stating all your concerns on European based audits for PTC so that we can come back come back with a common decision how, through ETSI standards and improvements in auditor practices, we can address all those concerns for all EU audits.
You don't seem to be happy that you are achieving what you want with the current direction and so I suggest the most productive is to adopt an alternative approach as I suggest.
Nick
Nick Pope
Nov 12, 2018, 11:00:54 AM11/12/18
to mozilla-dev-s...@lists.mozilla.org
Ryan,
I see the main question is what is the most productive way ahead. We can continue discussing a specific concern in the context of just 1 of the European auditor, or work in the EU on a considered approach to all the concerns which can be applied to all European based audits. The first does not seem to be working towards something that you are happy with and even then would only provide an answer in a limited context. With the second approach we can take into account all your concerns and work towards an approach that can be applied to all EU audits which is acceptable to all.
Nick
On Friday, November 9, 2018 at 9:18:40 PM UTC, Ryan Sleevi wrote:
Ryan Sleevi
Nov 12, 2018, 11:11:56 AM11/12/18
to Nick Pope, mozilla-dev-security-policy
Nick,
I find your continued suggestions to be actively harmful - to the
discussion, for sure, but also to the reputation of ETSI.
You've attempted to frame this, again, as an either/or approach - that is,
that we can only have one of these discussions. You've attempted to
"thread-jack" the conversation by suggesting that we ignore specific
failures of a specific auditor, repeatedly, by instead suggesting that
through closed-door negotiations and smokey-room summits (notably, in which
browsers will be absent) will somehow resolve the issues. That's difficult
to believe, and even more difficult to stomach, given the attempt to
deflect any responsibility or accountability in favor of some abstract
'process'.
That's not to dismiss there being value in improving ETSI. Certainly, if
ETSI is to provide any value to the Web Ecosystem going forward, it needs
to address those needs. There's nothing inherently valuable in the ETSI
audits that makes them immune from concern or rejection.
However, this thread is about specific failures of a specific auditor. If
you do not believe these are failures - that is, you do not believe the
ETSI EN 319 * series has any normative guidance on CABs with respect to
assessing compliance with the stated certificate profiles - then we should
reject ETSI for the time being. If you do agree, however, that there is
specific guidance throughout those series regarding the expectations of
CABs, and that there is a pattern of failing to examine or adhere to that,
then I hope you can see and agree on the critical necessity of why the CAB
is failing.
We still have yet to receive a meaningful post-mortem from TUVIT regarding
this failure, nor of any acknowledgement of the pattern, as demonstrated by
past CAs they have audited, in which they failed to detect or account for
material non-conformities. That silence and lack of a meaningful response -
as to what practices are applied in the audit, why they failed, and what
can be done to improve - is exactly why it's reasonable to discuss
rejection of their future audit statements.
Suggesting that taking this up with ETSI will resolve this is akin to
suggesting that the CA/Browser Forum should be consulted every time there's
material misissuance. That misunderstands the ecosystem, misunderstands the
purpose, and misunderstands how to appropriately protect users.
There needs to be a resolution, to this thread. If you would like to
continue suggesting improvements to ETSI, which while I agree with, do not
believe this is at all an appropriate time, I would request you create a
new thread to share your thoughts. They are not, despite any possible
intent, productive for this conversation.
I find your continued suggestions to be actively harmful - to the
discussion, for sure, but also to the reputation of ETSI.
You've attempted to frame this, again, as an either/or approach - that is,
that we can only have one of these discussions. You've attempted to
"thread-jack" the conversation by suggesting that we ignore specific
failures of a specific auditor, repeatedly, by instead suggesting that
through closed-door negotiations and smokey-room summits (notably, in which
browsers will be absent) will somehow resolve the issues. That's difficult
to believe, and even more difficult to stomach, given the attempt to
deflect any responsibility or accountability in favor of some abstract
'process'.
That's not to dismiss there being value in improving ETSI. Certainly, if
ETSI is to provide any value to the Web Ecosystem going forward, it needs
to address those needs. There's nothing inherently valuable in the ETSI
audits that makes them immune from concern or rejection.
However, this thread is about specific failures of a specific auditor. If
you do not believe these are failures - that is, you do not believe the
ETSI EN 319 * series has any normative guidance on CABs with respect to
assessing compliance with the stated certificate profiles - then we should
reject ETSI for the time being. If you do agree, however, that there is
specific guidance throughout those series regarding the expectations of
CABs, and that there is a pattern of failing to examine or adhere to that,
then I hope you can see and agree on the critical necessity of why the CAB
is failing.
We still have yet to receive a meaningful post-mortem from TUVIT regarding
this failure, nor of any acknowledgement of the pattern, as demonstrated by
past CAs they have audited, in which they failed to detect or account for
material non-conformities. That silence and lack of a meaningful response -
as to what practices are applied in the audit, why they failed, and what
can be done to improve - is exactly why it's reasonable to discuss
rejection of their future audit statements.
Suggesting that taking this up with ETSI will resolve this is akin to
suggesting that the CA/Browser Forum should be consulted every time there's
material misissuance. That misunderstands the ecosystem, misunderstands the
purpose, and misunderstands how to appropriately protect users.
There needs to be a resolution, to this thread. If you would like to
continue suggesting improvements to ETSI, which while I agree with, do not
believe this is at all an appropriate time, I would request you create a
new thread to share your thoughts. They are not, despite any possible
intent, productive for this conversation.
Jakob Bohm
Nov 12, 2018, 8:11:15 PM11/12/18
to mozilla-dev-s...@lists.mozilla.org
Ryan,
Could you please provide, in a single message, a list of all the
supposedly multiple failures by TUVIT, clearly marking each if it is:
Subject O: [Other] A failure outside the specific subjects below.
Subject D: [Discussion] A failure by TUVIT to satisfactorily answer your
questions regarding the subjects below.
Subject S: [Standards] Disagreement with TUVIT about the interpretations
of the various audit standards and whether or not TUVIT can unilaterally
do things without a change in standards.
Subject T: [T-Systems QC-statments] The concrete actions and audits done
by TUVIT in relation to the QC-statement misencoding at T-Systems.
Matthias,
Please use standard e-mail formatting with all quoted text prefixed by
"> " (greater than sign and space) in future messages, as the
indentation based formatting used in your previous messages gets lost in
transit, making it difficult to tell what sentences are replies to what
prior messages. If you are using a Microsoft client, there may be an
advanced configuration option to do so.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Could you please provide, in a single message, a list of all the
supposedly multiple failures by TUVIT, clearly marking each if it is:
Subject O: [Other] A failure outside the specific subjects below.
Subject D: [Discussion] A failure by TUVIT to satisfactorily answer your
questions regarding the subjects below.
Subject S: [Standards] Disagreement with TUVIT about the interpretations
of the various audit standards and whether or not TUVIT can unilaterally
do things without a change in standards.
Subject T: [T-Systems QC-statments] The concrete actions and audits done
by TUVIT in relation to the QC-statement misencoding at T-Systems.
Matthias,
Please use standard e-mail formatting with all quoted text prefixed by
"> " (greater than sign and space) in future messages, as the
indentation based formatting used in your previous messages gets lost in
transit, making it difficult to tell what sentences are replies to what
prior messages. If you are using a Microsoft client, there may be an
advanced configuration option to do so.
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
Ryan Sleevi
Nov 12, 2018, 10:08:45 PM11/12/18
to Jakob Bohm, mozilla-dev-security-policy
Jakob,
Please see
https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/lpwKQXOfAgAJ
, which was already provided previously.
It includes details regarding T-Systems areas of non-compliance that were
1) Demonstrably not identified by the auditor
2) Covered by existing audit criteria
3) Sharing the similar root cause as this incident
Even if we accept a notion that an auditor would not have been looking for
those issues at that time (despite the clear auditable criteria that
existed), the examination of root cause reveals a common pattern shared
with this incident, and a pattern where the auditor would have been
responsible for the review of the changes as part of the certification
scheme. T-Systems has still not provided a satisfactory response to the
questions raised by Gerv and Wayne in response to the past incident (
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074 ), which, while
separable from the concerns of TUVIT, should have factored into any such
considerations - such as Gerv's prescient expectation of exactly this issue
in https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c22
Please see
https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/lpwKQXOfAgAJ
, which was already provided previously.
It includes details regarding T-Systems areas of non-compliance that were
1) Demonstrably not identified by the auditor
2) Covered by existing audit criteria
3) Sharing the similar root cause as this incident
Even if we accept a notion that an auditor would not have been looking for
those issues at that time (despite the clear auditable criteria that
existed), the examination of root cause reveals a common pattern shared
with this incident, and a pattern where the auditor would have been
responsible for the review of the changes as part of the certification
scheme. T-Systems has still not provided a satisfactory response to the
questions raised by Gerv and Wayne in response to the past incident (
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074 ), which, while
separable from the concerns of TUVIT, should have factored into any such
considerations - such as Gerv's prescient expectation of exactly this issue
in https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c22
things things
Nov 13, 2018, 5:30:37 AM11/13/18
to dev-secur...@lists.mozilla.org
Ryan,
I feel you are trying to derail the discussion and are muddying the waters.
I hope you can see that this is actively damaging the community by promoting magniloquent indictments instead of discussing clear facts. It would be far more productive to provide a concrete and structured list of TUVITs failings, as suggested by Jakob. Up to now, there is no readable summary of facts to understand what this all is about. In your initial posting you wordily talked about an Issue A, then Issue C, and then you skipped that entirely.
I trust in Mozilla to not only hear the rethorics of individuals, but to maintain a balanced view of facts. Fine and excellent examples for such an approach can be found in the handling of Visa (https://wiki.mozilla.org/CA:Visa_Issues), Symantec (https://wiki.mozilla.org/CA:Symantec_Issues) and WoSign (https://wiki.mozilla.org/CA:WoSign_Issues).
I feel you are trying to derail the discussion and are muddying the waters.
I hope you can see that this is actively damaging the community by promoting magniloquent indictments instead of discussing clear facts. It would be far more productive to provide a concrete and structured list of TUVITs failings, as suggested by Jakob. Up to now, there is no readable summary of facts to understand what this all is about. In your initial posting you wordily talked about an Issue A, then Issue C, and then you skipped that entirely.
I trust in Mozilla to not only hear the rethorics of individuals, but to maintain a balanced view of facts. Fine and excellent examples for such an approach can be found in the handling of Visa (https://wiki.mozilla.org/CA:Visa_Issues), Symantec (https://wiki.mozilla.org/CA:Symantec_Issues) and WoSign (https://wiki.mozilla.org/CA:WoSign_Issues).
Jakob Bohm
Nov 13, 2018, 5:56:34 AM11/13/18
to mozilla-dev-s...@lists.mozilla.org
On 13/11/2018 04:08, Ryan Sleevi wrote:
> Jakob,
>
In the following, I have added a new subject category:
Subject U: [T-Systems local] Issues at T-Systems, rather than issues
in TUVIT's auditing of T-Systems.
I will use the following number:
U1: T-Systems misencoded the qc-statement extension required by the
EU scheme, but not present in X.509, BR nor WebPKI except for the
general requirement not to misencode anything.
(Subject T is TUVIT's auditing of issue U1).
> Please see
> https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/lpwKQXOfAgAJ
> , which was already provided previously.
> It includes details regarding T-Systems areas of non-compliance that were
> 1) Demonstrably not identified by the auditor
> 2) Covered by existing audit criteria
> 3) Sharing the similar root cause as this incident
>
So your additional complaints are:
O1 TUVIT did not list the issues from bug #1391074 (see below)
in the audit reports for 2017, and maybe didn't discover them,
neither by management assertion, nor by TUVIT searching
Mozilla systems for mention of T-Systems issues, nor by the
various concrete audit steps (such as sampling).
T2 You believe TUVIT should have done extra checks based on
their knowledge (if any!) of the issues in bug #1391074, and
that those extra checks might have increased the chance of
TUVIT discovering issue U1.
T3 You believe that there is a common root cause at T-Systems
for the issues in bug #1391074 and issue U1. The one
commonality I could dig out is that in bug #1391074 something
in the PKI system (not the templates apparently), had
incorrect ASN.1 definitions of two standard X.509 objects,
while in the qc-statements bug, something in the system (not
sure what), had an incorrect ASN.1 definition of an object
not in any part of X.509, PKIX or the BRs .
The message you linked above (Which seems to be message id
<mailman.103.1541172329....@lists.mozilla.org>) is
a random point in your ongoing debate, and seems to be full of arguments
rather than a clear, classified enumeration of issues.
It does not state clearly, except by a nameless numerical link to a past
bug, what previous incidents are being considered. I have tried to
summarize that bug below.
The referenced bug #1391074 seems to mention the following issues at
T-Systems (U1 is the qc-statements encoding issue itself, not its
auditing):
U2. A failure by T-Systems (not TUVIT) to respond within 24 hours to an
issue described only by another link to a discussion, specifically:
https://groups.google.com/d/msg/mozilla.dev.security.policy/PrsDfS8AMEk/w2AMK81jAQAJ
Unfortunately, the bug log did not capture what the problem reporting
addresses in CCADB were at the time, or which of those was used.
U3. One or two cases of T-Systems (not TUVIT) issuing certificates with
syntactically invalid dnsNames described only by two other such links:
https://groups.google.com/d/msg/mozilla.dev.security.policy/CfyeeybBz9c/lmmUT4x2CAAJ
and
https://groups.google.com/d/msg/mozilla.dev.security.policy/D0poUHqiYMw/Pf5p0kB7CAAJ
U4. Some instances of T-Systems (not TUVIT) issuing certificates with
minor syntax errors in the distinguished name (described, confusingly,
as metadata-only field values), again detailed only in the form of a
discussion link:
https://groups.google.com/d/msg/mozilla.dev.security.policy/Sae5lpT02Ng/-lsC11JnBwAJ
According to later replies in the bug discussion, this was apparently
a case of some kind of nonsense (such as empty string) in the OU field,
and some cases of empty ST field.
U5. Some instances of T-Systems (not TUVIT) issuing certificates with
double dots (a clear syntax error) in dnsNames, described by a
discussion link and a list of certificates:
https://groups.google.com/d/msg/mozilla.dev.security.policy/Sae5lpT02Ng/-lsC11JnBwAJ
online certificate list
https://misissued.com/batch/13/
U6. A list of cablint results not clearly classified as to their
relationships with prior mentions in the bug log.
U7. 3 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with "Key Encipherment" enabled
for ECDSA keys due to bugs and configuration errors.
U8. 1 instance, found by T-Systems themselves of T-Systems (not TUVIT)
issuing a certificate with no SAN extension.
U9. 10 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with IP addresses in the dnsName
SAN fields instead of the IP-address SAN fields (a clear
violation). This was apparently deliberate but with no attempt
to obtain a dispensation from the BRs and Mozilla policy.
U10. 4 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with non-existant country codes.
This is clearly a mis-issuance, as it gives an untrue identity
of the certificate holder.
U11. 1 instance, found by T-Systems themselves of T-Systems (not
TUVIT) issuing a certificate with an O field but neither L nor
ST field. A clear BR violation.
U12. Inadequate answers by T-Systems (not TUVIT) about root
causes and preventative measures.
S2. Discussion between Mozilla and T-Systems (not TUVIT) about
how to handle conflicts between a legally mandatory privacy
law at the time and a wish for full certificate details to
be published.
> Even if we accept a notion that an auditor would not have been looking for
> those issues at that time (despite the clear auditable criteria that
> existed), the examination of root cause reveals a common pattern shared
> with this incident,
You have not stated what you think that common pattern is.
Is it general sloppiness?
Is it ASN.1 encoding errors?
> and a pattern where the auditor would have been
> responsible for the review of the changes as part of the certification> scheme.
What changes specifically?
What part of the certification scheme specifically?
> T-Systems has still not provided a satisfactory response to the
> questions raised by Gerv and Wayne in response to the past incident (
> https://bugzilla.mozilla.org/show_bug.cgi?id=1391074 ), which, while
Do you mean T-Systems has not provided a sufficiently detailed root
cause analysis of the issues in bug#1391074?
Do you mean T-Systems has not provided a clear statement of specific
corrections for the root causes of the issues in bug#1391074 ?
> separable from the concerns of TUVIT, should have factored into any such
> considerations - such as Gerv's prescient expectation of exactly this issue
> in https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c22
>
Comment 22 in Bug #1391074 merely asks for more information, including
clear statements of measures to prevent similar errors in the future.
It also mentions bad templates as an example.
Comment 28 answers most (not all) of comment #22, although briefly. In
particular it explains that the one issues that caused Gerv to ask about
templates was not a bad template, but a software bug that used a
different template than the one configured.
Thus if issue U1 was in a template, it would likely not have been caught
by measures based on the identified root causes of the issues in
bug#1391074.
> Jakob,
>
In the following, I have added a new subject category:
Subject U: [T-Systems local] Issues at T-Systems, rather than issues
in TUVIT's auditing of T-Systems.
I will use the following number:
U1: T-Systems misencoded the qc-statement extension required by the
EU scheme, but not present in X.509, BR nor WebPKI except for the
general requirement not to misencode anything.
(Subject T is TUVIT's auditing of issue U1).
> Please see
> https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/lpwKQXOfAgAJ
> , which was already provided previously.
> It includes details regarding T-Systems areas of non-compliance that were
> 1) Demonstrably not identified by the auditor
> 2) Covered by existing audit criteria
> 3) Sharing the similar root cause as this incident
>
O1 TUVIT did not list the issues from bug #1391074 (see below)
in the audit reports for 2017, and maybe didn't discover them,
neither by management assertion, nor by TUVIT searching
Mozilla systems for mention of T-Systems issues, nor by the
various concrete audit steps (such as sampling).
T2 You believe TUVIT should have done extra checks based on
their knowledge (if any!) of the issues in bug #1391074, and
that those extra checks might have increased the chance of
TUVIT discovering issue U1.
T3 You believe that there is a common root cause at T-Systems
for the issues in bug #1391074 and issue U1. The one
commonality I could dig out is that in bug #1391074 something
in the PKI system (not the templates apparently), had
incorrect ASN.1 definitions of two standard X.509 objects,
while in the qc-statements bug, something in the system (not
sure what), had an incorrect ASN.1 definition of an object
not in any part of X.509, PKIX or the BRs .
The message you linked above (Which seems to be message id
<mailman.103.1541172329....@lists.mozilla.org>) is
a random point in your ongoing debate, and seems to be full of arguments
rather than a clear, classified enumeration of issues.
It does not state clearly, except by a nameless numerical link to a past
bug, what previous incidents are being considered. I have tried to
summarize that bug below.
The referenced bug #1391074 seems to mention the following issues at
T-Systems (U1 is the qc-statements encoding issue itself, not its
auditing):
U2. A failure by T-Systems (not TUVIT) to respond within 24 hours to an
issue described only by another link to a discussion, specifically:
https://groups.google.com/d/msg/mozilla.dev.security.policy/PrsDfS8AMEk/w2AMK81jAQAJ
Unfortunately, the bug log did not capture what the problem reporting
addresses in CCADB were at the time, or which of those was used.
U3. One or two cases of T-Systems (not TUVIT) issuing certificates with
syntactically invalid dnsNames described only by two other such links:
https://groups.google.com/d/msg/mozilla.dev.security.policy/CfyeeybBz9c/lmmUT4x2CAAJ
and
https://groups.google.com/d/msg/mozilla.dev.security.policy/D0poUHqiYMw/Pf5p0kB7CAAJ
U4. Some instances of T-Systems (not TUVIT) issuing certificates with
minor syntax errors in the distinguished name (described, confusingly,
as metadata-only field values), again detailed only in the form of a
discussion link:
https://groups.google.com/d/msg/mozilla.dev.security.policy/Sae5lpT02Ng/-lsC11JnBwAJ
According to later replies in the bug discussion, this was apparently
a case of some kind of nonsense (such as empty string) in the OU field,
and some cases of empty ST field.
U5. Some instances of T-Systems (not TUVIT) issuing certificates with
double dots (a clear syntax error) in dnsNames, described by a
discussion link and a list of certificates:
https://groups.google.com/d/msg/mozilla.dev.security.policy/Sae5lpT02Ng/-lsC11JnBwAJ
online certificate list
https://misissued.com/batch/13/
U6. A list of cablint results not clearly classified as to their
relationships with prior mentions in the bug log.
U7. 3 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with "Key Encipherment" enabled
for ECDSA keys due to bugs and configuration errors.
U8. 1 instance, found by T-Systems themselves of T-Systems (not TUVIT)
issuing a certificate with no SAN extension.
U9. 10 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with IP addresses in the dnsName
SAN fields instead of the IP-address SAN fields (a clear
violation). This was apparently deliberate but with no attempt
to obtain a dispensation from the BRs and Mozilla policy.
U10. 4 instances, found by T-Systems themselves of T-Systems (not
TUVIT) issuing certificates with non-existant country codes.
This is clearly a mis-issuance, as it gives an untrue identity
of the certificate holder.
U11. 1 instance, found by T-Systems themselves of T-Systems (not
TUVIT) issuing a certificate with an O field but neither L nor
ST field. A clear BR violation.
U12. Inadequate answers by T-Systems (not TUVIT) about root
causes and preventative measures.
S2. Discussion between Mozilla and T-Systems (not TUVIT) about
how to handle conflicts between a legally mandatory privacy
law at the time and a wish for full certificate details to
be published.
> Even if we accept a notion that an auditor would not have been looking for
> those issues at that time (despite the clear auditable criteria that
> existed), the examination of root cause reveals a common pattern shared
> with this incident,
Is it general sloppiness?
Is it ASN.1 encoding errors?
> and a pattern where the auditor would have been
> responsible for the review of the changes as part of the certification> scheme.
What part of the certification scheme specifically?
> T-Systems has still not provided a satisfactory response to the
> questions raised by Gerv and Wayne in response to the past incident (
> https://bugzilla.mozilla.org/show_bug.cgi?id=1391074 ), which, while
cause analysis of the issues in bug#1391074?
Do you mean T-Systems has not provided a clear statement of specific
corrections for the root causes of the issues in bug#1391074 ?
> separable from the concerns of TUVIT, should have factored into any such
> considerations - such as Gerv's prescient expectation of exactly this issue
> in https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c22
>
clear statements of measures to prevent similar errors in the future.
It also mentions bad templates as an example.
Comment 28 answers most (not all) of comment #22, although briefly. In
particular it explains that the one issues that caused Gerv to ask about
templates was not a bad template, but a software bug that used a
different template than the one configured.
Thus if issue U1 was in a template, it would likely not have been caught
by measures based on the identified root causes of the issues in
bug#1391074.
Ryan Sleevi
Nov 13, 2018, 7:51:28 AM11/13/18
to things things, dev-secur...@lists.mozilla.org
On Tue, Nov 13, 2018 at 5:30 AM things things via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> Ryan,
>
> I feel you are trying to derail the discussion and are muddying the waters.
>
> I hope you can see that this is actively damaging the community by
> promoting magniloquent indictments instead of discussing clear facts. It
> would be far more productive to provide a concrete and structured list of
> TUVITs failings, as suggested by Jakob.
Do you believe the initial message did not contain that?
dev-secur...@lists.mozilla.org> wrote:
> Ryan,
>
> I feel you are trying to derail the discussion and are muddying the waters.
>
> I hope you can see that this is actively damaging the community by
> promoting magniloquent indictments instead of discussing clear facts. It
> would be far more productive to provide a concrete and structured list of
> TUVITs failings, as suggested by Jakob.
Up to now, there is no readable summary of facts to understand what this
> all is about. In your initial posting you wordily talked about an Issue A,
> then Issue C, and then you skipped that entirely.
similar concerns expressed by Wayne in the subsequent response. Would you
like to discuss it further? Otherwise, it’s unclear your point.
>
Ryan Sleevi
Nov 13, 2018, 8:31:36 AM11/13/18
to Jakob Bohm, mozilla-dev-s...@lists.mozilla.org
I suppose I had unreasonably hoped it would be self-evident, particularly
for someone who claims to follow the issues, to understand how directly
that issue was related. Unfortunately, whether for intent or otherwise, it
appears not.
While I do not believe nor agree with your approach to framing the issues,
I do hope you can agree that both through the bug - which itself is an
amalgamation of and reference to several bugs - that during the prior two
audit cycles, T-Systems contained a substantial amount of misissuance that
were undetected by TUVIT and that shared the same root cause:
misconfiguration and misapplication of the relevant rules, both in terms of
ASN.1 and in terms of normative requirement.
If you are attempting to excuse such misissuance, rather than address it,
one would take a similar tact as you are here; suggesting, for example,
that it was T-Systems rather than TUVIT that did the misissuance or by
suggesting that the incidence was low to be insignificant. I was careful
not to try to muddy the conversation through an indictment on T-Systems, to
avoid diluting the conversation, and because they’ve already provided
several enumerations of the issues and that doing so again, as you’ve done,
does not add value. However, it should be readily apparent from both the
bug discussion and the list of issues a common pattern of misconfiguring
relevant profiles and failing to ensure they comply with the relevant
requirements.
In the context of ETSI, each of these configuration changes - particularly
once qualified - undergoes some review; whether after the fact
(pre-qualification) or prior to such change. Similarly, misissuance
involves a degree of notification to the CAB. As such, it is entirely
reasonable to expect a degree of supervision, as that is the value of the
certification scheme. All of this information would have been available at
the time of configuring qualified certificates, including the pattern of
issues existing when configuring profiles and templates.
As such, we functionally see two issues; the inadequate supervision that
resulted in the first batch of misissuance, which can be attempted to be
argued away by suggesting it was some small volume that sampling would not
have caught (despite the inconsistencies of that argument with the
criteria), and inadequate supervision leading to this current issue,
despite having all of that previous information available as context during
the review and despite their being 100% misissuance rate. Both of these
share a clear commonality of inadequate supervision, a key role played over
the past several years.
Audits understandably and obviously do not prevent a CA from making a
change tomorrow that undermines the past audits; there is no guarantee they
won’t start actively misissuing once the auditor has left the building. It
is, however, meant to provide assurance regarding the present (and past)
configuration. When a CA like T-Systems does misissue, whether this or
previous incidents, it is entirely reasonable to ask “Was this
configuration something the auditor previously reviewed, and did they catch
it?” and, in the case of ETSI, “was this a change the auditor approved in
relation to ongoing certification?”
The qcStatements demonstrates a failure of the latter, the bug demonstrates
a failure of the former, both speak to the process of review and the
qualifications of the reviewer.
If you don’t agree with the large swath of undetected past misissuance
being a concern, it would be helpful if you could explain why it isn’t
concerning. For example, do you believe that these requirements
(collectively, for any of these issues) were not covered by existing
criteria? Do you believe that sufficient documentation of TUVIT’s
methodology exists so as to explain why such failure to detect may be seen
as reasonable? Do you believe that ETSI does not require consideration by
auditors prior to operational and configuration changes? In short, do you
disagree that, when presented with CA misissuance, such as by T-Systems,
that it is both relevant and appropriate to question why the auditor failed
to detect and/or prevent such misissuance?
I am not arguing that an audit be a guarantee against misissuance; for
example, a statistical sample will be just that, a sample, and stuff can
reasonably slip through. I am, however, advocating that it’s both
appropriate and necessary to question whether sampling was even done, and
how it was constructed (e.g. CA selects the samples and sizes vs auditor),
and what was reviewed, in order to ascertain whether or not it was
“reasonable” to have missed something. In the case of T-Systems past
misissuances, the collective sum - especially with respect to things like
misconfigured templates - raises legitimate concerns about TUVITs approach
and methodology, and those concerns are each themselves distinct issues
with TUVIT for every misissuance “type” by T-Systems.
for someone who claims to follow the issues, to understand how directly
that issue was related. Unfortunately, whether for intent or otherwise, it
appears not.
While I do not believe nor agree with your approach to framing the issues,
I do hope you can agree that both through the bug - which itself is an
amalgamation of and reference to several bugs - that during the prior two
audit cycles, T-Systems contained a substantial amount of misissuance that
were undetected by TUVIT and that shared the same root cause:
misconfiguration and misapplication of the relevant rules, both in terms of
ASN.1 and in terms of normative requirement.
If you are attempting to excuse such misissuance, rather than address it,
one would take a similar tact as you are here; suggesting, for example,
that it was T-Systems rather than TUVIT that did the misissuance or by
suggesting that the incidence was low to be insignificant. I was careful
not to try to muddy the conversation through an indictment on T-Systems, to
avoid diluting the conversation, and because they’ve already provided
several enumerations of the issues and that doing so again, as you’ve done,
does not add value. However, it should be readily apparent from both the
bug discussion and the list of issues a common pattern of misconfiguring
relevant profiles and failing to ensure they comply with the relevant
requirements.
In the context of ETSI, each of these configuration changes - particularly
once qualified - undergoes some review; whether after the fact
(pre-qualification) or prior to such change. Similarly, misissuance
involves a degree of notification to the CAB. As such, it is entirely
reasonable to expect a degree of supervision, as that is the value of the
certification scheme. All of this information would have been available at
the time of configuring qualified certificates, including the pattern of
issues existing when configuring profiles and templates.
As such, we functionally see two issues; the inadequate supervision that
resulted in the first batch of misissuance, which can be attempted to be
argued away by suggesting it was some small volume that sampling would not
have caught (despite the inconsistencies of that argument with the
criteria), and inadequate supervision leading to this current issue,
despite having all of that previous information available as context during
the review and despite their being 100% misissuance rate. Both of these
share a clear commonality of inadequate supervision, a key role played over
the past several years.
Audits understandably and obviously do not prevent a CA from making a
change tomorrow that undermines the past audits; there is no guarantee they
won’t start actively misissuing once the auditor has left the building. It
is, however, meant to provide assurance regarding the present (and past)
configuration. When a CA like T-Systems does misissue, whether this or
previous incidents, it is entirely reasonable to ask “Was this
configuration something the auditor previously reviewed, and did they catch
it?” and, in the case of ETSI, “was this a change the auditor approved in
relation to ongoing certification?”
The qcStatements demonstrates a failure of the latter, the bug demonstrates
a failure of the former, both speak to the process of review and the
qualifications of the reviewer.
If you don’t agree with the large swath of undetected past misissuance
being a concern, it would be helpful if you could explain why it isn’t
concerning. For example, do you believe that these requirements
(collectively, for any of these issues) were not covered by existing
criteria? Do you believe that sufficient documentation of TUVIT’s
methodology exists so as to explain why such failure to detect may be seen
as reasonable? Do you believe that ETSI does not require consideration by
auditors prior to operational and configuration changes? In short, do you
disagree that, when presented with CA misissuance, such as by T-Systems,
that it is both relevant and appropriate to question why the auditor failed
to detect and/or prevent such misissuance?
I am not arguing that an audit be a guarantee against misissuance; for
example, a statistical sample will be just that, a sample, and stuff can
reasonably slip through. I am, however, advocating that it’s both
appropriate and necessary to question whether sampling was even done, and
how it was constructed (e.g. CA selects the samples and sizes vs auditor),
and what was reviewed, in order to ascertain whether or not it was
“reasonable” to have missed something. In the case of T-Systems past
misissuances, the collective sum - especially with respect to things like
misconfigured templates - raises legitimate concerns about TUVITs approach
and methodology, and those concerns are each themselves distinct issues
with TUVIT for every misissuance “type” by T-Systems.
things things
Nov 13, 2018, 9:46:39 AM11/13/18
to ry...@sleevi.com, dev-secur...@lists.mozilla.org
>> I hope you can see that this is actively damaging the community by promoting magniloquent indictments instead of discussing
>> clear facts. It would be far more productive to provide a concrete and structured list of TUVITs failings, as suggested by Jakob.
>> clear facts. It would be far more productive to provide a concrete and structured list of TUVITs failings, as suggested by Jakob.
> Do you believe the initial message did not contain that?
Yes. Your inital message contained a lot of information, a timeline about contacting TUVIT, expressions of your dissatisfaction with TUVITs answers etc etc. It also contained two paragraphs labeled "Issue A" and "Issue C", but it is far from a concrete and structured list.
I don't think that it is currently transparent or its lost in the approx 50 message with partly heated exchanges about ETSI and whatnot that followed, what the core of the issues is.
Ryan Sleevi
Nov 13, 2018, 10:24:14 AM11/13/18
to things things, Ryan Sleevi, MDSP
On Tue, Nov 13, 2018 at 9:46 AM things things <thing...@outlook.com>
wrote:
I think, then, that we'll have to agree to disagree on both approach and
substance.
It would appear that your desire is for a small, bulleted list of items,
and to make your opinion solely based on that, without any context. The
initial thread started by both contextualizing a set of issues and, from
there, enumerating specific issues. The discussion, to date, has been to
review those facts, ensure they're accurate and meaningfully presented, and
allow opportunity for both other concerns to be raised and for other
considerations. This will be, inherently, a messy process, but is
fundamental to the essence of building a shared understanding. There have
been several attempts to derail the thread, including suggestions these
issues shouldn't be discussed before December (at the earliest) or possibly
into the next year, but those are fundamentally unproductive.
>From the 40 messages, we've converged on a set of things starting to be
understood and agreed upon, and other issues still being debated. It would
be both premature and unproductive to attempt to distill that into a curt
list while the discussion is ongoing, especially given that the
responsiveness of TUVIT to the concerns - and in particular, the lack of
any explanation of methodology that would explain why the concerns are
unfounded.
If you consider past discussions - such as CAs like StartCom or Symantec -
you'll see that they similarly followed an evolutionary approach, in which
an initial issue was reported, it spiraled into a broader discussion, and
the *output* of that discussion was a structured list.
This is why I disagree with you on substance and approach; I think it would
be premature to attempt to distill that into a list while the discussions
are ongoing, to the point of seeming to attempt to stifle conversation.
Indeed, most of the messages following
https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/T6W4i2XHAwAJ
have not been attempting to discuss the substance of the issues, or to
further explore, but instead suggest that it's not appropriate to have this
conversation, or to attempt to restructure the conversation. It seems like
far more productive conversations can be made on the substance, rather than
structure-policing.
wrote:
substance.
It would appear that your desire is for a small, bulleted list of items,
and to make your opinion solely based on that, without any context. The
initial thread started by both contextualizing a set of issues and, from
there, enumerating specific issues. The discussion, to date, has been to
review those facts, ensure they're accurate and meaningfully presented, and
allow opportunity for both other concerns to be raised and for other
considerations. This will be, inherently, a messy process, but is
fundamental to the essence of building a shared understanding. There have
been several attempts to derail the thread, including suggestions these
issues shouldn't be discussed before December (at the earliest) or possibly
into the next year, but those are fundamentally unproductive.
>From the 40 messages, we've converged on a set of things starting to be
understood and agreed upon, and other issues still being debated. It would
be both premature and unproductive to attempt to distill that into a curt
list while the discussion is ongoing, especially given that the
responsiveness of TUVIT to the concerns - and in particular, the lack of
any explanation of methodology that would explain why the concerns are
unfounded.
If you consider past discussions - such as CAs like StartCom or Symantec -
you'll see that they similarly followed an evolutionary approach, in which
an initial issue was reported, it spiraled into a broader discussion, and
the *output* of that discussion was a structured list.
This is why I disagree with you on substance and approach; I think it would
be premature to attempt to distill that into a list while the discussions
are ongoing, to the point of seeming to attempt to stifle conversation.
Indeed, most of the messages following
https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/T6W4i2XHAwAJ
have not been attempting to discuss the substance of the issues, or to
further explore, but instead suggest that it's not appropriate to have this
conversation, or to attempt to restructure the conversation. It seems like
far more productive conversations can be made on the substance, rather than
structure-policing.
Jakob Bohm
Nov 13, 2018, 11:26:28 AM11/13/18
to mozilla-dev-s...@lists.mozilla.org
Unfortunately, you seem to be be ignoring what I wrote and talking about
something else.
On 13/11/2018 14:31, Ryan Sleevi wrote:
> I suppose I had unreasonably hoped it would be self-evident, particularly
> for someone who claims to follow the issues, to understand how directly
> that issue was related. Unfortunately, whether for intent or otherwise, it
> appears not.
>
This discussion has been overfilled with discussions about the meanings
of words, weather or not something is under one rule or another etc.
etc., making it very hard to get a clear overview of the primary issues.
Furthermore the start of the thread was off-list. Also neither I, nor
some other participants have access to the audit reports etc. in CCADB.
This basic combination of noise and missing data is why I asked for a
one-stop overview of your complaints against TUVIT, similar to the lists
compiled for previous situations with multiple complaints against one
party.
> While I do not believe nor agree with your approach to framing the issues,
> I do hope you can agree that both through the bug - which itself is an
> amalgamation of and reference to several bugs - that during the prior two
> audit cycles, T-Systems contained a substantial amount of misissuance that
> were undetected by TUVIT and that shared the same root cause:
> misconfiguration and misapplication of the relevant rules, both in terms of
> ASN.1 and in terms of normative requirement.
>
"Misconfiguration and misapplication of the relevant rules..." is so
broad as to describe the majority of CA failures without giving any
useful specifics to assess the situation. It's like saying someone's
crime was to "violate and break the relevant laws" (which would apply to
anything from jaywalking to mass murder).
It would also be useful to quantify the word substantial: Of all the
certificates issued by the audited CA organization, how large a
percentage suffered from each flaw, how many from none. This is a key
number when assessing if statistical sampling by the auditor should have
caught an issue. It is also a key number when assessing the level of
incompetence of the CA (but the CA is not the subject of this thread).
Bug #1391074 contained counts of affected certificates, but not how
many certificates did not have the listed issues. So it is not clear if
those issues were likely to be spotted by 3% sampling.
T4. One valid complaint is that TUVIT apparently (according to what I
read in this thread) didn't check the mailing list and bugzilla to
find out about the Bug #1391074 issue and include them in a relevant
audit statement (as this was already public information, the
confidentiality excuse doesn't apply).
T5. Another valid complaint is that TUVIT apparently did not see the
corrective measures described in bug #1391074, which should have been
in the CA's technical audit logs.
T6. Then there is the valid complain that the administrative paper trail
related to Bug #1391074 should have resulted in at least some event
observed by TUVIT as part of their ETSI/ISO monitoring during the
audit period, and that this should have caused TUVIT to look for the
rest of the issue and include it in the audit or audit followup
reports.
Issue U1 (Qc-statement misencoding) apparently affected all certificates
from one issuing CA, and should thus have been caught by sampling by the
auditor. The auditor has (according to earlier posts) admitted that the
bug was present in the sampled certificates from that issuing CA, but
that this was overlooked because that particular extension was not one
they had specific experience looking at. Once the problem was pointed
out the auditor looked at the previously collected evidence and
confirmed the problem by checking that detail from first principles
(similar to software developer hand-executing a function with pen and
paper to confirm a bug).
T7. So this IS a valid complaint against TUVIT: That the auditor didn't
check the Qc-statement encoding from first principles during the
original audit, and thus failed to spot the problem.
T8. Then there is the issue that TUVIT didn't issue an official
qualification of the CA's status once the issue was both known and
public.
S9. There is also the issue if TUVIT should have issued an official
qualification of the CA's status if they knew before the U1 issue
became public. This is subject to the debate about the
confidentiality provisions in the ISO audit standard versus the
expectations of Mozilla and Chrome.
S10. Then there is the issue if TUVIT was right or wrong in accepting a
slow phase out of the certificates affected by U1. This involves both
the principal issue if there should be zero tolerance for incorrect
certificates, the practical issue of how much harm this specific
standards validation can cause, and how much time should be allowed
for an orderly replacement process. Multiple months seems a quite
long time. 1 day quite a short time.
> If you are attempting to excuse such misissuance, rather than address it,
> one would take a similar tact as you are here; suggesting, for example,
> that it was T-Systems rather than TUVIT that did the misissuance or by
> suggesting that the incidence was low to be insignificant.
I am not excusing the misissuance, merely trying to catalog it as part
of the evidence to assess the level of diligence or failure by TUVIT.
It is of cause the purpose of any audit scheme to check for the absence
of irregularities, and to report if any were found. But it is quite
rare for the audit to essentially redo every piece of administrative
work done by the audited company.
Thus there is often the question of why an irregularity (in this case
incorrect CA issuances) did not make it into the public audited
accounts. And thus if the auditor was (also) at fault, which is the
topic of this thread.
> I was careful
> not to try to muddy the conversation through an indictment on T-Systems, to
> avoid diluting the conversation, and because they’ve already provided
> several enumerations of the issues and that doing so again, as you’ve done,
> does not add value.
For assessment of TUVIT, this cataloging of known T-System mistakes
versus what was in the audit reports and audit statements is a key
measure of the quality of the auditing.
> However, it should be readily apparent from both the
> bug discussion and the list of issues a common pattern of misconfiguring
> relevant profiles and failing to ensure they comply with the relevant
> requirements.
Of cause. But it is not clear from the documents you linked to (the
discussion message and but #1391074) that all, or even most, of the
issues were specifically profile misconfigurations rather than a litany
of other mistakes. This in turn ties in to the question how many of
those mistakes should have been obvious to the auditors versus being
buried too deep to be spotted.
The debate in bug #1391074 about the template used for ECDSA
certificates is a good example. According to the bug, the ECDSA
certificate profile/template was correct, but some piece of software
mishandled approved ECDSA certificate requests and used the RSA
certificate profile/template, for at least some of the issued
certificates. An incorrect ECDSA profile/template saying to set the
KeyEncryption bit should have been spotted by a configuration audit and
review (by TUVIT). But code bugs are notoriously harder to spot.
>
> In the context of ETSI, each of these configuration changes - particularly
> once qualified - undergoes some review; whether after the fact
> (pre-qualification) or prior to such change.
This is why it is interesting to look at each issue to determine if it
was subject to such review by or notification to TUVIT.
> Similarly, misissuance
> involves a degree of notification to the CAB.
Only once known (e.g. around the time of the bug reports). Because I
don't think you expect a CA to notify the auditor that it is about to
misissue, and then proceed to actually misissue instead of stopping
itself.
> As such, it is entirely
> reasonable to expect a degree of supervision, as that is the value of the
> certification scheme. All of this information would have been available at
> the time of configuring qualified certificates, including the pattern of
> issues existing when configuring profiles and templates.
Should have been available doesn't mean it was available. There is
always a limit to the depth of audits, and thus we need to assess if
TUVIT was being sloppy, complacent, complicit or just unlucky.
>
> As such, we functionally see two issues; the inadequate supervision that
> resulted in the first batch of misissuance, which can be attempted to be
> argued away by suggesting it was some small volume that sampling would not
> have caught (despite the inconsistencies of that argument with the
> criteria),
Due to the lack of a consistent list of issues, I have not seen specific
statements for each of the Bug #1391074 issues if that one should have
been spotted by the auditor before it was found by the community, and
why. For example were the 3 incorrect ECDSA certificates 100%, 1% or
0.01% of the issued certificates from that particular issuing CA? And
how many of the certificates from that issuing CA should have been
sampled under the criteria?
This is of cause in addition to the question if or why not TUVIT was
subsequently fully aware of the bug report and the reports uploaded to
bugzilla by the CA.
> and inadequate supervision leading to this current issue,
> despite having all of that previous information available as context during
> the review and despite their being 100% misissuance rate.
For the current issue (U1), there is also a major question of how hard or easy to spot that particular that encoding error was if not looking
specifically for that particular error. This applies both when
"supervising" the creation of whatever configuration, template or code
contained the bug and when auditing samples of issued certificates.
One could always take the extreme position that a supervisor has no
responsibility for the acts of the supervised, or the equally extreme
position that a supervisor is fully responsible for every act of the
supervised.
> Both of these
> share a clear commonality of inadequate supervision, a key role played over
> the past several years.
Only if you define away all specifics and reduce the words to
meaninglessness.
>
> Audits understandably and obviously do not prevent a CA from making a
> change tomorrow that undermines the past audits; there is no guarantee they
> won’t start actively misissuing once the auditor has left the building. It
> is, however, meant to provide assurance regarding the present (and past)
> configuration. When a CA like T-Systems does misissue, whether this or
> previous incidents, it is entirely reasonable to ask “Was this
> configuration something the auditor previously reviewed, and did they catch
> it?” and, in the case of ETSI, “was this a change the auditor approved in
> relation to ongoing certification?”
Indeed, but the question has to be asked specifically, not just as a
hypothetical generality. For example did TUVIT review the ASN.1 module
used by T-Systems to encode the qc-statements in the incorrect
certificates? Did TUVIT compare it to the right edition of the upstream
standard? Did subtleties of the ASN.1 semantics and conventions obscure
the difference?
>
> The qcStatements demonstrates a failure of the latter, the bug demonstrates
> a failure of the former, both speak to the process of review and the
> qualifications of the reviewer.
>
Indeed, none of these mistakes should, ideally, have gotten past a
perfect auditor. The question is how far from ideal TUVIT was.
> If you don’t agree with the large swath of undetected past misissuance
> being a concern, it would be helpful if you could explain why it isn’t
> concerning.
It is concerning, the question is how large this swath really was.
Which requires actually looking at it, not just making blanket
statements that it was "large" or "small".
> For example, do you believe that these requirements
> (collectively, for any of these issues) were not covered by existing
> criteria?
They were all clearly covered by the criteria that mistakes should not
happen and auditing should detect mistakes. Most, if not all, were also
covered by more specific criteria.
> Do you believe that sufficient documentation of TUVIT’s
> methodology exists so as to explain why such failure to detect may be seen
> as reasonable?
I have yet to see sufficient documentation either way, and the thread
has been a lot of grandstanding on all sides.
> Do you believe that ETSI does not require consideration by
> auditors prior to operational and configuration changes?
I am not a trained ETSI auditor, and don't know for certain. You have
certainly said ETSI does require consideration before deliberate
operational and configuration changes. On the other hand I doubt the
auditor is supposed to consider every software patch to the level of a
full code review.
> In short, do you
> disagree that, when presented with CA misissuance, such as by T-Systems,
> that it is both relevant and appropriate to question why the auditor failed
> to detect and/or prevent such misissuance?
>
It is relevant to ask, but it takes a considerable level of certainty
before starting formal proceedings to disqualify an auditor due to the
failings of a single audit subject.
In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
the time some E&Y branches where disqualified by Mozilla.
In the world of technical review and testing, TUV SUD is a major player,
reviewing the safety of many technical products far outside Germany,
they rank in the same league as UL in the US.
> I am not arguing that an audit be a guarantee against misissuance; for
> example, a statistical sample will be just that, a sample, and stuff can
> reasonably slip through. I am, however, advocating that it’s both
> appropriate and necessary to question whether sampling was even done, and
> how it was constructed (e.g. CA selects the samples and sizes vs auditor),
> and what was reviewed, in order to ascertain whether or not it was
> “reasonable” to have missed something.
> In the case of T-Systems past
> misissuances, the collective sum - especially with respect to things like
> misconfigured templates - raises legitimate concerns about TUVITs approach
> and methodology, and those concerns are each themselves distinct issues
> with TUVIT for every misissuance “type” by T-Systems.
>
Of all the T-Systems mistakes listed so far, only U1, and maybe U11 are
clearly cases of bad templates.
Many others were mistakes at other parts of the issuing process. Many
in the area of technically (not semantically) validating the data
inserted into templates. For example, allowing absolute form DNS names
when the RFCs require root-relative DNS names with the root period
omitted. No amount of checking and reviewing the templates could have
found that the values being inserted would be wrong.
These are still bad of cause, but they involve other parts of the audit.
something else.
On 13/11/2018 14:31, Ryan Sleevi wrote:
> I suppose I had unreasonably hoped it would be self-evident, particularly
> for someone who claims to follow the issues, to understand how directly
> that issue was related. Unfortunately, whether for intent or otherwise, it
> appears not.
>
of words, weather or not something is under one rule or another etc.
etc., making it very hard to get a clear overview of the primary issues.
Furthermore the start of the thread was off-list. Also neither I, nor
some other participants have access to the audit reports etc. in CCADB.
This basic combination of noise and missing data is why I asked for a
one-stop overview of your complaints against TUVIT, similar to the lists
compiled for previous situations with multiple complaints against one
party.
> While I do not believe nor agree with your approach to framing the issues,
> I do hope you can agree that both through the bug - which itself is an
> amalgamation of and reference to several bugs - that during the prior two
> audit cycles, T-Systems contained a substantial amount of misissuance that
> were undetected by TUVIT and that shared the same root cause:
> misconfiguration and misapplication of the relevant rules, both in terms of
> ASN.1 and in terms of normative requirement.
>
broad as to describe the majority of CA failures without giving any
useful specifics to assess the situation. It's like saying someone's
crime was to "violate and break the relevant laws" (which would apply to
anything from jaywalking to mass murder).
It would also be useful to quantify the word substantial: Of all the
certificates issued by the audited CA organization, how large a
percentage suffered from each flaw, how many from none. This is a key
number when assessing if statistical sampling by the auditor should have
caught an issue. It is also a key number when assessing the level of
incompetence of the CA (but the CA is not the subject of this thread).
Bug #1391074 contained counts of affected certificates, but not how
many certificates did not have the listed issues. So it is not clear if
those issues were likely to be spotted by 3% sampling.
T4. One valid complaint is that TUVIT apparently (according to what I
read in this thread) didn't check the mailing list and bugzilla to
find out about the Bug #1391074 issue and include them in a relevant
audit statement (as this was already public information, the
confidentiality excuse doesn't apply).
T5. Another valid complaint is that TUVIT apparently did not see the
corrective measures described in bug #1391074, which should have been
in the CA's technical audit logs.
T6. Then there is the valid complain that the administrative paper trail
related to Bug #1391074 should have resulted in at least some event
observed by TUVIT as part of their ETSI/ISO monitoring during the
audit period, and that this should have caused TUVIT to look for the
rest of the issue and include it in the audit or audit followup
reports.
Issue U1 (Qc-statement misencoding) apparently affected all certificates
from one issuing CA, and should thus have been caught by sampling by the
auditor. The auditor has (according to earlier posts) admitted that the
bug was present in the sampled certificates from that issuing CA, but
that this was overlooked because that particular extension was not one
they had specific experience looking at. Once the problem was pointed
out the auditor looked at the previously collected evidence and
confirmed the problem by checking that detail from first principles
(similar to software developer hand-executing a function with pen and
paper to confirm a bug).
T7. So this IS a valid complaint against TUVIT: That the auditor didn't
check the Qc-statement encoding from first principles during the
original audit, and thus failed to spot the problem.
T8. Then there is the issue that TUVIT didn't issue an official
qualification of the CA's status once the issue was both known and
public.
S9. There is also the issue if TUVIT should have issued an official
qualification of the CA's status if they knew before the U1 issue
became public. This is subject to the debate about the
confidentiality provisions in the ISO audit standard versus the
expectations of Mozilla and Chrome.
S10. Then there is the issue if TUVIT was right or wrong in accepting a
slow phase out of the certificates affected by U1. This involves both
the principal issue if there should be zero tolerance for incorrect
certificates, the practical issue of how much harm this specific
standards validation can cause, and how much time should be allowed
for an orderly replacement process. Multiple months seems a quite
long time. 1 day quite a short time.
> If you are attempting to excuse such misissuance, rather than address it,
> one would take a similar tact as you are here; suggesting, for example,
> that it was T-Systems rather than TUVIT that did the misissuance or by
> suggesting that the incidence was low to be insignificant.
of the evidence to assess the level of diligence or failure by TUVIT.
It is of cause the purpose of any audit scheme to check for the absence
of irregularities, and to report if any were found. But it is quite
rare for the audit to essentially redo every piece of administrative
work done by the audited company.
Thus there is often the question of why an irregularity (in this case
incorrect CA issuances) did not make it into the public audited
accounts. And thus if the auditor was (also) at fault, which is the
topic of this thread.
> I was careful
> not to try to muddy the conversation through an indictment on T-Systems, to
> avoid diluting the conversation, and because they’ve already provided
> several enumerations of the issues and that doing so again, as you’ve done,
> does not add value.
versus what was in the audit reports and audit statements is a key
measure of the quality of the auditing.
> However, it should be readily apparent from both the
> bug discussion and the list of issues a common pattern of misconfiguring
> relevant profiles and failing to ensure they comply with the relevant
> requirements.
discussion message and but #1391074) that all, or even most, of the
issues were specifically profile misconfigurations rather than a litany
of other mistakes. This in turn ties in to the question how many of
those mistakes should have been obvious to the auditors versus being
buried too deep to be spotted.
The debate in bug #1391074 about the template used for ECDSA
certificates is a good example. According to the bug, the ECDSA
certificate profile/template was correct, but some piece of software
mishandled approved ECDSA certificate requests and used the RSA
certificate profile/template, for at least some of the issued
certificates. An incorrect ECDSA profile/template saying to set the
KeyEncryption bit should have been spotted by a configuration audit and
review (by TUVIT). But code bugs are notoriously harder to spot.
>
> In the context of ETSI, each of these configuration changes - particularly
> once qualified - undergoes some review; whether after the fact
> (pre-qualification) or prior to such change.
was subject to such review by or notification to TUVIT.
> Similarly, misissuance
> involves a degree of notification to the CAB.
don't think you expect a CA to notify the auditor that it is about to
misissue, and then proceed to actually misissue instead of stopping
itself.
> As such, it is entirely
> reasonable to expect a degree of supervision, as that is the value of the
> certification scheme. All of this information would have been available at
> the time of configuring qualified certificates, including the pattern of
> issues existing when configuring profiles and templates.
always a limit to the depth of audits, and thus we need to assess if
TUVIT was being sloppy, complacent, complicit or just unlucky.
>
> As such, we functionally see two issues; the inadequate supervision that
> resulted in the first batch of misissuance, which can be attempted to be
> argued away by suggesting it was some small volume that sampling would not
> have caught (despite the inconsistencies of that argument with the
> criteria),
statements for each of the Bug #1391074 issues if that one should have
been spotted by the auditor before it was found by the community, and
why. For example were the 3 incorrect ECDSA certificates 100%, 1% or
0.01% of the issued certificates from that particular issuing CA? And
how many of the certificates from that issuing CA should have been
sampled under the criteria?
This is of cause in addition to the question if or why not TUVIT was
subsequently fully aware of the bug report and the reports uploaded to
bugzilla by the CA.
> and inadequate supervision leading to this current issue,
> despite having all of that previous information available as context during
> the review and despite their being 100% misissuance rate.
specifically for that particular error. This applies both when
"supervising" the creation of whatever configuration, template or code
contained the bug and when auditing samples of issued certificates.
One could always take the extreme position that a supervisor has no
responsibility for the acts of the supervised, or the equally extreme
position that a supervisor is fully responsible for every act of the
supervised.
> Both of these
> share a clear commonality of inadequate supervision, a key role played over
> the past several years.
meaninglessness.
>
> Audits understandably and obviously do not prevent a CA from making a
> change tomorrow that undermines the past audits; there is no guarantee they
> won’t start actively misissuing once the auditor has left the building. It
> is, however, meant to provide assurance regarding the present (and past)
> configuration. When a CA like T-Systems does misissue, whether this or
> previous incidents, it is entirely reasonable to ask “Was this
> configuration something the auditor previously reviewed, and did they catch
> it?” and, in the case of ETSI, “was this a change the auditor approved in
> relation to ongoing certification?”
hypothetical generality. For example did TUVIT review the ASN.1 module
used by T-Systems to encode the qc-statements in the incorrect
certificates? Did TUVIT compare it to the right edition of the upstream
standard? Did subtleties of the ASN.1 semantics and conventions obscure
the difference?
>
> The qcStatements demonstrates a failure of the latter, the bug demonstrates
> a failure of the former, both speak to the process of review and the
> qualifications of the reviewer.
>
perfect auditor. The question is how far from ideal TUVIT was.
> If you don’t agree with the large swath of undetected past misissuance
> being a concern, it would be helpful if you could explain why it isn’t
> concerning.
Which requires actually looking at it, not just making blanket
statements that it was "large" or "small".
> For example, do you believe that these requirements
> (collectively, for any of these issues) were not covered by existing
> criteria?
happen and auditing should detect mistakes. Most, if not all, were also
covered by more specific criteria.
> Do you believe that sufficient documentation of TUVIT’s
> methodology exists so as to explain why such failure to detect may be seen
> as reasonable?
has been a lot of grandstanding on all sides.
> Do you believe that ETSI does not require consideration by
> auditors prior to operational and configuration changes?
certainly said ETSI does require consideration before deliberate
operational and configuration changes. On the other hand I doubt the
auditor is supposed to consider every software patch to the level of a
full code review.
> In short, do you
> disagree that, when presented with CA misissuance, such as by T-Systems,
> that it is both relevant and appropriate to question why the auditor failed
> to detect and/or prevent such misissuance?
>
before starting formal proceedings to disqualify an auditor due to the
failings of a single audit subject.
In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
the time some E&Y branches where disqualified by Mozilla.
In the world of technical review and testing, TUV SUD is a major player,
reviewing the safety of many technical products far outside Germany,
they rank in the same league as UL in the US.
> I am not arguing that an audit be a guarantee against misissuance; for
> example, a statistical sample will be just that, a sample, and stuff can
> reasonably slip through. I am, however, advocating that it’s both
> appropriate and necessary to question whether sampling was even done, and
> how it was constructed (e.g. CA selects the samples and sizes vs auditor),
> and what was reviewed, in order to ascertain whether or not it was
> “reasonable” to have missed something.
> In the case of T-Systems past
> misissuances, the collective sum - especially with respect to things like
> misconfigured templates - raises legitimate concerns about TUVITs approach
> and methodology, and those concerns are each themselves distinct issues
> with TUVIT for every misissuance “type” by T-Systems.
>
clearly cases of bad templates.
Many others were mistakes at other parts of the issuing process. Many
in the area of technically (not semantically) validating the data
inserted into templates. For example, allowing absolute form DNS names
when the RFCs require root-relative DNS names with the root period
omitted. No amount of checking and reviewing the templates could have
found that the values being inserted would be wrong.
These are still bad of cause, but they involve other parts of the audit.
Ryan Sleevi
Nov 13, 2018, 12:21:27 PM11/13/18
to Jakob Bohm, mozilla-dev-security-policy
>
>
>
> On Tue, Nov 13, 2018 at 11:26 AM Jakob Bohm via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> Furthermore the start of the thread was off-list. Also neither I, nor
>> some other participants have access to the audit reports etc. in CCADB.
>>
>
> Sure you do. That information is publicly available through
>
>
> On Tue, Nov 13, 2018 at 11:26 AM Jakob Bohm via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> Furthermore the start of the thread was off-list. Also neither I, nor
>> some other participants have access to the audit reports etc. in CCADB.
>>
>
> https://wiki.mozilla.org/CA/Included_Certificates
>
>
>> This basic combination of noise and missing data is why I asked for a
>> one-stop overview of your complaints against TUVIT, similar to the lists
>> compiled for previous situations with multiple complaints against one
>> party.
>>
>
> Those are the output of these discussions, not the input or structure to
>
>> This basic combination of noise and missing data is why I asked for a
>> one-stop overview of your complaints against TUVIT, similar to the lists
>> compiled for previous situations with multiple complaints against one
>> party.
>>
>
> them. There are certainly broader complaints, but if you'll note, my focus
> has been on attempting to satisfactorily resolve the current set of issues.
> Several times you've attempted to move it to the meta discussion, while
> I've tried to again focus on the specific lack of resolution for those
> initially identified issues. The reference to the other issues is precisely
> because the explanation and resolution of *these* issues can inform or be
> compared with the *past* issues, which would be used to build the list
> seemingly so desired.
>
>
>> "Misconfiguration and misapplication of the relevant rules..." is so
>> broad as to describe the majority of CA failures without giving any
>> useful specifics to assess the situation. It's like saying someone's
>> crime was to "violate and break the relevant laws" (which would apply to
>> anything from jaywalking to mass murder).
>>
>
> While sympathetic to your frustration, I think that's a rather extreme
>
>> "Misconfiguration and misapplication of the relevant rules..." is so
>> broad as to describe the majority of CA failures without giving any
>> useful specifics to assess the situation. It's like saying someone's
>> crime was to "violate and break the relevant laws" (which would apply to
>> anything from jaywalking to mass murder).
>>
>
> interpretation. For example, CAs seem to believe that the majority of their
> failures are "human error" and that human error is corrected by "additional
> training". Perhaps you would like to propose a better wording to
> distinguish between the "Guaranteed to produce the wrong result, 100% of
> the time" configuration issues, in which a certificate profile is
> functionally unable to meet the stated configuration, and those which are
> tied to, for example, data validation issues (or lack thereof). My intent
> was to capture the former, while acknowledging that the latter is something
> that is primarily accounted for through design review, sampling, and
> testing.
>
>
>> It would also be useful to quantify the word substantial: Of all the
>> certificates issued by the audited CA organization, how large a
>> percentage suffered from each flaw, how many from none. This is a key
>> number when assessing if statistical sampling by the auditor should have
>> caught an issue. It is also a key number when assessing the level of
>> incompetence of the CA (but the CA is not the subject of this thread).
>>
>
> I already responded to this previously, and again in my more recent
>
>> It would also be useful to quantify the word substantial: Of all the
>> certificates issued by the audited CA organization, how large a
>> percentage suffered from each flaw, how many from none. This is a key
>> number when assessing if statistical sampling by the auditor should have
>> caught an issue. It is also a key number when assessing the level of
>> incompetence of the CA (but the CA is not the subject of this thread).
>>
>
> messages. In the issue that started this thread, we can see it's 100%. In
> the past issuance examples, we can see that it was 100% of certificates
> going through certain systems. While that is less than 100% of total
> volume, sampling methodology also must consider variances and other
> factors. For example, if a CA issues DV, OV, and EV, a sampling methodology
> would approach each profile distinctly for sample selection, rather than
> overall issuance. A sampling method for a CA may involve 100+ such samples
> (each representing a percentage), based on the design review that
> identifies variations and permutations relevant to the service provide.
> Similarly, the selection of 3% is relevant to CA self-audits primarily.
>
> This is where the initial request for the discussion about methodology - a
> discussion about how a CAB can miss 100% of certificates being misissued -
> is relevant. And, as of yet, unaddressed.
>
> Issue U1 (Qc-statement misencoding) apparently affected all certificates
>> from one issuing CA, and should thus have been caught by sampling by the
>> auditor. The auditor has (according to earlier posts) admitted that the
>> bug was present in the sampled certificates from that issuing CA, but
>> that this was overlooked because that particular extension was not one
>> they had specific experience looking at. Once the problem was pointed
>> out the auditor looked at the previously collected evidence and
>> confirmed the problem by checking that detail from first principles
>> (similar to software developer hand-executing a function with pen and
>> paper to confirm a bug).
>>
>
> I don't believe that is a correct summary. The auditor reported things
> Issue U1 (Qc-statement misencoding) apparently affected all certificates
>> from one issuing CA, and should thus have been caught by sampling by the
>> auditor. The auditor has (according to earlier posts) admitted that the
>> bug was present in the sampled certificates from that issuing CA, but
>> that this was overlooked because that particular extension was not one
>> they had specific experience looking at. Once the problem was pointed
>> out the auditor looked at the previously collected evidence and
>> confirmed the problem by checking that detail from first principles
>> (similar to software developer hand-executing a function with pen and
>> paper to confirm a bug).
>>
>
> were correct - i.e. no bug - and only after pushing further to state very
> clearly that there was a bug did the auditor confirm that, oh yes, there
> was a bug, we just overlooked it. Now, I can understand that the favorable
> reading for the auditor was simply that they were busy and on the road and
> favoring expediency over correctness - but we've seen CAs using this same
> reasoning for years. Multiple times now, CAs have faced serious
> misissuances, confidently and repeatedly stated they've identified all of
> them, and then be presented with an example certificate not identified by
> the CA that demonstrates the exact problem. Do you disagree that auditors
> should be aware of the perception of such responses and the harm to trust?
> Do you believe auditors should be held to a different standard?
>
>
>> S10. Then there is the issue if TUVIT was right or wrong in accepting a
>> slow phase out of the certificates affected by U1. This involves both
>> the principal issue if there should be zero tolerance for incorrect
>> certificates, the practical issue of how much harm this specific
>> standards validation can cause, and how much time should be allowed
>> for an orderly replacement process. Multiple months seems a quite
>> long time. 1 day quite a short time.
>>
>
> I've snipped a majority of your statements, mostly because I don't find
>
>> S10. Then there is the issue if TUVIT was right or wrong in accepting a
>> slow phase out of the certificates affected by U1. This involves both
>> the principal issue if there should be zero tolerance for incorrect
>> certificates, the practical issue of how much harm this specific
>> standards validation can cause, and how much time should be allowed
>> for an orderly replacement process. Multiple months seems a quite
>> long time. 1 day quite a short time.
>>
>
> them correct or helpful framing. To the extent I'm signalling specific
> things, it's because they are particularly egregious. I think Wayne has
> already responded in a way that conflicts with your statement of S10; you
> pose several extremes or absolutes, but that's not the issue. If we take a
> step back, the issue here is that TUVIT has taken the view that the ETSI
> specification supersedes the requirements of the Baseline Requirements;
> where the BRs are quite prescriptive in their requirements (24 hours - 5
> days), TUVIT has taken a position that any period not exceeding three
> months is acceptable. This, combined with the lack of reporting - which is
> not supported, in practice, by other CABs - creates a situation where, for
> audits conducted by TUVIT, there is zero community assurance that the
> provisions of 4.9.1.1 have been followed.
>
>
>> It is of cause the purpose of any audit scheme to check for the absence
>> of irregularities, and to report if any were found.
>
>
> Except that's not the point of the ETSI audit, which is at least why some
>
>> It is of cause the purpose of any audit scheme to check for the absence
>> of irregularities, and to report if any were found.
>
>
> discussion of the scheme is relevant. The "report if any were found" is
> functionally absent. The 'reporting' is done based on whether or not the
> certificate was issued, and the certificate is issued provided that any
> irregularities were resolved, to the auditor's satisfaction, within 90
> days. This is something 'unique' to ETSI audits, and not part of the
> underlying ISO/IEC 17065.
>
> Other ETSI-based auditors have recognized this gap, and thus ensured
> they've reported on irregularities. That shows how they can address the gap
> from the bare minimum required of ETSI and instead meet the expectations of
> the browsers consuming the reports.
>
>
>> But it is quite
>> rare for the audit to essentially redo every piece of administrative
>> work done by the audited company.
>>
>
> It's unclear your intended remark. During the sampling process, it is
>
>> But it is quite
>> rare for the audit to essentially redo every piece of administrative
>> work done by the audited company.
>>
>
> indeed a cross-check of all the administrative work - ensuring that
> sufficient evidence of all of the controls that exist to ensure the correct
> functioning of that administrative work were followed, with detailed
> analysis.
>
> The debate in bug #1391074 about the template used for ECDSA
>> certificates is a good example. According to the bug, the ECDSA
>> certificate profile/template was correct, but some piece of software
>> mishandled approved ECDSA certificate requests and used the RSA
>> certificate profile/template, for at least some of the issued
>> certificates. An incorrect ECDSA profile/template saying to set the
>> KeyEncryption bit should have been spotted by a configuration audit and
>> review (by TUVIT). But code bugs are notoriously harder to spot.
>>
>
> I've got no idea where you got that summary from, but that's certainly not
> The debate in bug #1391074 about the template used for ECDSA
>> certificates is a good example. According to the bug, the ECDSA
>> certificate profile/template was correct, but some piece of software
>> mishandled approved ECDSA certificate requests and used the RSA
>> certificate profile/template, for at least some of the issued
>> certificates. An incorrect ECDSA profile/template saying to set the
>> KeyEncryption bit should have been spotted by a configuration audit and
>> review (by TUVIT). But code bugs are notoriously harder to spot.
>>
>
> consistent with 3.3 of
> https://bug1391074.bmoattachments.org/attachment.cgi?id=8915934
>
> An EC profile/template was not configured. All certificates, regardless of
> key type, were configured to use the same profile.
>
> It is expected, of all CAs, that profiles are distinct per key type, least
> of all due to the necessity to ensure both the input keys are appropriate
> (e.g. correct strength) and the output certificate is correct from RFC 5280.
>
>
>> > In the context of ETSI, each of these configuration changes -
>> particularly
>> > once qualified - undergoes some review; whether after the fact
>> > (pre-qualification) or prior to such change.
>>
>
>> > In the context of ETSI, each of these configuration changes -
>> particularly
>> > once qualified - undergoes some review; whether after the fact
>> > (pre-qualification) or prior to such change.
>>
>> This is why it is interesting to look at each issue to determine if it
>> was subject to such review by or notification to TUVIT.
>>
>
> I'm not sure what you're saying. The model of certification requires this.
>> was subject to such review by or notification to TUVIT.
>>
>
> Is your framing of the question whether or not T-Systems notified TUVIT of
> these issues? If they didn't, they were contractually negligent under the
> ETSI model, and TUVIT should, as part of their explanation, indicate how
> they addressed those failures. I'm taking Occam's Razor here, and presuming
> that TUVIT was notified, as they were required to be, and that the failure
> rests with TUVIT.
>
>
>> > Similarly, misissuance
>> > involves a degree of notification to the CAB.
>>
>
>> > Similarly, misissuance
>> > involves a degree of notification to the CAB.
>>
>> Only once known (e.g. around the time of the bug reports). Because I
>> don't think you expect a CA to notify the auditor that it is about to
>> misissue, and then proceed to actually misissue instead of stopping
>> itself.
>>
>
> Yes, for all configuration changes post-misissuance, the CAB would have
>> don't think you expect a CA to notify the auditor that it is about to
>> misissue, and then proceed to actually misissue instead of stopping
>> itself.
>>
>
> been notified. One would thus reasonably expect that, as these
> notifications increase, the CAB would take a more careful look at both
> patterns of problems and specific root causes to ensure that future issues
> are premptively identified, rather than retroactively remediated. This is
> why 100% misissuance is particularly concerning *given* past issues.
>
> Yet the substance of the discussion - the "current" issue if you will -
> can be discussed without the lengthy debate and dissection being had in
> your message here. That's because TUVIT can respond with regards to their
> methodology and approach used. If that methodology *didn't* consider the
> past incidents, then we can go through that retroactive dissection. If they
> did, however, then we can allow TUVIT to respond as to what they did and
> how it impacted.
>
> As such, I can see limited value in the present conversation for
> continuing this belabored dissection - we should instead focus on the
> methodology and approach used for the most recent issue, and based on that,
> analyze retroactively, to evaluate whether or not adequate assurance is
> being provided.
>
>
>>
>> > As such, it is entirely
>> > reasonable to expect a degree of supervision, as that is the value of
>> the
>> > certification scheme. All of this information would have been available
>> at
>> > the time of configuring qualified certificates, including the pattern of
>> > issues existing when configuring profiles and templates.
>>
>
>>
>> > As such, it is entirely
>> > reasonable to expect a degree of supervision, as that is the value of
>> the
>> > certification scheme. All of this information would have been available
>> at
>> > the time of configuring qualified certificates, including the pattern of
>> > issues existing when configuring profiles and templates.
>>
>> Should have been available doesn't mean it was available. There is
>> always a limit to the depth of audits, and thus we need to assess if
>> TUVIT was being sloppy, complacent, complicit or just unlucky.
>>
>
> As captured above, I disagree with that, not on principle, but in
>> always a limit to the depth of audits, and thus we need to assess if
>> TUVIT was being sloppy, complacent, complicit or just unlucky.
>>
>
> execution. We need more information from TUVIT, rather than attempting to
> divine through first principles as this thread tries to do. If no further
> information is provided, then we don't need to bother with that assessment
> at all - an auditor who is unwilling or unable to provide reasonable
> information is an auditor that should not be trusted.
>
>
>> It is relevant to ask, but it takes a considerable level of certainty
>> before starting formal proceedings to disqualify an auditor due to the
>> failings of a single audit subject.
>>
>> In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
>> the time some E&Y branches where disqualified by Mozilla.
>
>
>> In the world of technical review and testing, TUV SUD is a major player,
>> reviewing the safety of many technical products far outside Germany,
>> they rank in the same league as UL in the US.
>>
>
> This gets so close to understanding the issue, but then radically misses
>
>> It is relevant to ask, but it takes a considerable level of certainty
>> before starting formal proceedings to disqualify an auditor due to the
>> failings of a single audit subject.
>>
>> In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
>> the time some E&Y branches where disqualified by Mozilla.
>
>
>> In the world of technical review and testing, TUV SUD is a major player,
>> reviewing the safety of many technical products far outside Germany,
>> they rank in the same league as UL in the US.
>>
>
> the point. Multiple E&Y branches were disqualified on the basis of a single
> audit, but E&Y is not blanket disqualified.
>
> TUV IT is a specific entity, the equivalent to an E&Y "branch". Mentioning
> TUV SUD in the context of TUV IT is akin to mentioning KPMG in a discussion
> about E&Y. TUV IT is part of the TUV NORD group, which is itself distinct
> from TUV SUD.
>
Jakob Bohm
Nov 14, 2018, 7:27:01 PM11/14/18
to mozilla-dev-s...@lists.mozilla.org
Once again, you snipped most of what I wrote.
Also not sure why your post has double reply marking.
On 13/11/2018 18:20, Ryan Sleevi wrote:
>>
>>
>>
>> On Tue, Nov 13, 2018 at 11:26 AM Jakob Bohm via dev-security-policy <
>> dev-secur...@lists.mozilla.org> wrote:
>>
>>> Furthermore the start of the thread was off-list. Also neither I, nor
>>> some other participants have access to the audit reports etc. in CCADB.
>>>
>>
>> Sure you do. That information is publicly available through
>> https://wiki.mozilla.org/CA/Included_Certificates
>>
I am quite surprised that a supposed report of included certificates
hides access to Audit report links. The CCADB entrypoints I had
previously looked at did not provide those deep details. But thanks
for the link.
Although at least for the first T-Systems CA in the table, the PDF was
just a summary attestation, apparently covering the period from before
Bug#1391074 was reported until a time overlapping the time when Qc-
Statements were misencoded (Though not necessarily under that root,
once again the lack of specifics makes it hard to check things).
One thing not provided by the public report is the history of T-Systems
problem reporting contact point, and thus if it included (at that time)
the specific contact points used in issue U2.
>>
>>> This basic combination of noise and missing data is why I asked for a
>>> one-stop overview of your complaints against TUVIT, similar to the lists
>>> compiled for previous situations with multiple complaints against one
>>> party.
>>>
>>
>> Those are the output of these discussions, not the input or structure to
>> them. There are certainly broader complaints, but if you'll note, my focus
>> has been on attempting to satisfactorily resolve the current set of issues.
>> Several times you've attempted to move it to the meta discussion, while
>> I've tried to again focus on the specific lack of resolution for those
>> initially identified issues. The reference to the other issues is precisely
>> because the explanation and resolution of *these* issues can inform or be
>> compared with the *past* issues, which would be used to build the list
>> seemingly so desired.
>>
However you seem to consistently refuse to clearly state, in one place,
what those current issues are.
>>
>>> "Misconfiguration and misapplication of the relevant rules..." is so
>>> broad as to describe the majority of CA failures without giving any
>>> useful specifics to assess the situation. It's like saying someone's
>>> crime was to "violate and break the relevant laws" (which would apply to
>>> anything from jaywalking to mass murder).
>>>
>>
>> While sympathetic to your frustration, I think that's a rather extreme
>> interpretation. For example, CAs seem to believe that the majority of their
>> failures are "human error" and that human error is corrected by "additional
>> training". Perhaps you would like to propose a better wording to
>> distinguish between the "Guaranteed to produce the wrong result, 100% of
>> the time" configuration issues, in which a certificate profile is
>> functionally unable to meet the stated configuration, and those which are
>> tied to, for example, data validation issues (or lack thereof). My intent
>> was to capture the former, while acknowledging that the latter is something
>> that is primarily accounted for through design review, sampling, and
>> testing.
>>
This was in direct reply to where you used that meaningless phrase. I
was simply telling you it was meaningless. Congrats on once again
removing context.
>>
>>> It would also be useful to quantify the word substantial: Of all the
>>> certificates issued by the audited CA organization, how large a
>>> percentage suffered from each flaw, how many from none. This is a key
>>> number when assessing if statistical sampling by the auditor should have
>>> caught an issue. It is also a key number when assessing the level of
>>> incompetence of the CA (but the CA is not the subject of this thread).
>>>
>>
>> I already responded to this previously, and again in my more recent
>> messages. In the issue that started this thread, we can see it's 100%. In
>> the past issuance examples, we can see that it was 100% of certificates
>> going through certain systems. While that is less than 100% of total
>> volume, sampling methodology also must consider variances and other
>> factors. For example, if a CA issues DV, OV, and EV, a sampling methodology
>> would approach each profile distinctly for sample selection, rather than
>> overall issuance. A sampling method for a CA may involve 100+ such samples
>> (each representing a percentage), based on the design review that
>> identifies variations and permutations relevant to the service provide.
>> Similarly, the selection of 3% is relevant to CA self-audits primarily.
>>
The issue in this thread is TUVIT competence and trustworthiness, which
cannot possibly be judged solely on ONE small CA incident (the U1 qc-
statement misencoding incident). I must therefore operate on the basis
that this applies to TUVIT's auditing of the totality of the T-Systems
failures mentioned so far.
So far, only issue U1 has been claimed to affect 100% of a certificate
category for an issuing CA, unless you create so many categories as to
cause a combinatorial explosion in the presumed audit work.
As for the 3% number, I thought it also applied to external audits, but
this is less important until there is a specific accusation of TUVIT
using a too low percentage for a specific audit activity.
For U1, the audit issue was not the sample percentage, but that TUVIT
did not check the qc-statement closely enough in the samples collected
until the bug was reported.
>> This is where the initial request for the discussion about methodology - a
>> discussion about how a CAB can miss 100% of certificates being misissued -
>> is relevant. And, as of yet, unaddressed.
>>
There were a few responses to this by TUVIT earlier in the thread, but
you dismissed their explanations (as you do again below).
>>> Issue U1 (Qc-statement misencoding) apparently affected all certificates
>>> from one issuing CA, and should thus have been caught by sampling by the
>>> auditor. The auditor has (according to earlier posts) admitted that the
>>> bug was present in the sampled certificates from that issuing CA, but
>>> that this was overlooked because that particular extension was not one
>>> they had specific experience looking at. Once the problem was pointed
>>> out the auditor looked at the previously collected evidence and
>>> confirmed the problem by checking that detail from first principles
>>> (similar to software developer hand-executing a function with pen and
>>> paper to confirm a bug).
>>>
>>
>> I don't believe that is a correct summary. The auditor reported things
>> were correct - i.e. no bug - and only after pushing further to state very
>> clearly that there was a bug did the auditor confirm that, oh yes, there
>> was a bug, we just overlooked it. Now, I can understand that the favorable
>> reading for the auditor was simply that they were busy and on the road and
>> favoring expediency over correctness - but we've seen CAs using this same
>> reasoning for years. Multiple times now, CAs have faced serious
>> misissuances, confidently and repeatedly stated they've identified all of
>> them, and then be presented with an example certificate not identified by
>> the CA that demonstrates the exact problem. Do you disagree that auditors
>> should be aware of the perception of such responses and the harm to trust?
>> Do you believe auditors should be held to a different standard?
>>
While there are vaguely similar cases of incorrect denials by CAs, the
message in which a TUVIT auditor said the wrong thing about the U1
incident is in the part of the thread that is not public and thus we as
a community cannot see if the auditor made it clear that this was a
preliminary answer to be completed later. Nor can we see if the second
look once back in the office required additional prompting or was self-
initiated as soon as practical.
>>
>>> S10. Then there is the issue if TUVIT was right or wrong in accepting a
>>> slow phase out of the certificates affected by U1. This involves both
>>> the principal issue if there should be zero tolerance for incorrect
>>> certificates, the practical issue of how much harm this specific
>>> standards validation can cause, and how much time should be allowed
>>> for an orderly replacement process. Multiple months seems a quite
>>> long time. 1 day quite a short time.
>>>
>>
>> I've snipped a majority of your statements, mostly because I don't find
>> them correct or helpful framing. To the extent I'm signalling specific
>> things, it's because they are particularly egregious. I think Wayne has
>> already responded in a way that conflicts with your statement of S10; you
>> pose several extremes or absolutes, but that's not the issue. If we take a
>> step back, the issue here is that TUVIT has taken the view that the ETSI
>> specification supersedes the requirements of the Baseline Requirements;
>> where the BRs are quite prescriptive in their requirements (24 hours - 5
>> days), TUVIT has taken a position that any period not exceeding three
>> months is acceptable. This, combined with the lack of reporting - which is
>> not supported, in practice, by other CABs - creates a situation where, for
>> audits conducted by TUVIT, there is zero community assurance that the
>> provisions of 4.9.1.1 have been followed.
>>
Your massive snipping seems more about creating false narratives and
meaningless responses.
The one paragraph above is my only specific statement of S10 and
contains no extremes or absolutes, just lots of questions of balance and
opinion. It is a statement that there is a disagreement between you as a
Mozilla representative and a TUVIT representative as to that issue of
interpretation. It is also an assignment of a number to that issue so it
can be referenced separately from other related issues, like T2, T3, T7,
T8 and S9 .
In terms of the BRs, issue S10 is a disagreement with TUVIT if a
misencoding that has (concretely) no severe technical effects should
fall under BR 4.9.1.1 #9 (24 hour revocation for BR violation) or if
this requirement should be softened in light of BR 4.9.5 #1 (considering
the nature of the problem when determining the time to process a
revocation request) and the fact that BR 4.9.4 (Revocation Grace Period)
is a "no stipulation". Plus a disagreement of how long the grace period,
if any, should be in the specific case of issue U1.
That TUVIT is citing ETSI equivalents to those BRs should be of much
less importance.
Below you are quoting me far out of context again:
>>
>>> It is of cause the purpose of any audit scheme to check for the absence
>>> of irregularities, and to report if any were found.
>>
>>
>> Except that's not the point of the ETSI audit, which is at least why some
>> discussion of the scheme is relevant. The "report if any were found" is
>> functionally absent. The 'reporting' is done based on whether or not the
>> certificate was issued, and the certificate is issued provided that any
>> irregularities were resolved, to the auditor's satisfaction, within 90
>> days. This is something 'unique' to ETSI audits, and not part of the
>> underlying ISO/IEC 17065.
>>
>> Other ETSI-based auditors have recognized this gap, and thus ensured
>> they've reported on irregularities. That shows how they can address the gap
>> from the bare minimum required of ETSI and instead meet the expectations of
>> the browsers consuming the reports.
>>
Ok, that is a new issue, lets number it as follows:
S11. Disagreement with TUVIT about how much the public audit summary
documents should say about detected incidents rather than blankly
stating (directly or indirectly) that all such were satisfactorily
resolved in less than 90 days. Minimum ETSI requirements may be too
weak in this area.
>>
>>> But it is quite
>>> rare for the audit to essentially redo every piece of administrative
>>> work done by the audited company.
>>>
>>
>> It's unclear your intended remark. During the sampling process, it is
>> indeed a cross-check of all the administrative work - ensuring that
>> sufficient evidence of all of the controls that exist to ensure the correct
>> functioning of that administrative work were followed, with detailed
>> analysis.
My intention is that a cross-check is not a 100% check. Some of your
past arguments seem to presume that anything slipping past the auditor
means the audit wasn't proper.
>>
>>> The debate in bug #1391074 about the template used for ECDSA
>>> certificates is a good example. According to the bug, the ECDSA
>>> certificate profile/template was correct, but some piece of software
>>> mishandled approved ECDSA certificate requests and used the RSA
>>> certificate profile/template, for at least some of the issued
>>> certificates. An incorrect ECDSA profile/template saying to set the
>>> KeyEncryption bit should have been spotted by a configuration audit and
>>> review (by TUVIT). But code bugs are notoriously harder to spot.
>>>
>>
>> I've got no idea where you got that summary from, but that's certainly not
>> consistent with 3.3 of
>> https://bug1391074.bmoattachments.org/attachment.cgi?id=8915934
>>
I was basing my extraction on what was on the Bugzilla page itself, I
don't have the capacity to do infinite recursion into every link and
attachment from your lack of providing your own list of issues.
>> An EC profile/template was not configured. All certificates, regardless of
>> key type, were configured to use the same profile.
>>
>> It is expected, of all CAs, that profiles are distinct per key type, least
>> of all due to the necessity to ensure both the input keys are appropriate
>> (e.g. correct strength) and the output certificate is correct from RFC 5280.
>>
The text on the Bugzilla page itself says that the issue wasn't a missing
template, but that a bug caused the PKI system to not use the right template
for at least 3 ECDSA certificates. This was in Comment #23 and more detailed
in Comment #28 (as issue C) at
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c23
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c28
Both of those bug comments were after the report in 8911534 and were
specifically clarifying that report.
I have not found a statement as to what the bug was or if it affected
all ECDSA certificates in a logical part of the system. There are
multiple possible guesses.
>>
>>>> In the context of ETSI, each of these configuration changes -
>>> particularly
>>>> once qualified - undergoes some review; whether after the fact
>>>> (pre-qualification) or prior to such change.
>>>
>>> This is why it is interesting to look at each issue to determine if it
>>> was subject to such review by or notification to TUVIT.
>>>
>>
>> I'm not sure what you're saying. The model of certification requires this.
>> Is your framing of the question whether or not T-Systems notified TUVIT of
>> these issues? If they didn't, they were contractually negligent under the
>> ETSI model, and TUVIT should, as part of their explanation, indicate how
>> they addressed those failures. I'm taking Occam's Razor here, and presuming
>> that TUVIT was notified, as they were required to be, and that the failure
>> rests with TUVIT.
>>
As I said elsewhere in previous messages, I find it unlikely that the ETSI
scheme requires auditor involvement for every possible Administrative action,
thus each technical incident at T-Systems (the ones with U numbers) need to
be individually classified to determine what, if any, related actions should
have been notified to TUVIT and if so, why this did not result in audit
actions by TUVIT.
For example, which of the actions or inactions that lead to issue U5 (double
dots in DNS names) involved a required review by TUVIT? The answer affects
how much blame for U5 can be laid on TUVIT, but does not affect if TUVIT can
be blamed for its post hoc auditing of the U5 incident.
Same questions for each U number.
>>
>>>> Similarly, misissuance
>>>> involves a degree of notification to the CAB.
>>>
>>> Only once known (e.g. around the time of the bug reports). Because I
>>> don't think you expect a CA to notify the auditor that it is about to
>>> misissue, and then proceed to actually misissue instead of stopping
>>> itself.
>>>
>>
>> Yes, for all configuration changes post-misissuance, the CAB would have
>> been notified. One would thus reasonably expect that, as these
>> notifications increase, the CAB would take a more careful look at both
>> patterns of problems and specific root causes to ensure that future issues
>> are premptively identified, rather than retroactively remediated.
Thanks for clarifying your past statement, which was a bit confusing.
>> This is
>> why 100% misissuance is particularly concerning *given* past issues.
>>
>> Yet the substance of the discussion - the "current" issue if you will -
>> can be discussed without the lengthy debate and dissection being had in
>> your message here. That's because TUVIT can respond with regards to their
>> methodology and approach used. If that methodology *didn't* consider the
>> past incidents, then we can go through that retroactive dissection. If they
>> did, however, then we can allow TUVIT to respond as to what they did and
>> how it impacted.
Unfortunately, most of the the thread has been meaningless grand standing
rather than specific questions about methodology and approach. It is small
wonder that the responses have been of the fire extinguisher and asbestos
suits kind.
>>
>> As such, I can see limited value in the present conversation for
>> continuing this belabored dissection - we should instead focus on the
>> methodology and approach used for the most recent issue, and based on that,
>> analyze retroactively, to evaluate whether or not adequate assurance is
>> being provided.
>>
You often see limited value in any conversation that disagrees with you.
>>
>>>
>>>> As such, it is entirely
>>>> reasonable to expect a degree of supervision, as that is the value of
>>> the
>>>> certification scheme. All of this information would have been available
>>> at
>>>> the time of configuring qualified certificates, including the pattern of
>>>> issues existing when configuring profiles and templates.
>>>
>>> Should have been available doesn't mean it was available. There is
>>> always a limit to the depth of audits, and thus we need to assess if
>>> TUVIT was being sloppy, complacent, complicit or just unlucky.
>>>
>>
>> As captured above, I disagree with that, not on principle, but in
>> execution. We need more information from TUVIT, rather than attempting to
>> divine through first principles as this thread tries to do. If no further
>> information is provided, then we don't need to bother with that assessment
>> at all - an auditor who is unwilling or unable to provide reasonable
>> information is an auditor that should not be trusted.
>>
I am not suggesting we divine from first principles. I suggest we actually
find out, rather than presume the worst possible situation and then proceed
to demand punishment according to such presumed violations.
>>
>>> It is relevant to ask, but it takes a considerable level of certainty
>>> before starting formal proceedings to disqualify an auditor due to the
>>> failings of a single audit subject.
>>>
>>> In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
>>> the time some E&Y branches where disqualified by Mozilla.
>>
>>
>>> In the world of technical review and testing, TUV SUD is a major player,
>>> reviewing the safety of many technical products far outside Germany,
>>> they rank in the same league as UL in the US.
>>>
>>
>> This gets so close to understanding the issue, but then radically misses
>> the point. Multiple E&Y branches were disqualified on the basis of a single
>> audit, but E&Y is not blanket disqualified.
I seem to recall that E&Y Hong Kong was involved with both the WoSign audit
and the audit of the Korean RA that misissued under a Symantec CA.
>>
>> TUV IT is a specific entity, the equivalent to an E&Y "branch". Mentioning
>> TUV SUD in the context of TUV IT is akin to mentioning KPMG in a discussion
>> about E&Y. TUV IT is part of the TUV NORD group, which is itself distinct
>> from TUV SUD.
>>
Oh, I thought it was TUV SUD.
But still, TUV IT seems to be the name for all of that TUV half's PKI
related audits, across all locations, similar to E&Y's global WebTrust
activities (as opposed to the WebTrust activities of E&Y's Hong Kong
branch office).
This Mozilla group has not had any reason (or purpose) to distrust E&Y
Hong Kong's auditing of financials etc., even though that is presumable
the same "entity" as the one doing bad WebTrust audits.
The difference between horizontal and vertical org charts should not
be the main dispositive issue in deciding if Mozilla should distrust
a global company or a local branch.
Also not sure why your post has double reply marking.
On 13/11/2018 18:20, Ryan Sleevi wrote:
>>
>>
>>
>> On Tue, Nov 13, 2018 at 11:26 AM Jakob Bohm via dev-security-policy <
>> dev-secur...@lists.mozilla.org> wrote:
>>
>>> Furthermore the start of the thread was off-list. Also neither I, nor
>>> some other participants have access to the audit reports etc. in CCADB.
>>>
>>
>> Sure you do. That information is publicly available through
>> https://wiki.mozilla.org/CA/Included_Certificates
>>
hides access to Audit report links. The CCADB entrypoints I had
previously looked at did not provide those deep details. But thanks
for the link.
Although at least for the first T-Systems CA in the table, the PDF was
just a summary attestation, apparently covering the period from before
Bug#1391074 was reported until a time overlapping the time when Qc-
Statements were misencoded (Though not necessarily under that root,
once again the lack of specifics makes it hard to check things).
One thing not provided by the public report is the history of T-Systems
problem reporting contact point, and thus if it included (at that time)
the specific contact points used in issue U2.
>>
>>> This basic combination of noise and missing data is why I asked for a
>>> one-stop overview of your complaints against TUVIT, similar to the lists
>>> compiled for previous situations with multiple complaints against one
>>> party.
>>>
>>
>> Those are the output of these discussions, not the input or structure to
>> them. There are certainly broader complaints, but if you'll note, my focus
>> has been on attempting to satisfactorily resolve the current set of issues.
>> Several times you've attempted to move it to the meta discussion, while
>> I've tried to again focus on the specific lack of resolution for those
>> initially identified issues. The reference to the other issues is precisely
>> because the explanation and resolution of *these* issues can inform or be
>> compared with the *past* issues, which would be used to build the list
>> seemingly so desired.
>>
what those current issues are.
>>
>>> "Misconfiguration and misapplication of the relevant rules..." is so
>>> broad as to describe the majority of CA failures without giving any
>>> useful specifics to assess the situation. It's like saying someone's
>>> crime was to "violate and break the relevant laws" (which would apply to
>>> anything from jaywalking to mass murder).
>>>
>>
>> While sympathetic to your frustration, I think that's a rather extreme
>> interpretation. For example, CAs seem to believe that the majority of their
>> failures are "human error" and that human error is corrected by "additional
>> training". Perhaps you would like to propose a better wording to
>> distinguish between the "Guaranteed to produce the wrong result, 100% of
>> the time" configuration issues, in which a certificate profile is
>> functionally unable to meet the stated configuration, and those which are
>> tied to, for example, data validation issues (or lack thereof). My intent
>> was to capture the former, while acknowledging that the latter is something
>> that is primarily accounted for through design review, sampling, and
>> testing.
>>
was simply telling you it was meaningless. Congrats on once again
removing context.
>>
>>> It would also be useful to quantify the word substantial: Of all the
>>> certificates issued by the audited CA organization, how large a
>>> percentage suffered from each flaw, how many from none. This is a key
>>> number when assessing if statistical sampling by the auditor should have
>>> caught an issue. It is also a key number when assessing the level of
>>> incompetence of the CA (but the CA is not the subject of this thread).
>>>
>>
>> I already responded to this previously, and again in my more recent
>> messages. In the issue that started this thread, we can see it's 100%. In
>> the past issuance examples, we can see that it was 100% of certificates
>> going through certain systems. While that is less than 100% of total
>> volume, sampling methodology also must consider variances and other
>> factors. For example, if a CA issues DV, OV, and EV, a sampling methodology
>> would approach each profile distinctly for sample selection, rather than
>> overall issuance. A sampling method for a CA may involve 100+ such samples
>> (each representing a percentage), based on the design review that
>> identifies variations and permutations relevant to the service provide.
>> Similarly, the selection of 3% is relevant to CA self-audits primarily.
>>
cannot possibly be judged solely on ONE small CA incident (the U1 qc-
statement misencoding incident). I must therefore operate on the basis
that this applies to TUVIT's auditing of the totality of the T-Systems
failures mentioned so far.
So far, only issue U1 has been claimed to affect 100% of a certificate
category for an issuing CA, unless you create so many categories as to
cause a combinatorial explosion in the presumed audit work.
As for the 3% number, I thought it also applied to external audits, but
this is less important until there is a specific accusation of TUVIT
using a too low percentage for a specific audit activity.
For U1, the audit issue was not the sample percentage, but that TUVIT
did not check the qc-statement closely enough in the samples collected
until the bug was reported.
>> This is where the initial request for the discussion about methodology - a
>> discussion about how a CAB can miss 100% of certificates being misissued -
>> is relevant. And, as of yet, unaddressed.
>>
you dismissed their explanations (as you do again below).
>>> Issue U1 (Qc-statement misencoding) apparently affected all certificates
>>> from one issuing CA, and should thus have been caught by sampling by the
>>> auditor. The auditor has (according to earlier posts) admitted that the
>>> bug was present in the sampled certificates from that issuing CA, but
>>> that this was overlooked because that particular extension was not one
>>> they had specific experience looking at. Once the problem was pointed
>>> out the auditor looked at the previously collected evidence and
>>> confirmed the problem by checking that detail from first principles
>>> (similar to software developer hand-executing a function with pen and
>>> paper to confirm a bug).
>>>
>>
>> I don't believe that is a correct summary. The auditor reported things
>> were correct - i.e. no bug - and only after pushing further to state very
>> clearly that there was a bug did the auditor confirm that, oh yes, there
>> was a bug, we just overlooked it. Now, I can understand that the favorable
>> reading for the auditor was simply that they were busy and on the road and
>> favoring expediency over correctness - but we've seen CAs using this same
>> reasoning for years. Multiple times now, CAs have faced serious
>> misissuances, confidently and repeatedly stated they've identified all of
>> them, and then be presented with an example certificate not identified by
>> the CA that demonstrates the exact problem. Do you disagree that auditors
>> should be aware of the perception of such responses and the harm to trust?
>> Do you believe auditors should be held to a different standard?
>>
message in which a TUVIT auditor said the wrong thing about the U1
incident is in the part of the thread that is not public and thus we as
a community cannot see if the auditor made it clear that this was a
preliminary answer to be completed later. Nor can we see if the second
look once back in the office required additional prompting or was self-
initiated as soon as practical.
>>
>>> S10. Then there is the issue if TUVIT was right or wrong in accepting a
>>> slow phase out of the certificates affected by U1. This involves both
>>> the principal issue if there should be zero tolerance for incorrect
>>> certificates, the practical issue of how much harm this specific
>>> standards validation can cause, and how much time should be allowed
>>> for an orderly replacement process. Multiple months seems a quite
>>> long time. 1 day quite a short time.
>>>
>>
>> I've snipped a majority of your statements, mostly because I don't find
>> them correct or helpful framing. To the extent I'm signalling specific
>> things, it's because they are particularly egregious. I think Wayne has
>> already responded in a way that conflicts with your statement of S10; you
>> pose several extremes or absolutes, but that's not the issue. If we take a
>> step back, the issue here is that TUVIT has taken the view that the ETSI
>> specification supersedes the requirements of the Baseline Requirements;
>> where the BRs are quite prescriptive in their requirements (24 hours - 5
>> days), TUVIT has taken a position that any period not exceeding three
>> months is acceptable. This, combined with the lack of reporting - which is
>> not supported, in practice, by other CABs - creates a situation where, for
>> audits conducted by TUVIT, there is zero community assurance that the
>> provisions of 4.9.1.1 have been followed.
>>
meaningless responses.
The one paragraph above is my only specific statement of S10 and
contains no extremes or absolutes, just lots of questions of balance and
opinion. It is a statement that there is a disagreement between you as a
Mozilla representative and a TUVIT representative as to that issue of
interpretation. It is also an assignment of a number to that issue so it
can be referenced separately from other related issues, like T2, T3, T7,
T8 and S9 .
In terms of the BRs, issue S10 is a disagreement with TUVIT if a
misencoding that has (concretely) no severe technical effects should
fall under BR 4.9.1.1 #9 (24 hour revocation for BR violation) or if
this requirement should be softened in light of BR 4.9.5 #1 (considering
the nature of the problem when determining the time to process a
revocation request) and the fact that BR 4.9.4 (Revocation Grace Period)
is a "no stipulation". Plus a disagreement of how long the grace period,
if any, should be in the specific case of issue U1.
That TUVIT is citing ETSI equivalents to those BRs should be of much
less importance.
Below you are quoting me far out of context again:
>>
>>> It is of cause the purpose of any audit scheme to check for the absence
>>> of irregularities, and to report if any were found.
>>
>>
>> Except that's not the point of the ETSI audit, which is at least why some
>> discussion of the scheme is relevant. The "report if any were found" is
>> functionally absent. The 'reporting' is done based on whether or not the
>> certificate was issued, and the certificate is issued provided that any
>> irregularities were resolved, to the auditor's satisfaction, within 90
>> days. This is something 'unique' to ETSI audits, and not part of the
>> underlying ISO/IEC 17065.
>>
>> Other ETSI-based auditors have recognized this gap, and thus ensured
>> they've reported on irregularities. That shows how they can address the gap
>> from the bare minimum required of ETSI and instead meet the expectations of
>> the browsers consuming the reports.
>>
S11. Disagreement with TUVIT about how much the public audit summary
documents should say about detected incidents rather than blankly
stating (directly or indirectly) that all such were satisfactorily
resolved in less than 90 days. Minimum ETSI requirements may be too
weak in this area.
>>
>>> But it is quite
>>> rare for the audit to essentially redo every piece of administrative
>>> work done by the audited company.
>>>
>>
>> It's unclear your intended remark. During the sampling process, it is
>> indeed a cross-check of all the administrative work - ensuring that
>> sufficient evidence of all of the controls that exist to ensure the correct
>> functioning of that administrative work were followed, with detailed
>> analysis.
past arguments seem to presume that anything slipping past the auditor
means the audit wasn't proper.
>>
>>> The debate in bug #1391074 about the template used for ECDSA
>>> certificates is a good example. According to the bug, the ECDSA
>>> certificate profile/template was correct, but some piece of software
>>> mishandled approved ECDSA certificate requests and used the RSA
>>> certificate profile/template, for at least some of the issued
>>> certificates. An incorrect ECDSA profile/template saying to set the
>>> KeyEncryption bit should have been spotted by a configuration audit and
>>> review (by TUVIT). But code bugs are notoriously harder to spot.
>>>
>>
>> I've got no idea where you got that summary from, but that's certainly not
>> consistent with 3.3 of
>> https://bug1391074.bmoattachments.org/attachment.cgi?id=8915934
>>
don't have the capacity to do infinite recursion into every link and
attachment from your lack of providing your own list of issues.
>> An EC profile/template was not configured. All certificates, regardless of
>> key type, were configured to use the same profile.
>>
>> It is expected, of all CAs, that profiles are distinct per key type, least
>> of all due to the necessity to ensure both the input keys are appropriate
>> (e.g. correct strength) and the output certificate is correct from RFC 5280.
>>
template, but that a bug caused the PKI system to not use the right template
for at least 3 ECDSA certificates. This was in Comment #23 and more detailed
in Comment #28 (as issue C) at
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c23
https://bugzilla.mozilla.org/show_bug.cgi?id=1391074#c28
Both of those bug comments were after the report in 8911534 and were
specifically clarifying that report.
I have not found a statement as to what the bug was or if it affected
all ECDSA certificates in a logical part of the system. There are
multiple possible guesses.
>>
>>>> In the context of ETSI, each of these configuration changes -
>>> particularly
>>>> once qualified - undergoes some review; whether after the fact
>>>> (pre-qualification) or prior to such change.
>>>
>>> This is why it is interesting to look at each issue to determine if it
>>> was subject to such review by or notification to TUVIT.
>>>
>>
>> I'm not sure what you're saying. The model of certification requires this.
>> Is your framing of the question whether or not T-Systems notified TUVIT of
>> these issues? If they didn't, they were contractually negligent under the
>> ETSI model, and TUVIT should, as part of their explanation, indicate how
>> they addressed those failures. I'm taking Occam's Razor here, and presuming
>> that TUVIT was notified, as they were required to be, and that the failure
>> rests with TUVIT.
>>
scheme requires auditor involvement for every possible Administrative action,
thus each technical incident at T-Systems (the ones with U numbers) need to
be individually classified to determine what, if any, related actions should
have been notified to TUVIT and if so, why this did not result in audit
actions by TUVIT.
For example, which of the actions or inactions that lead to issue U5 (double
dots in DNS names) involved a required review by TUVIT? The answer affects
how much blame for U5 can be laid on TUVIT, but does not affect if TUVIT can
be blamed for its post hoc auditing of the U5 incident.
Same questions for each U number.
>>
>>>> Similarly, misissuance
>>>> involves a degree of notification to the CAB.
>>>
>>> Only once known (e.g. around the time of the bug reports). Because I
>>> don't think you expect a CA to notify the auditor that it is about to
>>> misissue, and then proceed to actually misissue instead of stopping
>>> itself.
>>>
>>
>> Yes, for all configuration changes post-misissuance, the CAB would have
>> been notified. One would thus reasonably expect that, as these
>> notifications increase, the CAB would take a more careful look at both
>> patterns of problems and specific root causes to ensure that future issues
>> are premptively identified, rather than retroactively remediated.
>> This is
>> why 100% misissuance is particularly concerning *given* past issues.
>>
>> Yet the substance of the discussion - the "current" issue if you will -
>> can be discussed without the lengthy debate and dissection being had in
>> your message here. That's because TUVIT can respond with regards to their
>> methodology and approach used. If that methodology *didn't* consider the
>> past incidents, then we can go through that retroactive dissection. If they
>> did, however, then we can allow TUVIT to respond as to what they did and
>> how it impacted.
rather than specific questions about methodology and approach. It is small
wonder that the responses have been of the fire extinguisher and asbestos
suits kind.
>>
>> As such, I can see limited value in the present conversation for
>> continuing this belabored dissection - we should instead focus on the
>> methodology and approach used for the most recent issue, and based on that,
>> analyze retroactively, to evaluate whether or not adequate assurance is
>> being provided.
>>
>>
>>>
>>>> As such, it is entirely
>>>> reasonable to expect a degree of supervision, as that is the value of
>>> the
>>>> certification scheme. All of this information would have been available
>>> at
>>>> the time of configuring qualified certificates, including the pattern of
>>>> issues existing when configuring profiles and templates.
>>>
>>> Should have been available doesn't mean it was available. There is
>>> always a limit to the depth of audits, and thus we need to assess if
>>> TUVIT was being sloppy, complacent, complicit or just unlucky.
>>>
>>
>> As captured above, I disagree with that, not on principle, but in
>> execution. We need more information from TUVIT, rather than attempting to
>> divine through first principles as this thread tries to do. If no further
>> information is provided, then we don't need to bother with that assessment
>> at all - an auditor who is unwilling or unable to provide reasonable
>> information is an auditor that should not be trusted.
>>
find out, rather than presume the worst possible situation and then proceed
to demand punishment according to such presumed violations.
>>
>>> It is relevant to ask, but it takes a considerable level of certainty
>>> before starting formal proceedings to disqualify an auditor due to the
>>> failings of a single audit subject.
>>>
>>> In comparison, E&Y was involved in auditing multiple bad CAs and RAs by
>>> the time some E&Y branches where disqualified by Mozilla.
>>
>>
>>> In the world of technical review and testing, TUV SUD is a major player,
>>> reviewing the safety of many technical products far outside Germany,
>>> they rank in the same league as UL in the US.
>>>
>>
>> This gets so close to understanding the issue, but then radically misses
>> the point. Multiple E&Y branches were disqualified on the basis of a single
>> audit, but E&Y is not blanket disqualified.
and the audit of the Korean RA that misissued under a Symantec CA.
>>
>> TUV IT is a specific entity, the equivalent to an E&Y "branch". Mentioning
>> TUV SUD in the context of TUV IT is akin to mentioning KPMG in a discussion
>> about E&Y. TUV IT is part of the TUV NORD group, which is itself distinct
>> from TUV SUD.
>>
But still, TUV IT seems to be the name for all of that TUV half's PKI
related audits, across all locations, similar to E&Y's global WebTrust
activities (as opposed to the WebTrust activities of E&Y's Hong Kong
branch office).
This Mozilla group has not had any reason (or purpose) to distrust E&Y
Hong Kong's auditing of financials etc., even though that is presumable
the same "entity" as the one doing bad WebTrust audits.
The difference between horizontal and vertical org charts should not
be the main dispositive issue in deciding if Mozilla should distrust
a global company or a local branch.
Wayne Thayer
Nov 14, 2018, 10:39:25 PM11/14/18
to Jakob Bohm, mozilla-dev-security-policy
It should come as no big surprise that I have documented this issue as our
first auditor compliance bug[1]:
https://bugzilla.mozilla.org/show_bug.cgi?id=1507376
I only included a brief summary of this discussion (and a link to it).
Others are welcome to comment if you feel that I have omitted crucial
information, but please don't start a parallel discussion in the bug.
Ryan and Jacob: in my opinion, you have both remained reasonably respectful
in the midst of a contentious discussion. Thank you very much for that.
While I see some small steps being made toward a common understanding of
the issue, there is still fundamental and subjective disagreement on the
severity, and it's not clear to me that this thread is headed toward any
sort of a constructive conclusion.
I would greatly appreciate everyone's input on the actions (if any, other
than documenting the issue) that Mozilla should take as a result of this
discussion.
- Wayne
[1]
https://groups.google.com/d/msg/mozilla.dev.security.policy/MOdS7CH_kU8/Kcl6E9-ZAgAJ
first auditor compliance bug[1]:
https://bugzilla.mozilla.org/show_bug.cgi?id=1507376
I only included a brief summary of this discussion (and a link to it).
Others are welcome to comment if you feel that I have omitted crucial
information, but please don't start a parallel discussion in the bug.
Ryan and Jacob: in my opinion, you have both remained reasonably respectful
in the midst of a contentious discussion. Thank you very much for that.
While I see some small steps being made toward a common understanding of
the issue, there is still fundamental and subjective disagreement on the
severity, and it's not clear to me that this thread is headed toward any
sort of a constructive conclusion.
I would greatly appreciate everyone's input on the actions (if any, other
than documenting the issue) that Mozilla should take as a result of this
discussion.
- Wayne
[1]
https://groups.google.com/d/msg/mozilla.dev.security.policy/MOdS7CH_kU8/Kcl6E9-ZAgAJ
Ryan Sleevi
Nov 15, 2018, 3:51:43 PM11/15/18
to Wayne Thayer, Jakob Bohm, mozilla-dev-security-policy
On Wed, Nov 14, 2018 at 10:39 PM Wayne Thayer via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> While I see some small steps being made toward a common understanding of
> the issue, there is still fundamental and subjective disagreement on the
> severity, and it's not clear to me that this thread is headed toward any
> sort of a constructive conclusion.
>
I think one area that I've been trying to focus on, independent of the past
dev-secur...@lists.mozilla.org> wrote:
> While I see some small steps being made toward a common understanding of
> the issue, there is still fundamental and subjective disagreement on the
> severity, and it's not clear to me that this thread is headed toward any
> sort of a constructive conclusion.
>
issues that Jakob is exploring, is a better understanding of TUViT's
processes with respect to compliance. While it's certainly true that
they've acknowledged that they have not and did not develop tools to check
compliance of certificates against the published ASN.1 modules, I think it
would benefit the community to better understand TUViT's approach to
auditing and ensuring compliance. For example, how many processes rely on
human review? Is sampling employed, how are sample sizes selected, what is
tested within the sample, etc.
These are matters that can be discussed and explored without the
retrospective analysis, and provide insight into the current issue. The
benefit of the retrospective analysis is that we can then also explore and
understand if and how these processes were changed due to past oversights,
and whether or not past oversights should have been caught by the described
processes. This helps ensure that future issues can be detected more timely.
Separate from that discussion - of the present issue - is a question about
whether or not TUViT's adherence to the minimum amount required represents
a sufficient level of assurance going forward. If, as a community, the
approach TUViT is taking is not acceptably transparent, next steps can be
explored. These next steps may include suspending TUViT's recognition until
process changes to achieve the necessary transparency are met, or may
involve clarifying more generally the degree of transparency required for
audits within the program. This may be accompanied by a further exploration
of the ETSI accreditation standards with regards to best practices. Put
differently, the demonstration of more transparent reports from other
auditors accredited under ETSI-developed standards may indicate that TUViT
is failing to meet industry best practices, or it may serve as an
opportunity to codify those best practices as program requirements.
Obviously from the discussion, I believe disagree with Jakob on the best
approach to achieving these goals. I think it's far more important and
relevant to make sure we have a comprehensive understanding of the
/current/ issue with respect to competence and transparency before
comparing and contrasting that with past issues. I think if we can spend
our energies focused on this specific issue, then we can make some forward
progress.
Wayne Thayer
Nov 16, 2018, 11:46:34 AM11/16/18
to Ryan Sleevi, Jakob Bohm, mozilla-dev-security-policy
On Thu, Nov 15, 2018 at 1:51 PM Ryan Sleevi <ry...@sleevi.com> wrote:
>
> On Wed, Nov 14, 2018 at 10:39 PM Wayne Thayer via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> While I see some small steps being made toward a common understanding of
>> the issue, there is still fundamental and subjective disagreement on the
>> severity, and it's not clear to me that this thread is headed toward any
>> sort of a constructive conclusion.
>>
>
> I think one area that I've been trying to focus on, independent of the
> past issues that Jakob is exploring, is a better understanding of TUViT's
> processes with respect to compliance. While it's certainly true that
> they've acknowledged that they have not and did not develop tools to check
> compliance of certificates against the published ASN.1 modules, I think it
> would benefit the community to better understand TUViT's approach to
> auditing and ensuring compliance. For example, how many processes rely on
> human review? Is sampling employed, how are sample sizes selected, what is
> tested within the sample, etc.
>
> I thought you were focused on the current T-Systems issues. I do see value
>
> On Wed, Nov 14, 2018 at 10:39 PM Wayne Thayer via dev-security-policy <
> dev-secur...@lists.mozilla.org> wrote:
>
>> While I see some small steps being made toward a common understanding of
>> the issue, there is still fundamental and subjective disagreement on the
>> severity, and it's not clear to me that this thread is headed toward any
>> sort of a constructive conclusion.
>>
>
> I think one area that I've been trying to focus on, independent of the
> past issues that Jakob is exploring, is a better understanding of TUViT's
> processes with respect to compliance. While it's certainly true that
> they've acknowledged that they have not and did not develop tools to check
> compliance of certificates against the published ASN.1 modules, I think it
> would benefit the community to better understand TUViT's approach to
> auditing and ensuring compliance. For example, how many processes rely on
> human review? Is sampling employed, how are sample sizes selected, what is
> tested within the sample, etc.
>
in that "case study" approach, but now it sounds like you're asking TUViT
to describe some of their methodology? Have you posed specific questions in
this area to TUViT? I just reviewed this thread but didn't notice that.
TUViT posted an incident report to the bug that does describe the testing
and sampling performed for the T-Systems audit:
https://bugzilla.mozilla.org/show_bug.cgi?id=1507376#c2
These are matters that can be discussed and explored without the
> retrospective analysis, and provide insight into the current issue. The
> benefit of the retrospective analysis is that we can then also explore and
> understand if and how these processes were changed due to past oversights,
> and whether or not past oversights should have been caught by the described
> processes. This helps ensure that future issues can be detected more timely.
>
>
Separate from that discussion - of the present issue - is a question about
> whether or not TUViT's adherence to the minimum amount required represents
> a sufficient level of assurance going forward. If, as a community, the
> approach TUViT is taking is not acceptably transparent, next steps can be
> explored. These next steps may include suspending TUViT's recognition until
> process changes to achieve the necessary transparency are met, or may
> involve clarifying more generally the degree of transparency required for
> audits within the program. This may be accompanied by a further exploration
> of the ETSI accreditation standards with regards to best practices. Put
> differently, the demonstration of more transparent reports from other
> auditors accredited under ETSI-developed standards may indicate that TUViT
> is failing to meet industry best practices, or it may serve as an
> opportunity to codify those best practices as program requirements.
>
continue to support the creation of more program requirements around audits
and audit reporting. Do you have any suggestions for how we can make
progress on that?
Obviously from the discussion, I believe disagree with Jakob on the best
> approach to achieving these goals. I think it's far more important and
> relevant to make sure we have a comprehensive understanding of the
> /current/ issue with respect to competence and transparency before
> comparing and contrasting that with past issues. I think if we can spend
> our energies focused on this specific issue, then we can make some forward
> progress.
>
distinguish poor judgement from policy violations. To that end, I think
Nick's request for us to better define root program expectations is a
reasonable one. Analyzing current and past issues can certainly help us to
define these requirements.
Nick Pope
Nov 19, 2018, 10:25:18 AM11/19/18
to mozilla-dev-s...@lists.mozilla.org
Restating my earlier offer we would welcome a clear statement of any concerns or wishes resulting from the discussions, on this or other related threads, against the measures already proposed in TS 119 403-2 and its parent standard. We can then discuss this with the European stakeholders and see how we could best answer these concerns
Nick
On Friday, November 16, 2018 at 4:46:34 PM UTC, Wayne Thayer wrote:
> On Thu, Nov 15, 2018 at 1:51 PM Ryan Sleevi <ry...@sleevi.com> wrote:
>
...
Nick
On Friday, November 16, 2018 at 4:46:34 PM UTC, Wayne Thayer wrote:
> On Thu, Nov 15, 2018 at 1:51 PM Ryan Sleevi <ry...@sleevi.com> wrote:
>
Wayne Thayer
Nov 19, 2018, 6:01:54 PM11/19/18
to Nick Pope, mozilla-dev-security-policy
Hi Nick,
I had been thinking that 119 403-2 was just intended as an attestation
statement template, similar to the WebTrust reporting guidance [1]. Now I
understand that it can include more substantial requirements.
This is certainly not a complete list, but specific to this discussion I
would start with the following concerns:
* Reporting on major and minor non-conformities - I have yet to ever see an
ETSI attestation listing a major non-conformity, but I have shared several
examples listing minor non-conformities with ETSI representatives. We need
standards that require consistent disclosure of all types of
non-conformities in attestation statements. Disclosure is required even if
a CA fixes a non-conformity within an acceptable time frame (based on ETSI
standards).
* Disclosure when a CA violates the BR revocation timeline requirements,
even if their actions are perfectly acceptable under ETSI standards for
remediation.
* Disclosure of testing and sampling methodologies used in an audit.
- Wayne
[1] http://www.webtrust.org/practitioner-qualifications/item64422.aspx
On Mon, Nov 19, 2018 at 8:25 AM Nick Pope via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> Restating my earlier offer we would welcome a clear statement of any
> concerns or wishes resulting from the discussions, on this or other related
> threads, against the measures already proposed in TS 119 403-2 and its
> parent standard. We can then discuss this with the European stakeholders
> and see how we could best answer these concerns
>
> Nick
>
> On Friday, November 16, 2018 at 4:46:34 PM UTC, Wayne Thayer wrote:
> ...
I had been thinking that 119 403-2 was just intended as an attestation
statement template, similar to the WebTrust reporting guidance [1]. Now I
understand that it can include more substantial requirements.
This is certainly not a complete list, but specific to this discussion I
would start with the following concerns:
* Reporting on major and minor non-conformities - I have yet to ever see an
ETSI attestation listing a major non-conformity, but I have shared several
examples listing minor non-conformities with ETSI representatives. We need
standards that require consistent disclosure of all types of
non-conformities in attestation statements. Disclosure is required even if
a CA fixes a non-conformity within an acceptable time frame (based on ETSI
standards).
* Disclosure when a CA violates the BR revocation timeline requirements,
even if their actions are perfectly acceptable under ETSI standards for
remediation.
* Disclosure of testing and sampling methodologies used in an audit.
- Wayne
[1] http://www.webtrust.org/practitioner-qualifications/item64422.aspx
On Mon, Nov 19, 2018 at 8:25 AM Nick Pope via dev-security-policy <
dev-secur...@lists.mozilla.org> wrote:
> Restating my earlier offer we would welcome a clear statement of any
> concerns or wishes resulting from the discussions, on this or other related
> threads, against the measures already proposed in TS 119 403-2 and its
> parent standard. We can then discuss this with the European stakeholders
> and see how we could best answer these concerns
>
> Nick
>
> On Friday, November 16, 2018 at 4:46:34 PM UTC, Wayne Thayer wrote:
Nick Pope
Nov 21, 2018, 10:20:51 AM11/21/18
to mozilla-dev-s...@lists.mozilla.org
Wayne,
We will work on how to address this within the EU audit framework based on EN 319 403.
TS 119 403-2 already includes some additional requirements specifically for PTC to cover the CABF concerns for yearly audit and period of time audit review of events since the previous audit. An update to this document to address these concerns is an option that we will consider.
My I ask for you patience and forbearance in the time taken to carry out this work.
Nick
We will work on how to address this within the EU audit framework based on EN 319 403.
TS 119 403-2 already includes some additional requirements specifically for PTC to cover the CABF concerns for yearly audit and period of time audit review of events since the previous audit. An update to this document to address these concerns is an option that we will consider.
My I ask for you patience and forbearance in the time taken to carry out this work.
Nick
Nick Pope
Feb 12, 2019, 11:06:20 AM2/12/19
to mozilla-dev-s...@lists.mozilla.org
Wayne,
We have discussed your concerns with EU stakeholders and offer the following responses. These concerns have been discussed with Webtrust and we believe that the proposals below are in line with existing accepted practices.
Comment 1) We need standards that require consistent disclosure of all types of non-conformities in attestation statements.
Response
The current requirement in EN 319 403 is a TSP audit may be passed with pending non-conformities provided that these [non-conformities] do not impact the ability of the TSP to meet the intended service. If there are other non-conformities no Audit Attesttion is issued.
It is proposed that ETSI extend the requirements in TS 119 403-2 to require the Audit Attestation for Publicly Trusted Certificates to include a statement on each sub-clause of the referenced requirements where there is a finding of nonconformity which does not impact the ability of the TSP to meet its intended service.
Comment 2) Disclosure is required even if a CA fixes a non-conformity within an acceptable time frame (based on ETSI standards).
Response
As above, it is proposed that ETSI extend the requirements in TS 119 403-2 to require the Audit Attestation for Publicly Trusted Certificates to include a statement on each sub-clause of the referenced requirements where there is a finding of nonconformity noted during the audit which had been corrected prior issuing the Audit Attestation.
Comment 3) Disclosure when a CA violates the BR revocation timeline requirements, even if their actions are perfectly acceptable under ETSI standards for remediation.
Response
It is not clear to where the BR revocation requirements differ from those of the ETSI standards which would allow violation of timeline requirements to be considered as non-conformant in a way which does do not impact the ability of the TSP to meet the intended service, and hence can come under the allowance for time to remediate.
In any case such a situation would be covered by the proposed change in item 1)
Comment 4) Disclosure of testing and sampling methodologies used in an audit.
ETSI follows the practice it is understood is followed by Webtrust. Information on testing and sampling methodologies is available in detailed reports but is not disclosed in to the public in any detail.
Nick Pope
Vice Chair ETSI TC Electronic Signatures and Infrastructures
We have discussed your concerns with EU stakeholders and offer the following responses. These concerns have been discussed with Webtrust and we believe that the proposals below are in line with existing accepted practices.
Comment 1) We need standards that require consistent disclosure of all types of non-conformities in attestation statements.
Response
The current requirement in EN 319 403 is a TSP audit may be passed with pending non-conformities provided that these [non-conformities] do not impact the ability of the TSP to meet the intended service. If there are other non-conformities no Audit Attesttion is issued.
It is proposed that ETSI extend the requirements in TS 119 403-2 to require the Audit Attestation for Publicly Trusted Certificates to include a statement on each sub-clause of the referenced requirements where there is a finding of nonconformity which does not impact the ability of the TSP to meet its intended service.
Comment 2) Disclosure is required even if a CA fixes a non-conformity within an acceptable time frame (based on ETSI standards).
Response
As above, it is proposed that ETSI extend the requirements in TS 119 403-2 to require the Audit Attestation for Publicly Trusted Certificates to include a statement on each sub-clause of the referenced requirements where there is a finding of nonconformity noted during the audit which had been corrected prior issuing the Audit Attestation.
Comment 3) Disclosure when a CA violates the BR revocation timeline requirements, even if their actions are perfectly acceptable under ETSI standards for remediation.
Response
It is not clear to where the BR revocation requirements differ from those of the ETSI standards which would allow violation of timeline requirements to be considered as non-conformant in a way which does do not impact the ability of the TSP to meet the intended service, and hence can come under the allowance for time to remediate.
In any case such a situation would be covered by the proposed change in item 1)
Comment 4) Disclosure of testing and sampling methodologies used in an audit.
ETSI follows the practice it is understood is followed by Webtrust. Information on testing and sampling methodologies is available in detailed reports but is not disclosed in to the public in any detail.
Nick Pope
Vice Chair ETSI TC Electronic Signatures and Infrastructures
0 new messages