Policy 2.8: MRSP Issue #219: Require ETSI auditors to be ACAB-c members

519 views
Skip to first unread message

Ben Wilson

unread,
Dec 14, 2021, 5:37:49 PM12/14/21
to dev-secur...@mozilla.org

All,

This email starts discussion of whether ETSI auditors should be required to be members of the Accredited Conformity Assessment Bodies' Council (“ACAB’c” - https://www.acab-c.com/).

This is Issue #219 for the Mozilla Root Store Policy (MSRP), version 2.8, to be published in 2022. (See https://github.com/mozilla/pkipolicy/labels/2.8)

Mozilla continually seeks to improve the quality of CA audits. Therefore, we are considering a requirement that ETSI auditors be members of the ACAB’c, for which there is no cost to join. The ACAB’c has improved the consistency in how audit reports are provided to Mozilla, including how auditor qualifications are verified. (ACAB’c seeks “to harmonise the application of the conformity assessment requirements … with regard to the broader conformity assessment community and in partnership with the main stakeholders of the area, such as [the] CA/Browser Forum ….”  Members of the ACAB’c further undertake to meet “the minimum report content for … Browsers Manufacturers”.  (Code of Conduct, found at https://www.acab-c.com/terms-conditions-and-policies/.) Not only has ACAB’c maintained a Mozilla-compliant audit attestation letter template, but it has also provided guidance about what auditors are supposed to check, and it has taken other steps to keep audits current with Mozilla and CA/Browser Forum requirements.


From an audit quality standpoint, membership in the ACAB'c is necessary for any auditor using ETSI criteria to review CAs that issue publicly trusted server certificates, and therefore, ACAB'c membership should be a requirement stated in the MRSP.


Please provide your responses and comments in this thread.  Thanks.


Sincerely,


Ben Wilson

Mozilla Root Store Program

Moudrick Dadashov

unread,
Dec 14, 2021, 7:06:30 PM12/14/21
to Ben Wilson, dev-secur...@mozilla.org
With all due respect to ACAB-c,  currently the term CAB means a proffesional accredited by NAB.

I'd suggest to consult with the legal department if the proposed requirement comply with Article 11 ( Freedom of assembly and association) of European Convention  on Human Rights:

1.  Everyone  has  the  right  to  freedom  of  peaceful  assembly  and to freedom of association with others, including the right to form and to join trade unions for the protection of his interests.

2.  No restrictions shall be placed  on the exercise of  these rights other than such as are prescribed by law and are necessary in a democratic society in the interests  of national security  or public safety, for the prevention of disorder or crime, for the protection of health or morals or for the protection of the rights and freedoms of others. This Article shall not prevent the imposition of lawful restrictions  on the exercise of these rights by members  of the armed forces, of the police or of the administration of the State.

Thanks,
M.D.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/CA%2B1gtaYuv_0Zy4LZnxPkmbg9EGft6AtT3AXSSUM2Es7VWuUPgw%40mail.gmail.com.

Ben Wilson

unread,
Jan 25, 2022, 10:53:40 PM1/25/22
to dev-secur...@mozilla.org
I am proposing that we make this a "SHOULD".  ETSI auditors SHOULD be members of ACAB'c

See draft language here:

"ETSI auditors SHOULD be members of the [Accredited Conformity Assessment Bodies' Council][ACAB'c link].  WebTrust auditors MUST be [enrolled in the WebTrust program][WebTrust link]."

Ryan Sleevi

unread,
Jan 26, 2022, 1:00:29 AM1/26/22
to Ben Wilson, dev-secur...@mozilla.org
It would seem better for Mozilla users if it was a MUST. A SHOULD is an interesting starting point, but I’m not sure it does anything to help members of the community here, and there don’t seem to be clear arguments against it.

The benefit, of course, is attempting to ensure better consistency and aligning with the needs of Mozilla, which accredited CABs alone are not necessarily qualified nor incentivized to do, but at least ACAB-c has been willing to try.

Ben Wilson

unread,
Jan 26, 2022, 1:37:25 PM1/26/22
to Ryan Sleevi, dev-secur...@mozilla.org
I agree that a "MUST" is better. Does anyone have a stronger case for making it a "SHOULD"?

Moudrick Dadashov

unread,
Jan 26, 2022, 1:57:46 PM1/26/22
to Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org
Does it make sense to check if the surrogate QESC issuer is audited by an ACAB-C member?

Thanks,
M.D.

Ryan Sleevi

unread,
Jan 26, 2022, 2:38:47 PM1/26/22
to Moudrick Dadashov, Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org
Where would the surrogate QESC issuer apply? For S/MIME?

Ben Wilson

unread,
Feb 1, 2022, 6:06:53 PM2/1/22
to dev-secur...@mozilla.org
Here is another Github commit making ACAB'c membership mandatory.   Note:  It also requires WebTrust practitioners to be "enrolled" by CPA Canada in the WebTrust for Certification Authorities program.
 

Tim Hollebeek

unread,
Feb 3, 2022, 1:02:27 PM2/3/22
to Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org

This would effectively force a number of existing auditors with a long history of providing ETSI audits for Mozilla into joining ACAB-C.  It is not clear that simply being a member provides any benefits.  If there are clear problems to be solved here, it would be better to write explicit requirements about what is expected of auditors, instead of requiring their membership in an arbitrary organization.

 

As far as I’m aware, ACAB-C is a voluntary coordination body, and not in any way recognized as part of the European regulatory structure.

 

-Tim

 

Ryan Sleevi

unread,
Feb 3, 2022, 2:10:54 PM2/3/22
to Tim Hollebeek, Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org
Tim:

Why is recognizing as part of the European regulatory structure necessary, or even desirable? It has different goals for different users, which is fine. Similary, WebTrust is not recognized by any regulatory structure, and that doesn’t cause issues.

You state “it would be better”, but provide no justification or evidence to support that claim. It’s already been addressed why there are benefits.

It sounds as if you’re objecting on the basis of “this changes things”, and that’s not a substantive concern. Nor have you demonstrated why Mozilla, or anyone, must bear the burden of proof for convincing you of the benefit.

The recognition by European regulatory structure provides zero technical or policy benefit to browser root programs. In particular, there is no functioning accountability for any misrepresentation to browsers that is not also made to regulatory authorities. The work product of CABs provides to SBs is already different than that provided to browsers, so it’s somewhat questionable to assert any bearing at all.

Compared to ACAB-c, which has offered to self-regulate with respect to browser needs, and has actively worked to try and engage on providing some degree of inter-member understanding and consistency with respect to reporting.

It is not clear that your objection has any merit. If there are clear problems with joining ACAB-c, it would be better to write explicit requirements about why that is, instead of merely objecting.

Tim Hollebeek

unread,
Feb 3, 2022, 2:44:22 PM2/3/22
to ry...@sleevi.com, Ben Wilson, dev-secur...@mozilla.org

It is disrespectful to imply that my remarks mean anything beyond what I actually said.  This has been a pattern for a long time, where you reply quickly to my feedback, replace it with a strawman, and then argue against the strawman.  In particular, the WebTrust analogy is not very apt.  WebTrust is a compliance program that Mozilla relies upon, so it makes a lot of sense for auditors to have to be accredited under it and in good standing.  As far as I’m aware, ACAB-C is not an accreditation body for ETSI audits in anyway. 

 

I have asked you, repeatedly and publicly, and on many occasions, to not put words in my mouth when you reply.  Yet you persist.  This sort of behavior is why I rarely participate in this forum any more.  Particularly relevant to this forum, it’s also a violation of Mozilla Policy.

 

Clearly, it would be better, for example, if Mozilla desires that auditors use the ACAB-C reporting format, then they should require that. Forcing every auditor, including those who are government regulators and for whom this relationship might be awkward, into joining an organization simply to check a box on Mozilla’s compliance list, will not improve anything.  People will join for the checkbox, and then ignore the organization and not participate.

 

-Tim

Ryan Sleevi

unread,
Feb 3, 2022, 3:06:32 PM2/3/22
to Tim Hollebeek, ry...@sleevi.com, Ben Wilson, dev-secur...@mozilla.org
Tim,

As much as I disdain the tone policing here, it's certainly notmy intent to put words in your mouth. I had hoped it was clear in my previous message that I was responding to how your message was received, and clarifying it as such, in the hopes of ensuring you could clarify if it was not what you intended. You call that a strawman or putting words in your mouth, but I do struggle to think how we can better communicate, if we cannot articulate "When you say X, I hear this, is that what you mean?"

I think you can agree that when arguments are unclear, or unsubstantiated, it is difficult, if not impossible, to productively engage. I do hope we can find a better way of working together, and in particular, ensure concerns are well articulated in a way that reduces the likelihood of misinterpretation. However, I'm not sure it's fair, or reasonable, to suggest that it's somehow a violation of policy to highlight when your arguments are unclear, how they were interpreted, and what the problems may be with that interpretation. If you did not intend for it as such, it's a welcome opportunity for you to both clarify your arguments, and to better understand my own concerns. That's not strawmanning - that's trying to reach a shared understanding, and I hope you can agree, that's very valuable.

To that point, I think you may misunderstand what WebTrust is. It is not, as you suggest, an accreditation program, but a licensing program, in which a brand is licensed based on an expectation of meeting some quality control objectives. I'm hoping that if you believe ACAB-c is functionally different, you might be able to further articulate where that difference is, as it would appear to be functionally identical.

Your response to suggest that the goal is "simply to check a box on Mozilla's compliance list" seems to do the same disservice to the argument I presented you, as you accuse me of doing. I hope you can see how I articulated that the purpose is not merely checking a box, and how the affiliation has value. Perhaps this misunderstanding of my previous e-mail, and the misunderstanding of what WebTrust is, has compounded the miscommunications here?

Ben Wilson

unread,
Feb 3, 2022, 3:42:34 PM2/3/22
to Ryan Sleevi, Tim Hollebeek, dev-secur...@mozilla.org
I think the ACAB'c provides benefits that bring ETSI audits closer to what we see from WebTrust and Webtrust audits, even though the two organizations aren't similar in all respects. Both are needed or are "Must" haves for Mozilla. ACAB'c doesn't cost anything to join, it provides good templates that are responsive to Mozilla's needs, and it also makes it easier to determine if the CAB is accredited or not, among other benefits it provides.

Tim Hollebeek

unread,
Feb 3, 2022, 4:03:27 PM2/3/22
to Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org

Ben,

 

The policy requirements should be structured to match the policy goals.  You have mentioned two important ones, which I agree with.  The first can be solved by requiring the use of ACAB’c templates.  The second points to a legitimate issue that the NABs/CABs need to solve.  Relying on a non-official source for accreditation information has its own risks that should be taken seriously.

 

There’s also no guarantee that ACAB’C membership will be free in the future.  Organizations change.  ACAB’c could also adopt membership rules which some organizations are unable to comply with.

 

As was pointed out internally, ACAB’C is a very small association of mostly French and German auditors, with very few members.  As much as I appreciate their work on templates and other issues, I don’t think forcing people to join another organization is a good thing for organizations to do, no matter how well-intended it is.  It takes away their agency, which will certainly put a damper on their desire to participate.

 

-Tim

Ryan Sleevi

unread,
Feb 3, 2022, 5:16:39 PM2/3/22
to Tim Hollebeek, Ben Wilson, Ryan Sleevi, dev-secur...@mozilla.org
On Thu, Feb 3, 2022 at 4:03 PM Tim Hollebeek <tim.ho...@digicert.com> wrote:

Ben,

 

The policy requirements should be structured to match the policy goals.  You have mentioned two important ones, which I agree with.  The first can be solved by requiring the use of ACAB’c templates.  The second points to a legitimate issue that the NABs/CABs need to solve.  Relying on a non-official source for accreditation information has its own risks that should be taken seriously.


Tim,

I don't want to belabor this point, but you haven't highlighted if, how, or why you believe WebTrust is different. WebTrust is organizationally and functionally the same as ACAB'c in this regard, as far as professional association goes. Do you believe WebTrust is only valid if the US or Canadian governments recognize it - knowing full well they reject such audits as being insufficient?

This reply seems to demonstrate a fundamental misunderstanding about the role of CABs/NABs, or that there is some value that is not yet articulated. The burden of proof rests on you to demonstrate what this value is - and what these risks are, that you believe should be taken seriously. You have not yet done that.
 

There’s also no guarantee that ACAB’C membership will be free in the future.  Organizations change.  ACAB’c could also adopt membership rules which some organizations are unable to comply with.


Again, how is this functionally different from WebTrust, which charges a licensing fee and which has restrictions on who can join? This is a point that goes back 20 years, in particular, during the discussion of Scott Perry as an auditor who was not WebTrust licensed at the time and not a CPA. I mention Scott as an example, because Scott S. Perry is who DigiCert has used as their auditor (and which was recently acquired by Shellman).

The argument here does not establish why Mozilla should be concerned about free or not. Similarly, the point that ACAB'c "could" do something is nothing more that unsubstantiated FUD, because it ignores the fact that if there was a negative development, Mozilla - or anyone else - could respond if necessary.

As was pointed out internally, ACAB’C is a very small association of mostly French and German auditors, with very few members.  As much as I appreciate their work on templates and other issues, I don’t think forcing people to join another organization is a good thing for organizations to do, no matter how well-intended it is.  It takes away their agency, which will certainly put a damper on their desire to participate.


This is the closest we've got to actually establishing the substance of your objection, but it is entirely unclear what bearing it should have on this discussion. By this logic, requiring WebTrust licensed auditors is an equally unacceptable imposition - do you agree or not?

Is there some point you believe is being overlooked? This message is full of conclusions, but lacks the logical footing necessary to reach those conclusions. If you think it's being misunderstood, please articulate.

The fact that NABs/CABs have not solved this issue, that there has been years of discussion with ETSI, and that fundamentally the organizational goals of NABs/CABs is specifically to support that of Supervisory Bodies, and is not aligned with browser needs, appears to be entirely discarded here. There's zero reason to believe that continuing on the present course is somehow going to lead somewhere differently, other than in the abstract ideal state.

I don't disagree that there are arguments being made here, but their arguments that are easily refuted, or which don't logically hold. I hope I'm overlooking something.

Ben Wilson

unread,
Feb 3, 2022, 5:31:39 PM2/3/22
to Ryan Sleevi, Tim Hollebeek, dev-secur...@mozilla.org
Regarding "Relying on a non-official source for accreditation information has its own risks that should be taken seriously." - That isn't how it works - in the third column over on https://www.acab-c.com/members/, the link is to the official source, which is what we review.

Moudrick Dadashov

unread,
Feb 3, 2022, 6:18:15 PM2/3/22
to Ben Wilson, Ryan Sleevi, Tim Hollebeek, dev-secur...@mozilla.org
Maybe someone from ACAB-c could check the links in the table that give 404?

Thanks,
M.D.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Wiedenhorst, Matthias

unread,
Feb 4, 2022, 2:53:41 AM2/4/22
to dev-secur...@mozilla.org, m.m.da...@gmail.com

Hi Moudrick,

 

yes, we are aware of that and have requested those members to provide updated information some weeks ago. I agree, we should send out another reminder and demand to provide the information asap.

 

The problem was caused by the German accreditor that decided to re-structure their website, causing all the existing links to fail.

 

Best regards

Matthias

 

 

Von: dev-secur...@mozilla.org <dev-secur...@mozilla.org> Im Auftrag von Moudrick Dadashov
Gesendet: Freitag, 4. Februar 2022 00:18
An: Ben Wilson <bwi...@mozilla.com>
Cc: Ryan Sleevi <ry...@sleevi.com>; Tim Hollebeek <tim.ho...@digicert.com>; dev-secur...@mozilla.org <dev-secur...@mozilla.org>
Betreff: Re: Policy 2.8: MRSP Issue #219: Require ETSI auditors to be ACAB-c members

 

**WARNING** This email originates from an external sender. Please be careful when opening links and attachments!
**ACHTUNG** Diese E-Mail wurde von einem externen Sender verschickt. Bitte seien Sie vorsichtig beim Oeffnen von Internet-Links und Anhaengen!


______________________________________________________________________________________________________________________
Sitz der Gesellschaft/Headquarter: TÜV Informationstechnik GmbH * Am TÜV 1 * 45307 Essen, Germany
Registergericht/Register Court: Amtsgericht/Local Court Essen * HRB 11687 * USt.-IdNr./VAT No.: DE 176132277 * Steuer-Nr./Tax No.: 111/57062251
Geschäftsführung/Management Board: Dirk Kretzschmar 

TÜV NORD GROUP
Expertise for your Success
Please visit our website: www.tuv-nord.com
Besuchen Sie unseren Internetauftritt: www.tuev-nord.de

Adriano Santoni

unread,
Mar 29, 2022, 11:03:30 AM3/29/22
to dev-secur...@mozilla.org

It is not clear to me whether a decision has been made on this matter. Would Mozilla please clarify? If this new requirement were introduced in the MRSP with immediate effect, it would cause non trivial organizational problems for the CAs that are nearing their next audit cycle.

Adriano

ACTALIS S.p.A.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Ben Wilson

unread,
Mar 29, 2022, 11:35:27 AM3/29/22
to dev-secur...@mozilla.org
Adriano,

Right now, we're considering the following language:

"ETSI Audit Attestation Letters MUST follow the Audit Attestation Letter template on the [ACAB'c website](https://www.acab-c.com/downloads), and
ETSI auditors SHOULD be listed as [CAB-members on the ACAB'c website](https://www.acab-c.com/members/). WebTrust audit statements
MUST follow the practitioner guidance, principles, and illustrative assurance reports on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/principles-and-criteria), and SHOULD be listed as an enrolled WebTrust practitioner on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/licensed-webtrust-practitioners-international)."

Thanks,

Ben

Ben Wilson

unread,
Apr 4, 2022, 11:59:48 AM4/4/22
to dev-secur...@mozilla.org

Ryan Sleevi

unread,
Apr 5, 2022, 9:21:10 AM4/5/22
to Ben Wilson, dev-secur...@mozilla.org
Ben:

As a whole, this change seems a significant step backwards, in that it removes the requirement for both WebTrust licensee and ACAB'c membership. There doesn't seem to be any explanation for this change, and your reply on Feb 3 seemed to support.

In short, it's unclear how this addresses https://github.com/mozilla/pkipolicy/issues/219 - it seems to do quite the opposite.

Maybe if we take a step back from your precise wording changes: What's the end state you'd like to accomplish? It seems this does the opposite of what's on the bug, and if that's intended, it might be useful to have some rationale and discussion on that.

Kathleen Wilson

unread,
Apr 5, 2022, 12:19:08 PM4/5/22
to dev-secur...@mozilla.org, Ryan Sleevi, Ben Wilson
The problem that we ran into over the past year is that there can be business or other reasons that impact when a company like CPA Canada will enter into agreements (or end agreements) with other companies. So, while our desire is to require auditors to be either members of ACAB'c or listed on the CPA Canada website, there may be business reasons not related to CAs/PKI for which such relationships cannot be established or continued. We also learned over the past year that an auditor can be removed from such membership/list after they have already started or even finished the audit of the CA for that year, even when that auditor has been on the list for several previous years and has not done anything to warrant being removed.

Maybe we can replace the "SHOULD" with  "MUST (unless written permission is granted by Mozilla)"...

I'm not a fan of that type of wording, but at least it would be stronger than the "SHOULD", and would still enable us to handle certain situations that we have been running into without having to grant exceptions to written policy.

I would also prefer to say "prior written permission", but we ran into situations in which the audits and audit statements had already been completed before the auditor was removed from the membership/list (to no fault of their own).

So the text could become:

"ETSI Audit Attestation Letters MUST follow the Audit Attestation Letter template on the [ACAB'c website](https://www.acab-c.com/downloads), and
ETSI auditors MUST (unless written permission is granted by Mozilla) be listed as [CAB-members on the ACAB'c website](https://www.acab-c.com/members/). WebTrust audit statements
MUST follow the practitioner guidance, principles, and illustrative assurance reports on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/principles-and-criteria), and MUST (unless written permission is granted by Mozilla) be listed as an enrolled WebTrust practitioner on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/licensed-webtrust-practitioners-international)."

Kathleen


On Tuesday, April 5, 2022 at 6:21:10 AM UTC-7 Ryan Sleevi wrote:
Ben:

As a whole, this change seems a significant step backwards, in that it removes the requirement for both WebTrust licensee and ACAB'c membership. There doesn't seem to be any explanation for this change, and your reply on Feb 3 seemed to support.

In short, it's unclear how this addresses https://github.com/mozilla/pkipolicy/issues/219 - it seems to do quite the opposite.

Maybe if we take a step back from your precise wording changes: What's the end state you'd like to accomplish? It seems this does the opposite of what's on the bug, and if that's intended, it might be useful to have some rationale and discussion on that.

On Mon, Apr 4, 2022 at 11:59 AM Ben Wilson <bwi...@mozilla.com> wrote:
On Tue, Mar 29, 2022 at 9:35 AM Ben Wilson <bwi...@mozilla.com> wrote:
Adriano,

Right now, we're considering the following language:

"ETSI Audit Attestation Letters MUST follow the Audit Attestation Letter template on the [ACAB'c website](https://www.acab-c.com/downloads), and
ETSI auditors SHOULD be listed as [CAB-members on the ACAB'c website](https://www.acab-c.com/members/). WebTrust audit statements
MUST follow the practitioner guidance, principles, and illustrative assurance reports on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/principles-and-criteria), and SHOULD be listed as an enrolled WebTrust practitioner on the [CPA Canada website](https://www.cpacanada.ca/en/business-and-accounting-resources/audit-and-assurance/overview-of-webtrust-services/licensed-webtrust-practitioners-international)."

Thanks,

Ben

On Tue, Mar 29, 2022 at 9:03 AM 'Adriano Santoni' via dev-security-policy@mozilla.org <dev-security-policy@mozilla.org> wrote:

It is not clear to me whether a decision has been made on this matter. Would Mozilla please clarify? If this new requirement were introduced in the MRSP with immediate effect, it would cause non trivial organizational problems for the CAs that are nearing their next audit cycle.

Adriano

ACTALIS S.p.A.


Il 03/02/2022 23:31, Ben Wilson ha scritto:
Regarding "Relying on a non-official source for accreditation information has its own risks that should be taken seriously." - That isn't how it works - in the third column over on https://www.acab-c.com/members/, the link is to the official source, which is what we review.

On Thu, Feb 3, 2022 at 3:16 PM Ryan Sleevi <ry...@sleevi.com> wrote:


On Thu, Feb 3, 2022 at 4:03 PM Tim Hollebeek <tim.ho...@digicert.com> wrote:

Ben,

 

The policy requirements should be structured to match the policy goals.  You have mentioned two important ones, which I agree with.  The first can be solved by requiring the use of ACAB’c templates.  The second points to a legitimate issue that the NABs/CABs need to solve.  Relying on a non-official source for accreditation information has its own risks that should be taken seriously.


Tim,

I don't want to belabor this point, but you haven't highlighted if, how, or why you believe WebTrust is different. WebTrust is organizationally and functionally the same as ACAB'c in this regard, as far as professional association goes. Do you believe WebTrust is only valid if the US or Canadian governments recognize it - knowing full well they reject such audits as being insufficient?

This reply seems to demonstrate a fundamental misunderstanding about the role of CABs/NABs, or that there is some value that is not yet articulated. The burden of proof rests on you to demonstrate what this value is - and what these risks are, that you believe should be taken seriously. You have not yet done that.
 

There’s also no guarantee that ACAB’C membership will be free in the future.  Organizations change.  ACAB’c could also adopt membership rules which some organizations are unable to comply with.


Again, how is this functionally different from WebTrust, which charges a licensing fee and which has restrictions on who can join? This is a point that goes back 20 years, in particular, during the discussion of Scott Perry as an auditor who was not WebTrust licensed at the time and not a CPA. I mention Scott as an example, because Scott S. Perry is who DigiCert has used as their auditor (and which was recently acquired by Shellman).

The argument here does not establish why Mozilla should be concerned about free or not. Similarly, the point that ACAB'c "could" do something is nothing more that unsubstantiated FUD, because it ignores the fact that if there was a negative development, Mozilla - or anyone else - could respond if necessary.

As was pointed out internally, ACAB’C is a very small association of mostly French and German auditors, with very few members.  As much as I appreciate their work on templates and other issues, I don’t think forcing people to join another organization is a good thing for organizations to do, no matter how well-intended it is.  It takes away their agency, which will certainly put a damper on their desire to participate.


This is the closest we've got to actually establishing the substance of your objection, but it is entirely unclear what bearing it should have on this discussion. By this logic, requiring WebTrust licensed auditors is an equally unacceptable imposition - do you agree or not?

Is there some point you believe is being overlooked? This message is full of conclusions, but lacks the logical footing necessary to reach those conclusions. If you think it's being misunderstood, please articulate.

The fact that NABs/CABs have not solved this issue, that there has been years of discussion with ETSI, and that fundamentally the organizational goals of NABs/CABs is specifically to support that of Supervisory Bodies, and is not aligned with browser needs, appears to be entirely discarded here. There's zero reason to believe that continuing on the present course is somehow going to lead somewhere differently, other than in the abstract ideal state.

I don't disagree that there are arguments being made here, but their arguments that are easily refuted, or which don't logically hold. I hope I'm overlooking something.
--
You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsub...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsub...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsub...@mozilla.org.

Moudrick Dadashov

unread,
Apr 5, 2022, 1:54:27 PM4/5/22
to Kathleen Wilson, dev-secur...@mozilla.org, Ryan Sleevi, Ben Wilson
Just trying to see how harmonized the auditor requirements for variuose regional systems (e.g. North American vs European) are.

If CPA Canada is like a NAB in Europe, then what would be an  analog for ACAB?

Thanks,
M.D.



You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/0f4a08f2-a967-4b0c-84a0-215b2c9c87afn%40mozilla.org.

Ryan Sleevi

unread,
Apr 5, 2022, 1:57:24 PM4/5/22
to Kathleen Wilson, dev-secur...@mozilla.org, Ryan Sleevi, Ben Wilson
That explanation makes more sense, although there are sharp edges. For example, an audit conducted against the WebTrust Principles won't necessarily have the same legal implications with respect to professional conduct and standards. You could easily imagine a jurisdiction that, say, has requirements on attestation audits that conflict with expectations of ISAE 3000 / IFAC, and thus, CPA Canada may be unwilling to enter in a licensing agreement with entities within that jurisdiction. Those case-by-case situations require significant understanding of the audit standards within that jurisdiction to be confident that the material is providing the desired degree of assurance, and that's an expensive investigation.

Maybe, instead of following with the "(unless written permission is granted by Mozilla)" for both, perhaps:

- Mozilla MAY, at its sole discretion, decide to temporarily waive membership or enrollment requirements.

I understand not wanting to penalize CAs that began audits in good faith, especially if they find themselves needing to look for a new auditor. However, it seems like the exceptional case should be rare, and also temporary, since there are broader questions about the degree of supervision and good-standing that require careful, nuanced investigations.

Either way, it does seem a marked improvement over the current language, as otherwise, I could find a lucrative career as a personal auditor who uses the guidance, principles, and reports to grant everyone a solid "You're awesome", without much legal recourse, depending on which jurisdiction I decide to base my practice in :)

You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Dimitris Zacharopoulos

unread,
Apr 5, 2022, 2:57:44 PM4/5/22
to dev-secur...@mozilla.org
Hi Moudrick,

CPA Canada is not like a NAB in Europe. NABs supervise CABs which means they assess/examine/review audit work of CABs - I believe - on a yearly basis and decide on the accreditation of the CAB. CABs are "assessed" by NABs similarly as CAs are audited by CABs.

I am not sure CPA Canada works this way. Based on past discussions, please correct me if I'm wrong, my understanding is that WebTrust audit firms have a peer-review process (not sure if it is annual or not) which means that audit firms examine other audit firms' audits.

IMHO CPA Canada is more analogous to ACAB-c than to a NAB but with more "power"/authority over the WebTrust program (closer to the powers of EA https://european-accreditation.org/). ACAB-c has no authority over NABs/CABs or ETSI standards.


Best regards,
Dimitris.

Kathleen Wilson

unread,
Apr 5, 2022, 3:41:21 PM4/5/22
to Wojtek Porczyk, dev-secur...@mozilla.org, Ryan Sleevi, Ben Wilson
On 4/5/22 12:09 PM, Wojtek Porczyk wrote:
> All this sound awfully nontransparent. Can I hear more about those "other
> reasons" where auditors were delisted?

The situations that happened this past year (including in 2021) were due
to a business decision being made in one part of a company that ended up
having unexpected consequences in another part of the company.

> Is this related to (I'm just guessing)
> data protection?
no
> conflicting regulatory requirements?
no
> international santions?
no



>
> How can I get a list of audits (and/or auditors) where those written
> permissions have been granted?
>

Mozilla policy does not currently require auditors to be listed on the
ACAB'c or CPA Canada websites.

We are adding this requirement to version 2.8 of Mozilla's root store
policy, and we need to ensure that we do not have to make an exception
to our written policy if another company or organization makes a
non-related decision that has unexpected consequences on CAs and their
auditors.

Thanks,
Kathleen

Kathleen Wilson

unread,
Apr 5, 2022, 5:15:44 PM4/5/22
to dev-secur...@mozilla.org, Ryan Sleevi, dev-secur...@mozilla.org, Ben Wilson, Kathleen Wilson
On Tuesday, April 5, 2022 at 10:57:24 AM UTC-7 Ryan Sleevi wrote:

Maybe, instead of following with the "(unless written permission is granted by Mozilla)" for both, perhaps:

- Mozilla MAY, at its sole discretion, decide to temporarily waive membership or enrollment requirements.


I agree with your idea -- much better that what I had suggested.

Thanks,
Kathleen

 

Wojtek Porczyk

unread,
Apr 5, 2022, 6:06:06 PM4/5/22
to Kathleen Wilson, dev-secur...@mozilla.org, Ryan Sleevi, Ben Wilson
On Tue, Apr 05, 2022 at 09:19:08AM -0700, Kathleen Wilson wrote:
> The problem that we ran into over the past year is that there can be
> business or other reasons that impact when a company like CPA Canada will
> enter into agreements (or end agreements) with other companies. So, while
> our desire is to require auditors to be either members of ACAB'c or listed
> on the CPA Canada website, there may be business reasons not related to
> CAs/PKI for which such relationships cannot be established or continued. We
> also learned over the past year that an auditor can be removed from such
> membership/list after they have already started or even finished the audit
> of the CA for that year, even when that auditor has been on the list for
> several previous years and has not done anything to warrant being removed.
>
> Maybe we can replace the "SHOULD" with "MUST (unless written permission is
> granted by Mozilla)"...
>
> I'm not a fan of that type of wording, but at least it would be stronger
> than the "SHOULD", and would still enable us to handle certain situations
> that we have been running into without having to grant exceptions to
> written policy.
>
> I would also prefer to say "prior written permission", but we ran into
> situations in which the audits and audit statements had already been
> completed before the auditor was removed from the membership/list (to no
> fault of their own).
[snip]

All this sound awfully nontransparent. Can I hear more about those "other
reasons" where auditors were delisted? Is this related to (I'm just guessing)
data protection? conflicting regulatory requirements? international santions?

How can I get a list of audits (and/or auditors) where those written
permissions have been granted?

--
pozdrawiam / best regards
Wojtek Porczyk
signature.asc

Ryan Sleevi

unread,
Apr 5, 2022, 9:21:42 PM4/5/22
to Dimitris Zacharopoulos, dev-secur...@mozilla.org
On Tue, Apr 5, 2022 at 2:57 PM Dimitris Zacharopoulos <ji...@it.auth.gr> wrote:
Hi Moudrick,

CPA Canada is not like a NAB in Europe. NABs supervise CABs which means they assess/examine/review audit work of CABs - I believe - on a yearly basis and decide on the accreditation of the CAB. CABs are "assessed" by NABs similarly as CAs are audited by CABs.

I am not sure CPA Canada works this way. Based on past discussions, please correct me if I'm wrong, my understanding is that WebTrust audit firms have a peer-review process (not sure if it is annual or not) which means that audit firms examine other audit firms' audits.

IMHO CPA Canada is more analogous to ACAB-c than to a NAB but with more "power"/authority over the WebTrust program (closer to the powers of EA https://european-accreditation.org/). ACAB-c has no authority over NABs/CABs or ETSI standards.

The analogy there is not quite correct.

For example, to use North Carolina as an example: North Carolina General Statute Chapter 93 [1] sets forth rules regarding the use of the title "accountant" (and CPA), and established the Board of Certified Public Accountant Examiners as a professional licensing board. The Board is responsible for enforcing the Rules of Professional Conduct, as adopted by the Board. In North Carolina, for example, the board incorporates, by reference [2], the statements on auditing standards developed by AICPA [3], such that it is it is reason for discipline to claim to be a CPA without being licensed as one, or as a CPA, to fail to adhere to the professional standards as established by AICPA. AICPA is itself a Professional Accountancy Organization (PAO) that is a member of IFAC [4], which works to developed harmonized International standards which can correspondingly be incorporated (by reference) into various laws, treaties, etc, e.g. ISAE 3000 [5].

Thus, to demonstrate how poorly the concepts map, CPA Canada here behaves similar to ETSI, with respect to developing audit criteria and standards that satisfy regulatory needs (e.g. incorporation, by reference, of AICPA standards, CPA Canada standards or ISAE 3000 standards). The standards for conducting such audits behave similarly to Implementing Acts with respect to eIDAS, in that they have legal weight and recognition (and penalties for violating). IFAC functions as a somewhat mix between ACAB-c, ETSI, and EA, of which CPA Canada is a member, but that's somewhat orthogonal. Individual jurisdictions (e.g. North Carolina) can establish agencies (e.g. NC Board of CPA Examiners) that can function as NAB-like equivalents, while the WebTrust Task Force and CPA Canada functions akin to a Supervisory Body with respect to the WebTrust scheme, and like the NAB with respect to CABs against that scheme.

Hopefully you can see it's a bit of Apples and Oranges, in that these aren't necessarily equal things, and the concepts don't map cleanly between them.

For example, under the EA model, you have separate entities enforcing compliance with various aspects: the NABs enforce standards on the CABs, but may not supervise scheme-specific activities, which are deferred to a separate entity (the Supervisory Body, e.g. in the case of eIDAS). ETSI may develop standards, both TS and EN, with the latter involving votes according to the ENAP [6], but those don't necessarily have legal weight to them, although they can (e.g. through implementing or delegated acts). CABs may incorporate those ETSI requirements (e.g. to fulfill certain legal objectives), or may develop their own schemes that incorporate parts, or none, of the ETSI work, if fulfilling market goals rather than legal goals. The NAB accreditation of the CAB may only extend relevant to the fulfillment of the legal objective, and not necessarily provide the same degree of oversight or supervision for those market goals, dependent on the scheme. ACAB'c, however, provides a degree of voluntary scheme harmonization and supervision, to address that gap between audits being used for presumptive legal goals versus those satisfying market needs and requirements.

Although the comparison to ACAB-c and CPA Canada (since WebTrust is just a brand of a product by CPA Canada, and the WTTF is partially part of CPA Canada) may be tempting, I hope you can see that there are differences in what they provide and do. The comparison might be more apt if ACAB-c was developing internationally recognized and harmonized standards that were incorporated (e.g. by implementing or delegated acts by the EC), and if there were legally-recognized consequences for failing to uphold the code of conduct of ACAB-c (as there is with respect to CPA Canada's standards), but that's not quite the case.

This ties back with my explanation to Kathleen, in that the lack of a WebTrust license may mean that the entity performing the audit is not recognized as a CPA within the construct of IFAC's standards around audits and attestations, unless independently verified (as in the case of North Carolina, above). There's no single role within the "ETSI" framework to reflect this: it combines portions of the NAB/EA, portions of ISO (e.g. 17065, although ISAE 3000 is _far_ more descriptive and prescriptive), portions of the Supervisory Body for a particular scheme, and portions of ACAB-c for verification of suitability.

Dimitris Zacharopoulos

unread,
Apr 6, 2022, 1:43:34 AM4/6/22
to ry...@sleevi.com, dev-secur...@mozilla.org
Thanks Ryan for the additional context.

I was mainly trying to highlight some known differences between CPA Canada and NABs. In practice, can you please confirm whether CPA Canada performs audit reviews against their licensed practitioners, and if so, how frequently? As I explained, NABs actually review/assess audit cases of CAs performed by their CABs on an annual basis.


Best regards,
Dimitris.

Ryan Sleevi

unread,
Apr 6, 2022, 8:56:27 AM4/6/22
to Dimitris Zacharopoulos, Ryan Sleevi, dev-secur...@mozilla.org
On Wed, Apr 6, 2022 at 1:43 AM Dimitris Zacharopoulos <ji...@it.auth.gr> wrote:
I was mainly trying to highlight some known differences between CPA Canada and NABs. In practice, can you please confirm whether CPA Canada performs audit reviews against their licensed practitioners, and if so, how frequently? As I explained, NABs actually review/assess audit cases of CAs performed by their CABs on an annual basis.

Right, and my explanation was to try to capture that the NAB/CAB review is to a different goal, and purpose, so it's not really a comparison you can make, because they're seeking to achieve different results, following different processes.

In the context of WebTrust licensing, the WebTrust team at CPA Canada would perform a review prior to the issuance of a seal to a licensee. So, in one sense, "every" report is being reviewed. However, you can't directly compare this to what the NAB is doing for the CABs, and using a term like "audit review" may be confusing without understanding the nuanced model of the approaches and differences with WebTrust (/IFAC/CPA Canada/AICPA) and ETSI (/EA/NAB/CAB). The NAB review of the CAB is not necessarily a quality review towards the particular goals of a consumer of a particular audit report (i.e. it's not a quality-control review, like WebTrust), but rather, closer to a professional conduct review (e.g. against ISO CASCO's work). This should be clear in looking at the role the Supervisory Body performs with respect to reviewing the CARs from CABs, and how that's a functionally different step and goal than the NAB assessment.

Moudrick M. Dadashov

unread,
Apr 6, 2022, 8:06:52 PM4/6/22
to ry...@sleevi.com, Dimitris Zacharopoulos, dev-secur...@mozilla.org

Ryan, Dmitris thanks for your comments.

From the harmonisation point of view we need a simplified model that helps us to compare (map) two different conformity assessment bodies: accreditation bodies (NABs) and certification bodies (CABs).

Let’s focus on NABs.

ISO/IEC 17011 sets out the following requirements for accreditation bodies and the process of accreditation:
1) accreditation body requirements;
2) management requirements;
3) human resource requirements;
4) accreditation process requirements;
5) requirements for responsibilities of the accreditation body and the CAB

Looks like some jurisdictions (instead of accreditation) use an alternative system where a single membership organisation (like CPA Canada ?) whose accreditation is based on so called peer assessment. The relevant standard here is ISO/IEC 17040 which contains:
1) structural requirements;
2) human resource requirements;
3) information and documentation;
4) peer assessment process requirements;
5) confidentiality requirements,)
6) complaints handling requirements;
7) guidance on financial aspects, assessment techniques for use by peer assessment teams and information to be included in the peer assessment report.

As both approaches have the same goal and similiar results (accredited CABs), one way of harmonisation could be a minimal set of requirements so that both approaches become acceptable irrelevant to their underlying organisational complexities. Obviously we can do this with active participation of CABs.

Thanks,
M.D.

Sent from my Galaxy
--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Ryan Sleevi

unread,
Apr 7, 2022, 11:01:24 AM4/7/22
to Moudrick M. Dadashov, Dimitris Zacharopoulos, dev-secur...@mozilla.org, ry...@sleevi.com
Hi Moudrick,

I appreciate the comments, but this seems a fairly extreme simplification, and doesn’t seem to be accurate?

I don’t doubt that, with multiple years of effort, deep legal advice, and careful investigation, we might be able to find a way to compare these as apples to apples.

But that seems like a significant investment, and it’s unclear the benefit? Why do we need a simplified model here?

Ben Wilson

unread,
Apr 7, 2022, 11:27:20 AM4/7/22
to Ryan Sleevi, Moudrick M. Dadashov, Dimitris Zacharopoulos, dev-secur...@mozilla.org
FWIW -

Here is Ryan's presentation from the CA/Browser Forum's Shanghai F2F (Oct. 2018) where he compares ETSI and WebTrust - see from Slide 56 onhttps://cabforum.org/wp-content/uploads/CABF45-Sleevi-Whats-Wrong-With-the-Ecosystem.pdf

Also, FWIW, I plan on creating a Mozilla wiki page that outlines these two types of third-party CA reviews, along the lines of slides 56-79 of Ryan's presentation.

Ben

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.

Moudrick Dadashov

unread,
Apr 7, 2022, 12:01:36 PM4/7/22
to Ryan Sleevi, Dimitris Zacharopoulos, dev-secur...@mozilla.org
Thanks Ryan,

actually simplification applies only to the organisational infrastructure and allows us to  concentrate on two major  players: CABs and accreditation  bodies.

Why we need this? If we succeed with minimal requirements for both players, it should help to harmonise the root program requirements.

Although it does require significant efforts, but both the accredition and certification rely on the same semi-formalised "conformity assessmens scheme" concept, which  methodologically is very close to CP/CPS for CAs.

Thanks,
M.D.

Ryan Sleevi

unread,
Apr 7, 2022, 3:22:45 PM4/7/22
to Moudrick Dadashov, Ryan Sleevi, Dimitris Zacharopoulos, dev-secur...@mozilla.org
On Thu, Apr 7, 2022 at 11:59 AM Moudrick Dadashov <m.m.da...@gmail.com> wrote:
Thanks Ryan,

actually simplification applies only to the organisational infrastructure and allows us to  concentrate on two major  players: CABs and accreditation  bodies.

Why we need this? If we succeed with minimal requirements for both players, it should help to harmonise the root program requirements.

Although it does require significant efforts, but both the accredition and certification rely on the same semi-formalised "conformity assessmens scheme" concept, which  methodologically is very close to CP/CPS for CAs.

Moudrick,

Thanks. I think your reply demonstrates the confusion that I was trying to address. It is not fair nor accurate to think about this in terms of Conformity Assessment Bodies and Accreditation Bodies. That is **not** a common element between the two programs, nor is it a common goal.

Indeed, I have tried to communicate in the past as to why attempting to view CAs through the lens of conformity assessment and accreditation is a *technically flawed* approach with respect to security. That's not to say that the very notion of conformity assessment is flawed, but that, in PKI, an accreditation and conformity assessment approach provides different results than those necessary for security.

If conformity assessment was the end-all be all, for example, you would not see requirements for vendor security assessments in work such as GDPR, because the vendor would simply need to be "accredited". That's not the case, however, because situations such as that are context-specific, and require individual approaches to understanding the overall relationship. Conformity assessment can *help*, sure, but it's *not* a replacement.

So no, it's a non-goal to focus on CABs and accreditation bodies, as they are not the "two major players".

Moudrick M. Dadashov

unread,
Apr 7, 2022, 5:03:10 PM4/7/22
to ry...@sleevi.com, Dimitris Zacharopoulos, dev-secur...@mozilla.org
Thanks, Ryan


"Conformity assessment can *help*, sure, but it's *not* a replacement."

Sorry, I don’t know where this replacement comes from...


"So no, it's a non-goal to focus on CABs and accreditation bodies, as they are not the "two major players"."

The reason why these bodies are major players is obvious: accreditation is the only process how CABs become CABs (and maintain their status) and certification is the only process that enable CAs to participate in the Root inclusion program.

Thanks,
M.D.





Sent from my Galaxy


-------- Original message --------
From: Ryan Sleevi <ry...@sleevi.com>
Date: 4/7/22 22:22 (GMT+02:00)
To: Moudrick Dadashov <m.m.da...@gmail.com>
Subject: Re: Policy 2.8: MRSP Issue #219: Require ETSI auditors to be ACAB-c members

Ryan Sleevi

unread,
Apr 7, 2022, 6:07:43 PM4/7/22
to Moudrick M. Dadashov, Ryan Sleevi, Dimitris Zacharopoulos, dev-secur...@mozilla.org
On Thu, Apr 7, 2022 at 5:03 PM Moudrick M. Dadashov <m.m.da...@gmail.com> wrote:
"Conformity assessment can *help*, sure, but it's *not* a replacement."

Sorry, I don’t know where this replacement comes from...

Put differently: It is not correct to view WebTrust as conformity assessment.

Does that make it easier?
 
"So no, it's a non-goal to focus on CABs and accreditation bodies, as they are not the "two major players"."

The reason why these bodies are major players is obvious: accreditation is the only process how CABs become CABs (and maintain their status) and certification is the only process that enable CAs to participate in the Root inclusion program.

Right, I think we're still talking past eachother. Certification is one of two ways in which CAs can participate in the Root Inclusion Program.

Certification / Accreditation, as a model, only apply to the ETSI framework. They do not apply to the WebTrust framework. It is not correct to treat WebTrust as a certification scheme or a conformity assessment scheme.

Does that make it clearer? 

Ben Wilson

unread,
Apr 7, 2022, 6:59:19 PM4/7/22
to dev-secur...@mozilla.org

--
You received this message because you are subscribed to the Google Groups "dev-secur...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-po...@mozilla.org.
Reply all
Reply to author
Forward
0 new messages