Further Improving the CCADB Incident Reporting Guidelines (FEEDBACK REQUESTED)

[Ryan Dickson]

Hi everyone, 

In October 2023, the CCADB Steering Committee, with valuable feedback from this community, updated the CCADB Incident Reporting Guidelines (IRGs). While the resulting updates have led to some reports becoming more useful and effective, Root Store Operators have continued to stress the importance of high-quality incident reports during CA/Browser Forum Face-to-Face updates and elsewhere. 

In the spirit of continuous improvement, the CCADB Steering Committee has worked over the past few months to further enhance the effectiveness of the IRGs.

Objectives for this update to the IRGs include:

• Clarifying Root Store Operator expectations
• Aligning report format and content with those expectations
• Improving clarity regarding the difference between “preliminary" and “full" reports, and the timelines for disclosing these reports
• Improving Root Cause Analysis
• Tracking commitments made by CA Owners in response to incidents
• Increasing accountability and generating more actionable insights
• Improving consistency in report quality
• Emphasizing continuous improvement and lessons learned
• Encouraging familiarity with historical incident reports
• Defining a standard process for closing reports
• Aligning the incident reporting format with Steering Committee objectives planned for 2025+

The set of proposed updates are available here. 

Beyond the above changes, we are considering making the following recommendations:

• To better encourage blamelessness, when posting incident reports or responding to comments on incident reports for which they are affiliated, participants are encouraged to respond from a Bugzilla account associated with one of the CA e-mail aliases disclosed to the CCADB, rather than an individual contributor’s account. Some CAs already do this, and we’d like this to become a standard practice.
• To better respect a desire for individual privacy and potential risk of retaliation, individuals participating in the incident reporting process should feel welcome to participate responsibly from an account that does not identify the individual posting or their organizational affiliation.

These proposals should not be considered “final”, but instead a “work in-progress” that we hope to enhance through community contributions. We welcome your feedback on these proposed updates and recommendations by January 15, 2025. Please share your thoughts by replying to this email or, preferably, by suggesting edits directly on GitHub.

Thanks,

Ryan (on behalf of the CCADB Steering Committee)

[Ryan Dickson]

Hi everyone,

Thanks to some early feedback from members of the community, we’ve made a few updates to the proposal made in the original Pull Request.

The updated proposal is available here. We’ve closed the original Pull Request, but will allow it to persist to help describe changes between versions and retain community feedback. 

Again, these changes should not be considered “final”, but instead a “work in-progress” that we hope to enhance through continued community contributions. We welcome your feedback on these proposed updates and recommendations by January 15, 2025. Please share your thoughts by replying to this email or, preferably, by suggesting edits directly on GitHub.

Thanks,

Ryan (on behalf of the CCADB Steering Committee)

[Martijn Katerbarg]

Ryan, all,

 

We’ve added feedback to the GitHub Pull Request for anything addressing the proposed language.

 

Besides that, we wanted to provide feedback to the recommendations the CCADB Steering Committee is considering.

 

>To better encourage blamelessness, when posting incident reports or responding to comments on incident reports for which they are affiliated, participants are encouraged to respond from a Bugzilla account associated with one of the CA e-mail aliases disclosed to the CCADB, rather than an individual contributor’s account. Some CAs already do this, and we’d like this to become a standard practice.

>To better respect a desire for individual privacy and potential risk of retaliation, individuals participating in the incident reporting process should feel welcome to participate responsibly from an account that does not identify the individual posting or their organizational affiliation.

 

We certainly see and agree that both the items above are practices that should be allowed, for a multitude of reasons. However, we would also like to raise that there are members and participants who prefer using their direct names and accounts. In some cases we believe seeing who posts can make a difference in context and on how a comment can be interpreted.

 

With that in mind, we would like to see the quoted to-be-considered recommendations moved to a “clear allowance” state. If the CCADB Steering Committee feels strongly about making this a recommendation, we would request adding (and keeping) an allowance for deviating from such behavior as well.

 

Regards,

Martijn Katerbarg
Sectigo

 

--
You received this message because you are subscribed to the Google Groups "CCADB Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public+un...@ccadb.org.
To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/CADEW5O8hJvwpZZkCJweoFfDqy%2B0k50-iV76D3qXnWFJv0PWi_w%40mail.gmail.com.

[Mike Shaver]

I agree. I think that it's fine for representatives to have their own voice in the discussions, as long as it's clear when they are speaking officially for their employer and when they are providing their personal context on an issue. I think many in the community (myself included) prefer to converse with other people rather than abstract corporate entities, when possible.

That said, it's important that critique of a company/organization's practices not be confused with criticism of the individual. (Such individual criticism might be appropriate, if it's about that individual's behaviour or personal statements, but it should be considered distinct from criticism of their employer's practices or undertakings, IMO.)

Mike

To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/SA1PR17MB6503876E59709A10E5519618E3022%40SA1PR17MB6503.namprd17.prod.outlook.com.

[Sandy Balzer]

SwissSign believes that the current format addresses most key areas and provides all necessary information. While we acknowledge that there is always room for improvement and more detailed reporting, we would like to stress the importance of minimizing the complexity of initial reporting.

Industries with high learning needs from unplanned events and bugs—such as aviation, healthcare, and software development—have shown that reducing the complexity of opening tickets encourages quicker reporting. This approach allows for capturing a broader range of incidents and fosters inclusive discussions by enabling contributors across all skill levels to participate effectively. Simplifying the reporting process also aligns with best practices observed in high-reliability organizations, which prioritize quick and effective responses to emerging issues.

Additionally, we fear that the proposed "official commitments" may lead to incident reports being written by legal departments instead of the technical/compliance departments.

In conclusion, we recommend keeping the complexity of initial reporting low and standardized, as it is in the current format. This approach supports faster reporting, broader participation, and a more inclusive and effective learning process.

Kind regards

Sandy Balzer

[Mike Shaver]

Hi Sandy,

I agree very much that any process added should "pay for itself" in terms of the value to the ecosystem that comes from additional effort on the part of reporters. I do feel, though, that the quality of reports and especially of Action Item selection and follow-up, have been to the detriment of the goals of the reporting process.

Could you elaborate on what aspects of the proposal you feel add significant complexity to initial reporting, and perhaps help us better understand how they could impact SwissSign's ability to make prompt and fulsome reports? The aviation example is one where I feel that the emphasis on follow-up investigation, auditable transparency, and evaluation of remediations points towards more rigour in the CCADB reporting process, I will say.

Thanks,

Mike

To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/add70d66-9b39-405e-ab6a-bc5b59b352e6n%40ccadb.org.

[Sandy Balzer]

Dear Mike,

For us the current format and depth of the incident reports are sufficient to determine follow-up checks / reviews on our systems to ensure that we don't have the same issues. Of course we don't speak for other CAs, Root Store programs or the wider community. If there are cases where incident reports are not clear enough / don't go into sufficient detail, we prefer to request further information as a follow up to the initial incident report. We feel that this would be more efficient than to require more details in the initial report in all cases.

SwissSign has and will of course be able to produce prompt and fulsome reports, also with the suggested changes to the incident reporting process. It is our experience that more detailed reports carry a higher risk of making statements that could be misinterpreted, wrongly generalized or taken ouf of context. This may lead to situations where non-technical departments (e.g. legal, corporate comms...) have to review technical incident reports to minimize such risks, adding cost and time to the process without adding value to the ecosystem.

Looking at the example of incident reporting in aviation shows a (maybe?) significant difference: Incidents are reviewed by experts (e.g. NTSB) and the process is only partially transparent. Of course, the Web PKI is in no way comparable to aviation where lives are at stake, but maybe we could consider a similar setup where incidents are reviewed by experts and reports being made public only after initial assessment / review has been completed?

Kind regards
Sandy

[Ryan Dickson]

Hi everyone,

Thanks again for your continued collaboration and feedback on the set of proposed updates to the CCADB Incident Reporting Guidelines.

Short of considering final comments from the community, the CCADB Steering Committee intends to finalize this draft (Pull Request) with a planned effective date of March 1, 2025.

Please let us know if there are any questions.

Thanks, 

Ryan (on behalf of the CCADB Steering Committee)

To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/69da00d5-835d-4a9d-bf50-da67735c013an%40ccadb.org.

[Ryan Dickson]

Hi everyone,

The updated Incident Reporting Guidelines have been published on CCADB.org. As previously announced, these updates will take effect on March 1, 2025.

Thank you to all those who collaborated and offered review and feedback on the set of proposed updates.

- Ryan (on behalf of the CCADB Steering Committee)

[Ryan Hurst]

For what it’s worth, I recently wrote a blog post about incident response best practices, inspired in part by the updated reporting guidelines. It examines how we can improve the industry's approach to incident response by embracing transparency and a learning mindset and it offers some practical steps for CAs to handle incidents constructively. It’s posted here: [https://unmitigatedrisk.com/?p=982]

To view this discussion visit https://groups.google.com/a/ccadb.org/d/msgid/public/CADEW5O9HGquF_bgJtbxespJ4iiT9xHxrATe7LmmgaON3qzZQdw%40mail.gmail.com.