Core Infrastructure Initiative (CII) compliance for the Jenkins core - status thread

[Oleg Nenashev]

Hi all,

This is a follow-up to the Community Bridge funding thread and to contributor summit discussions about CII. As discussed there, Linux Foundation expects all projects on Community Bridge to be also a part of the Core Infrastructure Initiative which is their program for strengthening security in open-source projects. In particular, there is a badge program here. All Community Bridge projects are expected to eventually pass certification there.

I believe that being compliant with CII is a net positive thing for us, because it can help to promote the project and to address some quality-related and certification queries from current and potential Jenkins users (e.g. see this recent thread). It also unlocks access to  targeted security project funding / engineering time donations by CII corporate members (Assistance program) and to tooling like Snyk.

I started working on a CII checklist for the Jenkins core, plugins are out of the scope for me at the moment. You can find the current status on this page. We are currently at the 80% completion state, and there are some open topics which need to be clarified. I have summarized the topics below after the email, and I will start follow-up threads for them so that they can be discussed separately.

CII is definitely a case when the remaining 20% for the work require 80% of effort, but I hope to gradually get to the full certification checklist for the Jenkins core. Even if we do not pass the certification criteria there, it is nice to have a documented status for quality/security expectations. I will appreciate any feedback about the CII compliance in general and about the self-certification page. Unfortunately documentation-as-code is not supported there, but I am happy to incorporate any suggested changes.

Best regards,

Oleg

#### Open topics: 

Problem 1. Incoming issues triage (section status). We do not longer have an active triage team which would be regularly reviewing incoming issues in Jira. Alex Earl made a proposal to have an official triage team in 2017 (dev list thread), but it was not implemented at the moment. I was doing regular issue triage until Dec 2018 before I stepped down (see the same thread). Right now we regularly look at the Jenkins release community ratings and reported regressions, but I would not say we have a real triage process, especially for RFEs and bugs reported to non-core components 

• CII Criteria:
  
• " The project MUST acknowledge a majority of bug reports submitted in the last 2-12 months (inclusive); the response need not include a fix."
  
• " The project SHOULD respond to a majority (>50%) of enhancement requests in the last 2-12 months (inclusive).  "
  
• My assumption is that we are below these criteria
  
• Potential solution: Maybe we should revise this topic. Since we have more active core maintainers now, maybe we could have a rotation for the incoming issues in Jenkins Jira. To be discussed in a separate thread
  

Problem 2. Quality and Code analysis warnings (section status). The project MUST enable one or more compiler warning flags, a "safe" language mode, or use a separate "linter" tool to look for code quality errors or common simple mistakes, if there is at least one FLOSS tool that can implement this criterion in the selected language. Jenkins core addresses it, because we have a bunch of tools enabled like Spotbugs, Animal Sniffer or Maven Enforcer. But there are some downstream criteria

• Problematic CII criteria:
  
• The project should fix warnings or mark them in the source code as false positives. Ideally there would be no warnings, but a project MAY accept some warnings (typically less than 1 warning per 100 lines or less than 10 warnings).  
• It is SUGGESTED that projects be maximally strict with warnings in the software produced by the project, where practical.
• Problem: We ignore some warnings without explicitly supressing them (Javadoc and other minor things). And we definitely do not set maximally strict requirements, our SpotBugs runs on the High threshold by default. Stefan Spieker is doing a great job with the issues cleanup, for "Medium", but there are still a lot of issues left
  
• Potential solution: Fail the Suggested criteria for now, review the warnings we get from tools and address quick-wins. Suppress the rest?

Problem 3. Security requirements (status). There is a bunch of certification criteria there which requires a careful review and response (usage of encryption, delivery process, etc.). My understanding is that we are not fully compliant with the certification rules there, and that making Jenkins core fully compliant would be a stretch goal. It does not mean we have security issues, but the formal criteria there set a high bar and opinionated requirements about how security issues should be handled.

• Plan: I will be following up with the Security team on this certification section.

[Tracy Miranda]

Hi Oleg,

Thanks for putting this together and establishing that baseline score!

IMHO it is a great exercise to run through as proven by the issues you raised in the email. (Also nice to see the badge linked when I click on Jenkins on the CDF and CNCF landscapes). 

I look forward to the follow on threads, plus also plan to take a more detailed look at the report. 

Thanks,

Tracy

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLC9Zo38XW8qpKE7vzRfS-EDR_016WViObFdU37a-No-ow%40mail.gmail.com.

[Oleg Nenashev]

Updates here:

• Right now we are at the 80% mark w.r.t the compliance: https://bestpractices.coreinfrastructure.org/en/projects/3538
• We would be interested to pass Core Infrastructure Initiative certification as a part of the CDF graduation process (see this thread).
• I started working on addressing the current issues in the certification:
• Issue Triage: We need a formal process w.r.t providing initial feedback to bug reports and feature requests. I restarted a thread about the Bug Triage team for the Jenkins core. See https://groups.google.com/d/msg/jenkinsci-dev/XToix3QpL_k/u6-7awD4AwAJ and further comments
• Security checklist: I started a Google Doc for the Security checklist. It should help us to perform a joint review of the requirements and to prepare a response.

Any feedback about the wording and the security checklist would be appreciated.

Best regards,

Oleg

On Tuesday, February 18, 2020 at 9:00:44 PM UTC+1, Tracy Miranda wrote:

Hi Oleg,

Thanks for putting this together and establishing that baseline score!

IMHO it is a great exercise to run through as proven by the issues you raised in the email. (Also nice to see the badge linked when I click on Jenkins on the CDF and CNCF landscapes). 

I look forward to the follow on threads, plus also plan to take a more detailed look at the report. 

Thanks,

Tracy

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

[Oleg Nenashev]

Hi all,

Just a quick update, after submitting the security checklist and our current Jira metrics to https://bestpractices.coreinfrastructure.org/en/projects/3538, I am happy to announce that we have reached the 133% mark and hence the Jenkins project is now officially passing the Core Infrastructure Initiative certification. Thanks a lot to all contributors, and special thanks to the Jenkins Security team (esp. Daniel and Wadeck) for multiple cycles of reviews in the checklist!

Next steps would be to keep working on the CII certification towards silver (200%) and gold (300%) grades. There are much more strict requirements on these levels (e.g. strict license file requirements, infra authorization guidelines, etc. etc.). There will be a lot of work to get there, but I think we can keep working on requirements which we consider beneficial to the Jenkins project and the community

Best regards,

Oleg