Jenkins Security Scan now generally available

[Daniel Beck]

Hi everyone,

I've published the previously private[1] Jenkins code scanning rules for CodeQL. These are static analysis rules covering mostly Jenkins-specific issues, like unprotected Stapler web methods and use of APIs that are generally not a good idea in the context of Jenkins plugins.

While this uses the CodeQL CLI and Java language support, the queries are entirely custom, so this is set up so it can run side-by-side with the normal GitHub CodeQL security scanner (or any other such tool), which would identify more generic issues.

You can now enable them for your plugins by setting up a GitHub Workflow. For details about setting this up inside and outside the jenkinsci GitHub org, see the documentation on jenkins.io[2].

The existing mechanisms to run this scan on plugin repos -- signing up through INFRA tickets and labeling repos with jenkins-security-scan-enabled[3] -- will be retired, so I recommend you set this up even if you already get scan results.

Regards
Daniel

1: https://www.jenkins.io/blog/2020/11/04/codeql/
2: https://www.jenkins.io/redirect/jenkins-security-scan
3: https://groups.google.com/g/jenkinsci-dev/c/xpsIgJJy44U/m/w-O0JbpTBgAJ

[Jesse Glick]

Do we generally recommend this for any plugin? If so, it would be great to add this to `archetypes`.

[Daniel Beck]

On Tue, Feb 22, 2022 at 6:59 PM 'Jesse Glick' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

Do we generally recommend this for any plugin? If so, it would be great to add this to `archetypes`.

That's where I think we should end up, but I'd like to get some more scan quality feedback first. The private beta wasn't as successful as I'd hoped in getting that. Perhaps it no longer being an intransparent black box helps with that :)

[Alex]

Huge +1 from me.

It's nice to have the rules publicly available and it overall integrating seamless with GitHub's code scan alerts. Hopefully we can get some more feedback on it, due it now being available to everyone and super simple to enable for plugin devs.

olamy commented on my security scan test PR this morning, that it would be nice to have this as step available in the Jenkinsfile to use on https://ci.jenkins.io/

I'm not sure how feasible that is without defeating the purpose of the GitHub action, though I'm dropping his feedback here nevertheless :P

~Alex

[Herve Le Meur]

I thought about integrating it in ci.jenkins.io shared pipelines, but didn't took the time to discuss it with Daniel yet.

This first GHA step will be nice to round the corners before eventually planning a larger integration I think.

Hervé

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8a6ea7f4-c6d6-4515-a3f3-1c6041a9d256n%40googlegroups.com.

[Gavin Mogan]

> I'm not sure how feasible that is without defeating the purpose of the GitHub action, though I'm dropping his feedback here nevertheless :P

I know one of my goals when i get a minute is to try to update
analytics/warnings-ng to support the github log format -
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-warning-message
- which might make things easier, if not for this specifically, other
github action projects in the future.

On Tue, Feb 22, 2022 at 12:10 PM 'Herve Le Meur' via Jenkins

Developers <jenkin...@googlegroups.com> wrote:
>
> I thought about integrating it in ci.jenkins.io shared pipelines, but didn't took the time to discuss it with Daniel yet.
> This first GHA step will be nice to round the corners before eventually planning a larger integration I think.
>
> Hervé
>
> On Tue, Feb 22, 2022 at 8:41 PM Alex <mc.ca...@gmail.com> wrote:
>>
>> Huge +1 from me.
>>
>> It's nice to have the rules publicly available and it overall integrating seamless with GitHub's code scan alerts. Hopefully we can get some more feedback on it, due it now being available to everyone and super simple to enable for plugin devs.
>>
>> olamy commented on my security scan test PR this morning, that it would be nice to have this as step available in the Jenkinsfile to use on https://ci.jenkins.io/
>> I'm not sure how feasible that is without defeating the purpose of the GitHub action, though I'm dropping his feedback here nevertheless :P
>>
>> ~Alex
>> db...@cloudbees.com schrieb am Dienstag, 22. Februar 2022 um 19:20:12 UTC+1:
>>>
>>>
>>> On Tue, Feb 22, 2022 at 6:59 PM 'Jesse Glick' via Jenkins Developers <jenkin...@googlegroups.com> wrote:
>>>>
>>>> Do we generally recommend this for any plugin? If so, it would be great to add this to `archetypes`.
>>>
>>> That's where I think we should end up, but I'd like to get some more scan quality feedback first. The private beta wasn't as successful as I'd hoped in getting that. Perhaps it no longer being an intransparent black box helps with that :)
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8a6ea7f4-c6d6-4515-a3f3-1c6041a9d256n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAL-LwjzkAdS0T7Ew568x7N05nTZhpNxe%3DkQyF0ZgWT0JfV1ncQ%40mail.gmail.com.

[Jesse Glick]

Tried it out on a plugin I maintain. Seems to work. I suppose any results would appear in `/security/code-scanning` to repo admins only? Will the Checks tab of a PR or trunk commit always be green so long as scanning completed, even if there are violations?

The scan should pass `-ntp` to Maven builds—noisy. It is a bit slower than I expected too, though I suppose it does not matter much (still finishes before the Jenkins build).

Nice work!

[Daniel Beck]

On Tue, Feb 22, 2022 at 10:17 PM 'Jesse Glick' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

I suppose any results would appear in `/security/code-scanning` to repo admins only?

PR-specific results are shown directly and publicly in the PR. GitHub compares results from the PR to results from the target branch to only show differences. (And if you add the workflow file in a PR, that PR will have all findings, because they're all new…)

 

Will the Checks tab of a PR or trunk commit always be green so long as scanning completed, even if there are violations?

At the moment, yes, but it's configurable. I can have a rule fail the check by increasing its level from warning to error, but there's at most one rule I'm confident enough about where that would make sense. Feedback welcome of course.

 

The scan should pass `-ntp` to Maven builds—noisy. It is a bit slower than I expected too, though I suppose it does not matter much (still finishes before the Jenkins build).

I'll look into what I can figure out. Unfortunately the Maven invocation is part of the GH proprietary bits. For now I'm wrapping the log spam in a collapsible block in the GH Action output.

Regarding duration, note that you can set up a Maven cache if you haven't. That might speed things up some.

[Jesse Glick]

On Tue, Feb 22, 2022 at 4:41 PM 'Daniel Beck' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

PR-specific results are shown directly and publicly in the PR. GitHub compares results from the PR to results from the target branch to only show differences.

Oh nice.

 

And if you add the workflow file in a PR, that PR will have all findings, because they're all new…

Maybe something to warn maintainers about it: contrary to usual recommendations, it may be best to create this workflow directly in trunk, lest you reveal a bunch of previously undiscovered mistakes. Of course anyone could have run the same scan offline, but this seems to publicize problems more than seems comfortable.

note that you can set up a Maven cache if you haven't.

Yes, I tried with the Maven cache enabled, so perhaps the PR runs would be significantly faster than the initial trunk run.

[Basil Crow]

I added this to all the plugins I maintain. Very nice!

After adding this, new unrelated pull requests (see, for example,
jenkinsci/email-ext-plugin#346, which just adds a line to the readme)
all seem to fail in the "Run Scan" step with:

Interpreting results.
[…]
curl: (22) The requested URL returned error: 403
Failed to upload results
Error: Process completed with exit code 1.

Does https://github.com/jenkinsci/email-ext-plugin/blob/master/.github/workflows/jenkins-security-scan.yml
look correct? The proximate source of the failure appears to be
https://github.com/jenkins-infra/jenkins-security-scan/blob/75608943dcc0a4c27e7fa30e27279748cc351dcf/.github/actions/jenkins-codeql/scan.sh#L71-L73
but I have not yet investigated more closely.

[Jesse Glick]

On Wed, Feb 23, 2022 at 3:29 PM Basil Crow <m...@basilcrow.com> wrote:

curl: (22) The requested URL returned error: 403
Failed to upload results

I get the same on a trunk commit. 

[Daniel Beck]

On Wed, Feb 23, 2022 at 9:29 PM Basil Crow <m...@basilcrow.com> wrote:

curl: (22) The requested URL returned error: 403
Failed to upload results
Error: Process completed with exit code 1.

Interesting. It probably happens because it's a PR from a fork and the GITHUB_TOKEN used only has read permission for SecurityEvents. I'll look into solutions tomorrow.

[Olivier Lamy]

Thanks a lot Herve!

It would be awesome to build/improve Jenkins (and all the family) with Jenkins tools provided by the Jenkins community running on platforms provided and maintained by the Jenkins community.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAL-LwjzkAdS0T7Ew568x7N05nTZhpNxe%3DkQyF0ZgWT0JfV1ncQ%40mail.gmail.com.

--

Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy

[Daniel Beck]

On Wed, Feb 23, 2022 at 10:03 PM Daniel Beck <db...@cloudbees.com> wrote:

Interesting. It probably happens because it's a PR from a fork and the GITHUB_TOKEN used only has read permission for SecurityEvents. I'll look into solutions tomorrow.

It looks like GitHub's action can do what I cannot because it uses an undocumented API.

I'll update this thread once it works, meanwhile you can watch https://github.com/jenkins-infra/jenkins-security-scan/issues/3 

[Jean-Marc Meessen]

Thanks for looking into this, Daniel, and let us know about the status. Looks trickier than expected. 

In the meantime, I will (try) to turn code scanning off so that I can get the PRs through CI.

/- Jmm

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAMo7PtKKZVTPDQG%3D_zYbf3NHgasJm1weaPBs5TTdTtsreuaHXw%40mail.gmail.com.

[Daniel Beck]

On Fri, Feb 25, 2022 at 2:43 PM Jean-Marc Meessen <jean...@meessen-web.org> wrote:

In the meantime, I will (try) to turn code scanning off so that I can get the PRs through CI.

Removing https://github.com/jenkinsci/.github/blob/011201ac97f9e2757cca0415590952eaee704e5b/workflow-templates/jenkins-security-scan.yaml#L6-L7 should do it. The rest still works.

Sorry for the inconvenience. The existence of a single documented upload API and an action to do the upload tricked me into thinking this just works :(

[Daniel Beck]

On Fri, Feb 25, 2022 at 11:49 AM Daniel Beck <db...@cloudbees.com> wrote:

It looks like GitHub's action can do what I cannot because it uses an undocumented API.

I'll update this thread once it works, meanwhile you can watch https://github.com/jenkins-infra/jenkins-security-scan/issues/3 

I've updated the workflow to properly work with pull requests from forks.

The result with the now used GH action to upload the scan result differs from the scan result upload API's, so I've decided to increase the version of the workflow to v2. v1 still works as before, but you need v2 for full PR support.

https://github.com/jenkins-infra/jenkins-security-scan/releases/tag/v2

[Basil Crow]

After upgrading a dozen or so plugins to Security Scan v2, the Jenkins
Security Scan workflow on the main branch failed with:

Called workflows cannot be queued onto self-hosted runners across
organisations/enterprises. Failed to queue this job. Labels:
'ubuntu-latest'.

List of example plugins below:

https://github.com/jenkinsci/database-sqlite-plugin
https://github.com/jenkinsci/email-ext-plugin
https://github.com/jenkinsci/emailext-template-plugin
https://github.com/jenkinsci/exclusion-plugin
https://github.com/jenkinsci/git-bisect-plugin
https://github.com/jenkinsci/gitlab-oauth-plugin
https://github.com/jenkinsci/http-request-plugin
https://github.com/jenkinsci/jakarta-activation-api-plugin
https://github.com/jenkinsci/jakarta-mail-api-plugin
https://github.com/jenkinsci/javax-activation-api-plugin

[Daniel Beck]

On Mon, Feb 28, 2022 at 8:00 PM Basil Crow <m...@basilcrow.com> wrote:

After upgrading a dozen or so plugins to Security Scan v2, the Jenkins
Security Scan workflow on the main branch failed with:

Called workflows cannot be queued onto self-hosted runners across
organisations/enterprises. Failed to queue this job. Labels:
'ubuntu-latest'.

Interesting, I haven't seen this during development and that includes repos in jenkinsci.

Some searching indicates you're being rate-limited: https://github.community/t/called-workflows-cannot-be-queued-onto-self-hosted-runners-across-organisations-enterprises-failed-to-queue-this-job-labels-ubuntu-latest/229355/10 (which got a GH team response, they seem to be looking into this).

If you pushed out the changes to the YAML files in quick succession, that might explain it? It looks like you were particularly active around 18:10.

Looks like I'll need to look into adding this to the pipeline library sooner rather than later :-)

[Daniel Beck]

Hi everyone,

Some feedback the Jenkins security team received for code scanning was that it is inconvenient to mark findings as false positives through the GitHub UI.

Thanks to work by https://github.com/yaroslavafenkin the Jenkins Security Scan now supports two different ways to suppress findings in code: using comments or using a @SuppressWarnings annotation.

The detailed finding descriptions on the GitHub UI explain how to use these to suppress specific findings (re-run the scan if needed to get an updated description).

Regards

Daniel

[Basil Crow]

Does the Jenkins Security Scan need to be adapted to use the artifact
caching proxy?

[Basil Crow]

On Sun, Mar 19, 2023 at 7:13 PM Basil Crow <m...@basilcrow.com> wrote:
> Does the Jenkins Security Scan need to be adapted to use the artifact
> caching proxy?

Any answer to this question?

[Daniel Beck]

On Mon, Mar 20, 2023 at 3:13 AM Basil Crow <m...@basilcrow.com> wrote:

Does the Jenkins Security Scan need to be adapted to use the artifact
caching proxy?

The workflow would need to be adapted to use the caching proxy, but given uncertainty around unauthenticated future use of Artifactory I am hesitant to change something now (potentially requiring callers to adapt) only to change it again fairly soon. You also mentioned to me a few weeks ago that this is probably no big deal given GH's caching of Maven dependencies, so I didn't look into this further.