Coverity Scan during CI

[Paulo Matos]

Hi,

Travis CI has a coverity scan addon (for static analysis).
I decided to give it a go with Racket using the following configuration:
https://github.com/LinkiTools/racket/blob/wip-qemu-test/.travis.yml

This configuration will run coverity on racket when the branch you
commit to is called coverity_scan. Since doing coverity scanning is
expensive you don't want to do it all the time.

However with Racket's size we can do it twice a day, but we don't need
even that. From their webpage (https://scan.coverity.com/faq#frequency):
"The number of weekly builds per project are as follows:
Up to 14 builds per week, with a maximum of 2 build per day, for
projects with 500K to 1 million lines of code "

Coverity reports racket as having 598,267 loc to analyze.
I can show you the dashboard that I see on coverity although it's not
public for security reasons - members of the project can however, be
invited to see it. Dashboard attached. I cannot see the specifics of the
faults found until the project confirms I am either its owner (I am
not), or part of the dev group (which I am also not), so most likely I
won't be given permission to see the details of the security flaws.

Their explanation is as follows (from the faq linked above):
"Who may be granted access to a Registered Project?

Generally, access to the detailed analysis results for most Registered
Projects is granted only to members of the Registered Project approved
by the Registered Project administrator, to ensure that potential
security defects in the Registered Project may be resolved before the
general public sees them.

Coverity Scan uses the Responsible Disclosure approach. Scan provides
the analysis results to the project developers only, and do not reveal
details to the public until an issue has been resolved. For a thorough
discussion of Responsible Disclosure, you can refer to comments by Bruce
Schneier, or Matt Blaze, or the Wikipedia article on Full Disclosure

Since projects that do not resolve their outstanding defects are leaving
their users exposed to the consequences of those flaws, Synopsys will
work to encourage a project to resolve all of their defects. Synopsys
may set a deadline for the publication of all the analysis results for a
project."

I think it would be interesting to have a regular report of the faults
in the C code of racket. This could be done by having a script merging
on a regular basis (once every 24 or 48 hours) the master branch into
coverity_scan to trigger it. Someone would probably have to look into
the faults reports and open bugs/pull requests if required.

Would this be something that the racket core team would like to see?
If so, I can create a pull request for the travis changes.

Someone would need to register the project in coverity, who would be the
coverity admin. Then that someone can invite members at their discretion
to look into the faults and create bug reports or pull requests if
necessary. I am happy to look at some faults but it should be someone
from the racket team to register this here:
https://scan.coverity.com/projects

On a sidenote, I know Sam has been looking into moving to Azure
Pipelines but this doesn't mean we can keep using travis jobs for the
coverity scan only. One effort won't block the other.

Kind regards,

--
Paulo Matos

[Sam Tobin-Hochstadt]

Hi Paulo,

This sounds like an excellent project. I'm happy to do the necessary
administration on the Racket side if you let me know what to do, but
even better would be if you could do it. Is there a way to arrange for
you to be the owner of the project on Coverity?

Sam

> --
> You received this message because you are subscribed to the Google Groups "Racket Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+...@googlegroups.com.
> To post to this group, send email to racke...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/b91da5d4-bcfe-6027-8950-f1c3b3300295%40linki.tools.
> For more options, visit https://groups.google.com/d/optout.

[Paulo Matos]

Awesome. I can create a PR for this.
I will try to create the project myself and add you as a member of the
coverity project. Hopefully Synopsys will accept it.

I will keep you up-to-date.

On 14/09/2018 15:44, Sam Tobin-Hochstadt wrote:
> Hi Paulo,
>
> This sounds like an excellent project. I'm happy to do the necessary
> administration on the Racket side if you let me know what to do, but
> even better would be if you could do it. Is there a way to arrange for
> you to be the owner of the project on Coverity?
>
> Sam

--
Paulo Matos

[Paulo Matos]

On 14/09/2018 16:47, 'Paulo Matos' via Racket Developers wrote:
> Awesome. I can create a PR for this.
> I will try to create the project myself and add you as a member of the
> coverity project. Hopefully Synopsys will accept it.
>
> I will keep you up-to-date.

PR submitted: #2274
https://github.com/racket/racket/pull/2274

--
Paulo Matos