Re: Question about security rules with SonarQube

[G. Ann Campbell]

Hi Julie,

You emailed me privately, but I'm including the SonarQube Google Group in my response because we generally believe it's best to have these conversations openly so others can benefit too.

My answers are inline below.

:-)

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

On Fri, Aug 18, 2017 at 11:24 AM, Julie Denning <julie....@razrhq.com> wrote:

Ann,

We are a small marketing company that has been using Veracode for our SAST tool, but are planning to switch and would like to switch to SonarQube.

Welcome to the fold. :-D

 

What we've found is that

• There are some CWE checks that both Veracode and SonarQube perform
  
• SonarQube has additional CWE checks, mostly code quality, that Veracode does not have 

In fact, code quality / maintainability is where we started so it's probably not surprising that we have more rules in this area than others.

 

• Veracode has a large number of CWE checks that SonarQube doesn’t have, including cryptographic issues, code injection, various C/C++ issues, backdoor checks, information leaks, cross-site scripting, and others
  

We've been working hard in the last couple of years to improve our technology to be able to reliably cover more Security-related issues. Doing it properly requires being able to follow a value from method call to method call to method call, retaining retaining all along the context that it was submitted by the user. We're not there yet, but we hope to be soon. 

 

• The nature of SonarQube’s fast light-weight scans leads to a large number of FPs and a low number of true positives generated. For example: SonarQube’s SQL Injection rule doesn’t check to see if an attacker can pass a string to a SQL command, it just checks to see if the string being passed is non-constant. That is very FP prone.
  

I don't think that's an entirely fair statement, unless the context is pre-narrowed in your mind to Vulnerabilities. For Bugs and Code Smells we adhere rigorously to a low-FP standard.

However, we do accept a higher false-positive ratio for Vulnerabilities, with the assumption that in most cases it will take a security auditor to sort it out anyway. That said, we are actively working to improve our underlying technology and expect to be able to reduce this FP ratio "soon".

• SonarQube is capable of finding only a few of the security flaws that Veracode can report on.
  

We readily admit that we're not there yet and do advise that for now SonarQube should be used alongside analyzers that specialize in security.

 

I'm "cold emailing" you because in my digging around the SonarSource site, I saw that you have published a lot of info/blogs/etc. related to the security rules.

Can you help me understand what from the OWASP Top 10 ​is not being tested?

Wow. That's a big question.

Each category in the OWASP Top 10 is incredibly broad, and not all of them are verifiable with static code analysis. Let's take, for instance, "A2-Broken Authentication and Session Management". Fully enforcing this requires an understanding of what functions in an application require authentication. 

It would be easy to say, for instance, that any database writes require authentication. But what if you've put a simple poll on your homepage? Voting would presumably require a database update, but requiring authentication here would mean that anonymous users couldn't participate. Maybe that's what you want, maybe it's not, but there's no way to know from a static analysis perspective. (This, BTW, is an example of why we tolerate a higher ratio of FPs for Security-related rules.) Raising issues on this also requires an understanding of what "broken" means in the context of your application, which is again difficult from a static analysis perspective and why we assume (hope!) a security auditor is involved.

But I don't mean to bust your chops. The easiest way to give you the kind of answer I think you're looking for is to point you at the rules lists on sonarsource.com. Here's the list of OWASP-related rules for Java. You can click any rule title to expand its description, and at the bottom of the description, you'll see a reference to the category to which we believe it's related. To see the lists for other languages, choose a language from the list, click on "Rules", then filter by "OWASP". If you'd rather see rules by OWASP category than language, then you can use the rule tag search on SonarCloud.io. Here's a search for A1.

That said, because the OWASP categories are so incredibly broad we could have 15 rules related to a given category and still not fully cover it because this is one of those things that's like "how big is the sky?". Just when you think you've got it all nailed down, you find out there's a little more. I raise the point because I would personally be suspicious of any vendor that said they "fully covered" any of the OWASP categories.

This is important to us because we need to be PCI compliant and the auditors validate that we are testing our apps against each of the OWASP Top 10.

I'd appreciate any help.

Thank you,

--

Julie Denning | Director, Information Security

[shekard...@gmail.com]

Hello Ann,

Does licensed SonarQube (SonarSource) has any additions to the Security Vulnerability scanning? also Dynamic/Interactive security testing (SAST/DAST) can be achieved?

Also FindSecBugs plugin is for Java based projects, do we have any plugins for C# to add more rules under vulnerabilities?

Thanks,

Shekar

[G. Ann Campbell]

Hi Shekar,

Individual analyzers are licensed, not rule sets. If you're able to use a given analyzer, then you automatically have access to all its rules.

For SAST/DAST, as I told Julie, you should look to other vendors.

And finally, I'm not aware of addition security rules for C#.

Ann