SonarQube 6 - Coverage

[Ernesto Ono]

Hi

How do I configure the quality gates in SonarQube V 6.0 ? I am using an example program in C# and TFS 2015 with SonarQube in Build Definitions

The result of SonarQube and my configuration in Quality Gates

Best Regards

Ernesto Ono

[G. Ann Campbell]

Hi Ernesto,

This is very similar in nature to the thread you started yesterday. Just because you didn't get an immediate answer, that didn't mean you weren't going to get one. In general, you should probably wait at least 3 or 4 days, and then you should bump the existing thread, rather than starting a new one. That's because this group isn't just for your immediate gratification but for posterity as well. Also (while I'm griping) your screenshots are huge. I had to expand my browser window to full screen width to be able to see them, which makes me less inclined to fool with the thread. No idea whether I'm the only one on that, but in general, you'll "help us help you" if you focus both text and screenshots on the problem.

Okay, now that I've got that off my chest, I'm prepared to try to be helpful. :-)

There are a couple things going on here. 

First, your Quality Gate:

You've got redundant conditions in the Quality Gate:

• Coverage Over Leak Period is less than 70
• Coverage on New Code is less than 70

We're aware that this is confusing and plan to handle it. In the meantime though, you probably want to remove one of these. I'd vote for "Coverage Over Leak Period" just because "Coverage on New Code" is more explicit.

You've also got a condition on Line Coverage. That's a slightly a different thing, but still overlaps with the first two. Just FYI.

You've also got Uncovered Lines Over Leak Period is less than 5. I think that's probably not what you meant. Instead, you probably meant to alert when Uncovered Lines Over Leak Period is greater than 5. However, "5 lines" seems like a rather unforgiving threshold to me, and one that gets harder to meet the more changes there are in the leak period. That's why we use and recommend percentage metrics for quality gate thresholds. If it were my quality gate, I'd drop this condition.

Now let's talk about your quality gate failures. You keep sending screenshots with arrows pointing at your all-time coverage percentage. But your quality gate doesn't test all-time coverage. It tests (in multiple ways) coverage in the Leak Period. So your arrows should be pointing to the numbers in the yellow blocks on the right of the screen. Moved over, you'll find they're pointing at "-", a.k.a. the 0% that's showing up in the Quality Gate section of the page.

So now you've got numbers that match up from one place on the screen to another. Yay. But why, with no changes, is your quality gate failing? I suspect this has something to do with the fact that this project is brand new in SonarQube. Once you start touching code (and writing tests for it!) this should normalize.

Ann

On Thursday, 13 October 2016 11:54:09 UTC-4, Ernesto Ono wrote:

Hi

How do I configure the quality gates in SonarQube V 6.0 ? I am using an example program in C# and TFS 2015 with SonarQube in Build Definitions

The result of SonarQube and my configuration in Quality Gates

Best Regards

Ernesto Ono

[Ernesto Ono]

Hi Ann

Thank you for your answered

I was testing and I deleted the "Coverage" and "Line Coverage" Metrics, and I changed the "Uncovered Lines" to "is greater than" value Error 0 and then I tested my program, but I see the result quality gate "Passed". I would like to ask if is it correct ?? because it should be failed, I have 2 uncovered lines.

Best regards

Ernesto Ono

[G. Ann Campbell]

Hi Ernesto,

You your Measures screenshot shows me that you have 2 all-time uncovered lines. It does not show me that you have two uncovered lines that were edited in the leak period. From your previous Project Space screenshot, you have 0 uncovered lines in the leak period. Which is why the project passes the quality gate.

Ann

[Ernesto Ono]

Hi Ann

You´re right, I showed 2 times the same screenshot. I was wrong. I wanted to show this:

Sorry

Best regards

Ernesto Ono

[G. Ann Campbell]

Hi Ernesto,

Again, you're showing all-time values. I don't see any numbers here for Leak Period coverage.

Ann

[Ernesto Ono]

Hi Ann

My Over Leak Period in Error value is 0 and "is greater than"

I have 2 Uncovered Lines I think the Quality Gate should be show Failed

[Ernesto Ono]

I was testing again chaging the quality gates. I deleted the Coverage on New Code metric and I added only Coverage metric and the Error value 70. I added the Issue metric and the operator "is greater than" and Error 0.

I see the result. The Issue all is correct because my example program has 2 issue. One of them a variable doesn't using and the other a function does not have lines code, that all correct detecting the SonarQube 2 Issue.

But the Coverage and Uncovered Lines I think are Incorrect result by SonarQube. My metric in Coverage is 70 and the program has 75; it should be Pass but the SonarQube marked Failed.

The metric Uncover Lines is 0 and I have 2 lines uncovered lines that I think it should be Failed and the SonarQube doesn't detect Failed, marked Pass.