What is the model for 'breaking a build' without allowing people to game the system by waiting for bad build information to be cycled in?

[julian...@gmail.com]

I'm currently on SonarCube 5.5 - and previously we were on Sonar 5.0 - breaking our build with the maven sonar plugin:

<plugin>
    <groupId>org.sonarsource.scanner.maven</groupId>
    <artifactId>sonar-maven-plugin</artifactId>
    <version>3.0.1</version>
</plugin>

Now we had a cycle where we would break on builds that didn't meet the standard, and only upload results for results that 'passed the build breaker'. 

Before we were using 'preview' to check and break the build for builds that didn't meet the standard. 

Obviously with the 5.5 upgrade this all broke. 

Now it looks like 'preview' has been renamed, and the way that you get a broken build is changed. 

What I'm trying to wrap my head around is - under the new system - it looks like you have to upload results to get a taskid to get a broken build. 

To me looks like you have to upload results to get a taskid - which means you corrupt the database with the 'lesser standard'. (Perhaps I'm misunderstanding this). This allows people to 'game the system' - breaking the build the first time, making no change, and then having their change slide through on the second analysis. (As opposed to only uploading data when it passes - as part of a second pipeline step). 

My question is: What is the model for 'breaking a build' without allowing people to game the system by waiting for bad build information to be cycled in?

[G. Ann Campbell]

Hi,

FYI, the standard courtesies (Hi, Thanks, ...) are appreciated in this group.

Now to your questions. There are a couple different things going on here. 

Let's start with the problem of defects "ageing out". I think this is really an issue of mindset, and perhaps of "leak period."

You don't actually give a lot of specifics about your case, so I'm going to speculate:

* You've set your leak period to previous_analysis

* You've set your quality gate to something like: no new Bugs in the leak period

Under those two conditions, the scenario you describe is both plausible and likely. So let's change the conditions. If you set the leak period to previous_version then there's no way for defects to "age out" of the consideration period. Either they're fixed, or they're consciously allowed to go into production. 

Regarding "corrupting" the SonarQube project with the "lesser standard", this is exactly what you should do. Presumably, we're not talking about branch analysis, but about your head/master branch. In that case, you should be analyzing and publishing for everyone to see the current state of your source code. If you only analyze the "good" checkins, and suppress information about the bad ones, then you deliberately promote a false sense of security about the state of your code.

Instead, you should be publishing the current state of your code, "warts and all".

Ann

[julian...@gmail.com]

Hi Ann, 

Thanks for taking the time to get back to me. Any further information on local courtesies would be much appreciated. 

Thanks for directing my attention to the leak period. We had it set to the default (no entry) which is 'previous version'.  (But you're correct that my invalid assumption was that it was set to something with the meaning of previous analysis.) 

I've been reading about the distinction between 'previous analysis' and 'previous version':

http://sonarqube-archive.15.x6.nabble.com/Quality-profile-that-compares-against-last-good-analysis-td5022849.html

Could you clarify that when the leak period is set to 'previous version' that the analysis will only compare to older 'good' analyses and failed ones will be ignored? (I'm trying to guard against the risk of people gaming the system by letting defects age out). I'm also trying to make sure the quality only goes up. 

Cheers

Julian

[G. Ann Campbell]

Hi Julian,

On Sun, Nov 27, 2016 at 12:37 AM, <julian...@gmail.com> wrote:

Could you clarify that when the leak period is set to 'previous version' that the analysis will only compare to older 'good' analyses and failed ones will be ignored? (I'm trying to guard against the risk of people gaming the system by letting defects age out). I'm also trying to make sure the quality only goes up. 

 

I don't understand your distinction between 'good' analyses and failed ones.

If the analysis fails, then the data doesn't reach SonarQube, so there's no way it can be included in your history.

To make sure defects don't age out, set your leak period to a fixed point - previous_version, or a specific date. Then you can make sure that quality only goes up by keeping an eye on changes in the leak period.

Ann

---

G. Ann CAMPBELL | SonarSource

Product Owner

http://sonarsource.com

[roshan shariff]

Hi,

can you please help me how to configure build breaker plugin for sonarqube with maven?

thanks,

[G. Ann Campbell]

Hi,

Please don't excavate old threads. You should start a new one instead.

Ann

---

G. Ann Campbell | SonarSource

Product Manager

@GAnnCampbell

http://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/mZT6eF2njfU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/c18865e5-3817-4bb3-8171-ba4f26dd1ff7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.