Breaking the build in SonarQube 5.2

[Matthew Harrison]

From reading:

https://groups.google.com/forum/#!topic/sonarqube/lupyII6ATvo

and

https://groups.google.com/forum/#!topic/sonarqube/r3q50y-Jqus

It seems like the build breaker plugin is not going to continue for 5.2.  So I'm just wondering if there will be a way of failing the build on our build server, should the analysed code not pass the quality gate for a project? 

Actually reading that second post it does seem there is a plan for this, maybe this will be covered off in the documentation?

Thanks,

Matt

[Fabrice Bellingard]

Hi Matthew,

indeed, the build breaker plugin won't be available for SQ 5.2+. The idea is to develop a core feature to answer the use cases previously covered by this plugin. This is what we call the "what if" feature => https://jira.sonarsource.com/browse/SONAR-6763

Best regards,

Fabrice BELLINGARD | SonarSource SonarQube Platform Product Manager http://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/67d9d0bb-2437-4fb5-9f74-ab4da9c576f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Matthew Harrison]

Hi, so just to clarify my understanding of what will change.  Apologies if its something that's been spelled out before.  With 5.2 (lets say we're working with Jenkins) the SonarQube plugin will get all the information for the analysis, and pass that over to the server to run the analysis, once the data has been sent then the plugin will regard its work as done, and the build can continue, regardless of what happens during the analysis on the server.  Is my understanding correct?

The JIRA Ticket is related to that, but in our case we're using SVN (probably not important) and a commit to trunk model.  So for us, the code will have been committed, and we're ok with the analysis being done and shown in SQ.  But it does mean that in order to ensure we keep improving the code quality we would like the build to break, so that we quickly see it, and can address the issue.

Hopefully that gives an idea of where we're coming from - maybe a slightly different use-case from the ones outlined in the ticket?

Thanks,

Matt

[Fabrice Bellingard]

On Wed, Oct 21, 2015 at 11:42 PM, Matthew Harrison <matthew....@equinox.co.nz> wrote:

Hi, so just to clarify my understanding of what will change.  Apologies if its something that's been spelled out before.  With 5.2 (lets say we're working with Jenkins) the SonarQube plugin will get all the information for the analysis, and pass that over to the server to run the analysis, once the data has been sent then the plugin will regard its work as done, and the build can continue, regardless of what happens during the analysis on the server.  Is my understanding correct?

Yes, you got it: in 5.2, this is exactly what will be happening.

 

The JIRA Ticket is related to that, but in our case we're using SVN (probably not important) and a commit to trunk model.  So for us, the code will have been committed, and we're ok with the analysis being done and shown in SQ.  But it does mean that in order to ensure we keep improving the code quality we would like the build to break, so that we quickly see it, and can address the issue.

From what I understand, you're not using a pre-commit hook to prevent code to be pushed into trunk. So this means that you "just" want to be notified when your new changes decrease the overall quality of your code. This need is already covered by the following points:

• Make sure you have a quality gate activated on your project
• Make sure this quality gate checks some conditions (like "new blocker issues" or "new critical issues") based on the "Since previous analysis" differential period
• Subscribe to the "New quality gate status" notification => you will get a mail when a new analysis turns your "green" quality gate into a "red" one 

This will, you'll be notified when commits decrease code quality - based on our requirements. No need to depend on the CI tool for that (@SonarSource, we used to rely on the CI tools but believe me, you don't need them for this need). And the good part of it is that you can already do that with older versions of SQ.

BTW, in SQ 5.2, we have update the default quality gate that we believe should be used by everybody => https://jira.sonarsource.com/browse/SONAR-6143

This is the one we use internally @SonarSource and that helps us to manage our technical debt. Associated to the notification, this is the most powerful solution we've had so far.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/e4be83b8-ed4e-4707-a5ac-16e408627dbe%40googlegroups.com.

[Matthew Harrison]

I guess where we're coming from is that we're ok if code gets committed to trunk that reduces code quality, but would like the build to pick that up and show that this has happened.  In that sense its a bit more than just wanting to be notified - we want to show up that it happened, and to stop the line.  We use the CI server to send out notifications, but probably more of interest is to use a display of the build jobs/pipelines as an information radiator to show if everything is building ok (running unit/component/system tests etc.).

Sending out email notifications from the CI server is helpful, but having the ability to stop the pipeline, and show that up to the team (I think) is a useful function - and I would like that to include maintaining and increasing the code quality as part of that system.  

I'll certainly have a look at the updated quality gate - sounds interesting.

Hopefully I'm explaining my concerns in a way that makes sense?   And can you suggest any ways that I could make this work - maybe we could run a preview analysis as part of the build (so we can see if it will fail the gate), and then run the main analysis?  Or can/will Sonar call a URL on gate failing (maybe I can tie that into the information radiator)? Maybe I could listen for emails (although that sounds like a bad idea :) ).  

Thanks for your help,

Matt  

[mjdet...@gmail.com]

Being able to fail a CI build is a huge loss in functionality in my opinion.  And I certainly don't like that the build breaking feature was removed without the replacement being developed yet.  I don't see how this isn't getting more attention.

It's a good way to put a hard stop in the CI pipeline.  For example, you can stop the build before producing artifacts or moving on to longer running functional tests.  The idea is to stop people from consuming artifacts that do not meet the quality gate.  A preview mode would be ok for checking against the gate, but if you want to record a full analysis then you'd have to run the analysis again (which can take a long time for large projects).

A pre-commit hook of some sorts would be useful but it still would not cover all cases.

[Jorge Costa]

+1

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/304367b7-319b-4bd9-afef-ba4a48b722f0%40googlegroups.com.

[Timothy McNally]

This seems like such a massive loss of functionality with no perceived end user benefits or if there are I am not seeing them.
In our system we use Jenkins to build and test pull requests and flag them as acceptable to be merged or not. This process relies on the ability to run a Sonar preview run with the build breaker plugin in order to fail the build if quality gates are violated. I understand that this is not the "Sonarsource way", but it is a perfectly valid use case that is no longer supported. Also looking at Sonarqube's roadmap it looks like the functional replacement for buildbreaker is not planned until 5.4 which isn't estimated to be completed until April 2016.

[Günter Wirth]

Agree with the others this is a massive loss of functionality. Also wanna have back an option for synchronous analysis: ok on client side should really mean ok.

[Fabrice Bellingard]

Guys, you don't need to argue more. We said that it would be back as a core feature, what do you want more? Removing the connection to the DB was a huge refactoring and we could not wait any longer to release a version that can be deployed and used by thousands of people who don't have the same requirements as yours. If this is a showstopper for you, just don't upgrade to 5.2 and wait for 5.4.

Best regards,

Fabrice BELLINGARD | SonarSource SonarQube Platform Product Manager http://sonarsource.com

On Sun, Nov 8, 2015 at 5:56 PM, Günter Wirth <wirth....@me.com> wrote:

Agree with the others this is a massive loss of functionality.  Also wanna have back an option for synchronous analysis: ok on client side should really mean ok.

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/f822372b-b357-4731-aaea-e104aa606401%40googlegroups.com.

[mjdet...@gmail.com]

I don't think it's arguing. It's more like pleading. Users are trying to make their voices heard. You even said yourself "No need to depend on the CI tool" and the JIRA item for the what if feature does not specifically outline it. Users are trying to make their case for the return of this functionality, not just in preview mode but also the full analysis mode.

[Jorge Costa]

Just in case someone is looking for a way to brake the build if server analysis is broken in server side, ive developed a wrapper for sonar msbuild runner that will that for you: https://github.com/SonarOpenCommunity/sonar-cxx-msbuild-tasks, and https://github.com/SonarOpenCommunity/sonar-cxx-msbuild-tasks/releases/download/1.1/CxxSonarQubeMsbuidRunner.zip 

it was initially devoloped for the cxx community plugin, and it will run all external tools for you. but i am using also for c# and any multilanguage project that uses msbuild runner.

br,

jc

you can find it here: 

[gabriel....@gmail.com]

So the updated version doesn't break the build but rather polls the DB constantly till it determines if there is a failure?

So basically Sonar goes from a way of enforcing and keeping up code quality to a useless scanner that everyone will ignore. Sigh....

[mjdet...@gmail.com]

I've since released updates to make the build breaker compatible for SQ 5.3 and later.

https://github.com/SonarQubeCommunity/sonar-build-breaker

There was another thread discussing the addition of the latest plugin versions to the update center.

[G. Ann Campbell]

Only if you choose to use the BuildBreaker @Gabriel, which is against our recommendations.

Ann

[Peter van Zetten]

I like the "don't use build breaker" viewpoint - in general, it is meaningless to have a build failure in addition to a red quality gate in SQ.

But with that in mind the blog post misses a major case for build breaking in our workflow - pre-merge validation. We use Gerrit as a code review platform and gatekeeper of all our repositories. Many of our projects use the Gerrit Trigger Plugin for Jenkins to define automated steps as 'review' criteria, which gives us a huge amount of control over the repo: "Every commit in isolation must meet conditions X Y and Z or it cannot be merged with the target branch." The simplest control flow is that if the Jenkins job fails, the corresponding flag in Gerrit is set to -1 and the commit is blocked from merging.

For most of our projects, this involves at a minimum building and testing in isolation (no more broken dev/master branch!), but many are using SQ in preview mode to check our quality as well. When you add the build breaker plugin, this causes the jenkins job to fail and the commit is marked rejected in Gerrit (with a nice link to the SQ report). This workflow allows us to pre-emptively block any commit breaching the quality profile - very very useful for QA and requiring developers to follow the standards. Then when a commit has passed all its checks and is merged, we do a proper analysis and the SQ database is updated (along with any other testing/staging/deployment).

So what we're getting from the Gerrit+Jenkins+SQ combo is "if this commit is merged, the quality analysis result will be ...". Adding build breaker gives us "and it cannot be merged until you fix X Y and Z".

The blog post correctly describes why breaking a regular branch build is bad, but ignores this "what if" approach - we absolutely do want a synchronous response in this scenario.

I guess the "correct" solution in the SQ >5.2 world would be for the SonarLint tool to reach some level of maturity by which it can run a one-off analysis against the same profile from the SQ server. Then the Jenkins-Gerrit jobs can just call sonarlint instead of "sonar-runner/mvn sonar:sonar -Dsonar.analysis.mode=preview" and take an error return code accordingly.

I can still imagine some of the quality gate conditions being problematic (e.g. "XXX on new code" - does SonarLint understand "new"?) but it'd be a good start and allow us to reject anything introducing a blocker violation.

[gabriel....@gmail.com]

What your missing is that as a way of enforcing code quality standards this was very meaningful since it prevented bad code from getting in and then having to go back and fix later.  The system wouldn't allow the bad code in to the CI at all, which is very meaningful.  With out that Sonar goes from a useful enforceable quality metric to a useless system that simply gives you code scans that everyone will ignore since the code has been delivered into the system already and do way to stop the process.

Most car makers had a system like this years ago, create the car, then go back in line to fix the problem, rinse repeat.  Toyoto found a better way which was to stop the line and fix the process that was making the problem to ensure it doesn't happen any more. Kanban, agile, etc are based on that model, and Sonar assisted with that by ensuring code quality.  Without that we take a step back in that we now have bad code in the CI and no way to stop it.

[Petr Vilčinský]

Hi,

that's why I wrote the following shell script that I am using in Jenkins.
It checks the report generated in preview mode and if there are CRITICAL or BLOCKER issues it breaks the build.
You also need https://stedolan.github.io/jq/ version 1.4 to parse the report.

cd build/sonar/issues-report
BLOCKER=`{ echo "["; pcregrep -M '(?s)var severityFilter.*?]' issues-report-light.html | tail -n +2 | head -n -1; echo "]"; } | jq14 '.[]' | jq14 'select(.key == "blocker")'.newtotal`
CRITICAL=`{ echo "["; pcregrep -M '(?s)var severityFilter.*?]' issues-report-light.html | tail -n +2 | head -n -1; echo "]"; } | jq14 '.[]' | jq14 'select(.key == "critical")'.newtotal`

if [[ "${BLOCKER}" -eq "0" && "${CRITICAL}" -eq "0" ]]; then
    echo "No new SonarQube issues found."
else
    echo "New SonarQube issues found."
    echo "NEW BLOCKER: ${BLOCKER}"
    echo "NEW CRITICAL: ${CRITICAL}"
    exit 1
fi

Dne úterý 28. června 2016 18:00:48 UTC+2 gabriel....@gmail.com napsal(a):

[Jeff]

Per Matthew DeTullio's message, is build breaker now working again in SQ > 5.3 via community plugin?  (I haven't upgraded to SQ > 5.3 yet)

I struggled with the blog post recommending it not be used.  It feels like a conclusion based on the limited scope of the SonarQube process and assumes everyone else follows that process (and if not, you are doing it wrong).

I believe Build Breaker is very useful and for our process is necessary.  

FWIW, our process is moving toward fully-automated "Continuous Deployment" from check-in to production.  One core tenant of continuous delivery/deployment is that if something breaks, "stop the line! and fix it" as Gabriel mentioned. 

We develop and build (and run SonarQube) inside a secure development environment with very limited access to anything so using a dashboard would be difficult due to our security policy prohibiting external access/visibility into systems with source code.

Additionally, a "notification" is an anti-pattern for continuous deployment unless you can "stop the line".  And while the blog post suggested it is bad that a job could fail (be red) for multiple reasons, I don't happen to agree.  It isn't any different than any other condition (missing tools, mis-configured Job, bad settings, compile error, packaging error, failing unit tests, etc.) I don't have and issue with a 'red' build caused by a Quality Gate.  If the cause of the failure is hard to determine, then I think this is more of an issue with the way a build status is reported more than adding another cause of failure.

Overall, my point is that many processes have different requirements and are all still valid.  Hopefully the Build Breaker works (even as a community plugin) and is continued moving forward.

Regards,

Jeff

--

You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/22820482-8b08-47a0-a3f8-f7df05b4734f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jeff Vincent
See my LinkedIn profile at:
http://www.linkedin.com/in/rjeffreyvincent

--

You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/22820482-8b08-47a0-a3f8-f7df05b4734f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jeff Vincent
See my LinkedIn profile at:
http://www.linkedin.com/in/rjeffreyvincent

[G. Ann Campbell]

Hi guys,

First, yes BuildBreaker works AFAIK. (I haven't used it myself for obvious reasons. ;-) 

Second, to the points about preventing the leak, this is exactly why we introduced SonarLint, which raises issues in your IDE as you type. Altho @Peter, as you spotted, it can't raise issues on coverage, for instance.

There's also pull request analysis, but that's not universally available across SCMs. (And ditto the coverage question.)

We hope to add a post-report-processing webhook mechanism - maybe in 6.1 - to help in this area, and we're also talking about a "what-if mode" to do what you described @Peter, but that's less fully-fleshed right now.

Ann

---

G. Ann CAMPBELL | SonarSource

Product Owner

http://sonarsource.com

--
You received this message because you are subscribed to a topic in the Google Groups "SonarQube" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonarqube/gR07SqqpAyQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/CAPJK9%3DLyT-zydbEz4UXZKjG8SxBgnMBFWag1hxP%2BHEZkce-Omg%40mail.gmail.com.

[gabriel....@gmail.com]

Yes SonarLint is definitely interesting something we need to look into. This depends on users installing and using, But what you are forgetting is if you work in a large company with thousands of developers you need a way to quality check prior to integration into the CI system.  To ensure standards are met, by removing the Build Breaker capability until you provide similar capacity in the 6.x release, where you provide a similar quality check and check in.  You have basically made Sonar nothing but a suggestion.  With no teeth to enforce quality standards.  That is the gap, we have no way to hold the line on the quality standards because in this small company it was more efficient to simply adopt a new approach(which i can see working if you have a team of 50ish) but it fails spectacularly when you are dealing with a much larger group.

[eric.d...@gmail.com]

I completely agree with most of everyone's points here. At the end of the day a codebase that does not pass a quality gate should not end up producing a consumable artifact (i.e. stop the line!). Quality gates mean different things to different organizations (I know for us it has to do with # of issues as well as code coverage). We 65,000 employees in our company with thousands of developers, many of whom are not full-time employees of the company. Do we really expect all of these contingent workers to install Sonarlint into their IDE and configure it to not allow them to check in code into source control unless the quality gates have passed? I'm not so sure.