Quality Gate for new code coverage never failing

[Kumar Gaurav]

Dear all.. I am trying to evaluate quality gates related to new changes. I have a quality gate for new code coverage and it never fails, even its clear that condition is broken. In log everything looks fine. 

My sonar version is: 6.7.1 (build 35068). I have "sonar.scm.disabled" set to False (default) in SCM Settings. I have github url in "sonar.links.scm" as well.. And I have SonarQube :: Plugins :: SCM :: Git version 1.3.0.869 installed. I am also bit surprised that its not asking for any github credential anywhere.

Following is the analysis log:

[INFO] -------------  Scan Code Analyzer Maven Plugin

[INFO] Load server rules

[INFO] Load server rules (done) | time=1552ms

[INFO] Base dir: /Users/kgaurav2/Documents/code-analysis

[INFO] Working dir: /Users/kgaurav2/Documents/code-analysis/target/sonar

[INFO] Source paths: pom.xml, src/main/java

[INFO] Source encoding: UTF-8, default locale: en_AU

[INFO] Index files

[INFO] 14 files indexed

[INFO] Quality profile for java: java-quality-profile

[INFO] Quality profile for xml: xml-quality-profile

[INFO] Sensor JavaSquidSensor [java]

[INFO] Configured Java source version (sonar.java.source): 8

[INFO] JavaClasspath initialization

[INFO] JavaClasspath initialization (done) | time=30ms

[INFO] JavaTestClasspath initialization

[INFO] JavaTestClasspath initialization (done) | time=7ms

[INFO] Java Main Files AST scan

[INFO] 13 source files to be analyzed

[INFO] 13/13 source files have been analyzed

[WARNING] Classes not found during the analysis : [javax.annotation.meta.When]

[INFO] Java Main Files AST scan (done) | time=2624ms

[INFO] Java Test Files AST scan

[INFO] 0 source files to be analyzed

[INFO] 0/0 source files have been analyzed

[INFO] Java Test Files AST scan (done) | time=6ms

[INFO] Sensor JavaSquidSensor [java] (done) | time=3349ms

[INFO] Sensor CoberturaSensor [cobertura]

[WARNING] Cobertura report not found at /Users/kgaurav2/Documents/code-analysis/target/site/cobertura/coverage.xml

[INFO] Sensor CoberturaSensor [cobertura] (done) | time=1ms

[INFO] Sensor Embedded CSS Analyzer Sensor [css]

[INFO] 0 source files to be analyzed

[INFO] Sensor Embedded CSS Analyzer Sensor [css] (done) | time=102ms

[INFO] 0/0 source files have been analyzed

[INFO] Sensor SurefireSensor [java]

[INFO] parsing [/Users/kgaurav2/Documents/code-analysis/target/surefire-reports]

[INFO] Sensor SurefireSensor [java] (done) | time=2ms

[INFO] Sensor JaCoCoSensor [java]

[INFO] Sensor JaCoCoSensor [java] (done) | time=3ms

[INFO] Sensor SonarJavaXmlFileSensor [java]

[INFO] 1 source files to be analyzed

[INFO] Sensor SonarJavaXmlFileSensor [java] (done) | time=762ms

[INFO] 1/1 source files have been analyzed

[INFO] Sensor XML Sensor [xml]

[INFO] Sensor XML Sensor [xml] (done) | time=179ms

[INFO] Sensor Zero Coverage Sensor

[INFO] Sensor Zero Coverage Sensor (done) | time=40ms

[INFO] Sensor CPD Block Indexer

[INFO] Sensor CPD Block Indexer (done) | time=71ms

[INFO] SCM provider for this project is: git

[INFO] 14 files to be analyzed

[INFO] 14/14 files analyzed

[INFO] 4 files had no CPD blocks

[INFO] Calculating CPD for 9 files

[INFO] CPD calculation finished

[INFO] Analysis report generated in 141ms, dir size=129 KB

[INFO] Analysis reports compressed in 63ms, zip size=53 KB

[INFO] Analysis report uploaded in 2822ms

I am not sure whether any other settings is needed. Does anyone have an idea of what might be going on. Please let me know if any other information is needed.

Thank a lot,

Kumar

[nicolas...@sonarsource.com]

Hi Kumar,

Analysis doesn't need github credentials because it's running in a checked-out version of your project, so SCM information is available locally. You can see that the SCM integration is doing its job in fact: [INFO] SCM provider for this project is: git (in the logs).

Seems like your coverage report is not found though: [WARNING] Cobertura report not found at /Users/kgaurav2/Documents/code-analysis/target/site/cobertura/coverage.xml . That's one area you might want to look into.

More generally speaking I recommend you put Quality Gate aside for now: check the numbers you get on the project homepage, confirm that you do have coverage on new code computed and reported there, and once that is sorted out then Quality Gate conditions are just a set of conditions on top of the data you're seeing (i.e. right now it seems like your problem may be more with the data computation itself, then the QualityGate logic).

Best regards,

Nicolas

[Kumar Gaurav]

Thanks Nicolas for the explanation,

I have Cobertura plugin installed on sonarqube but I am not running cobertura for test coverage.. I am running jacoco instead.

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install org.sonarsource.scanner.maven:sonar-maven-plugin:sonar

I see the numbers for overall code coverage.. but nothing shows under 'On new code'. Screenshot:

Do you think cobertura plugin might be messing things up?

Thanks a lot for help,

Regards,

Kumar 

[Kumar Gaurav]

As additional info: I can see people's name assigned to issues. That I assume is coming from scm.

Thanks

Kumar

[nicolas...@sonarsource.com]

Hi Kumar,

Indeed things seem alright with respect to SCM integration . Can you check the Size facet on the left and see if you actually have New Lines reported ? (see here for example) Otherwise you could then check your Leak Period settings, make a dummy test by pushing new fake code, and see how it's reported in SonarQube. (don't focus on coverage for now, but only look at whether New Lines are reported and/or highlighted in the code).

Best regards,

Nicolas

[Kumar Gaurav]

Hi Nicolas,

Thanks for looking into it. I don't see "New Lines" reported in Size facet on the left. My Leak Period Setting is 'previous_analysis' . Following is the screenshot of my left size facet.

Thanks & Regards,

Kumar

[nicolas...@sonarsource.com]

So what if you add code, commit it, then run a new analysis: is the new code visible in SonarQube ? are there 'New Lines' reported ? As discussed, I recommend you poke around with a sample project, pushing changes, analysing them, changing your Leak Period (e.g. using the recommended 'since_previous_version'), to understand the mechanics and expectations.

Best regards,

Nicolas

[Kumar Gaurav]

Hi Nicolas, sorry for late reply. Holiday time :). So, I tried different things and it was not working.. I had this doubt that database might have some issues because same image was working with embedded database. So I changed my database from mysql -> psql and installed latest version of sonarqube. Its working completely fine!

Thanks for your help,

Regards,

Kumar