Seeing different coverage metrics for the same set of jacoco binaries analyzed on two different machines

[plaxmi...@gmail.com]

Hello,

I'm having an issue with inconsistent IT code coverage metrics reported to SonarQube server:

- I'm running sonar analysis (mvn sonar:sonar) on two different machines via jenkins job

- both use the same jacoco binaries generated for IT coverage.

- However, on machine 1, sonar shows 35% while on machine 2 for the same project, it shows 42%. I'm not able to reproduce lower coverage on machine 2.

- both machines use same jenkins job configurations, pom configurations and jacoco binaries.

Details of both machines:

machine 1: Jenkins 1.580.2, SonarQube 3.7, Maven 3.1.1, sonar-maven-plugin 2.6, jacoco-maven-plugin 0.7.1.201405082137, jdk 1.6.0_30. 

OS on machine1: CentOS

machine 2: Jenkins 1.580, SonarQube 3.7, Maven 3.1.1, sonar-maven-plugin 2.6, jacoco-maven-plugin 0.7.1.201405082137, jdk 1.6.0_65.

OS on machine1: MacOS X 10.9.5

Can anyone please help with this issue?

Thanks,

Lakshmi

[Freddy Mallet]

Hi Lakshmi,

Don't know exactly where to start to help you investigate this issue ... Could you first confirm that the JaCoCo reports used to feed the SonarQube analysis is exactly the same one on both machine ? And by the way with version of SonarQube and of the SonarQube Java plugin are you using ?

Thanks

Freddy MALLET | SonarSource

Product Director & Co-Founder
http://sonarsource.com

--
You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/6d56570f-e8f9-42bb-b3d8-0d0e40edccf9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[plaxmi...@gmail.com]

Hi Freddy,

Thanks for responding and sorry for the long delay in replying.

I did verify that the jacoco reports used are exactly the same on both machines and both the machines have SonarQube 3.7 and SonarQube java plugin 1.3.

Any pointers to debug the issue?

Appreciate any help in this regard!

Thanks,

Lakshmi

[plaxmi...@gmail.com]

Hi Freddy,

Just wanted to provide few more details that we started noticing, if that helps in investigating the issue further:

1) some of the classes (new java classes that we added) are not displayed in "IT Coverage" component but they are displayed in "Classes" component with appropriate coverage.

2) some of the classes are reported with incorrect coverage (in both "IT Coverage" component and "Classes" component) though they have been covered in integrations tests and the tests have been executed successfully.

Also wanted to update that the database that we use on machine 1 is MySql while on machine 2 it's the embedded h2 database.

Any help is highly appreciated!

Thanks,

Lakshmi

[Nicolas Peru]

Hi, 

The version you are mentioning of the SonarQube platform and java plugin are quite old and no longer supported. Please first try to upgrade those version to (at least) LTS version of the platform (4.5.x) and tell us if you still encounter the issue. 

Please note that the latest versions of the platform and java plugin will require java 7. 

Cheers,

Nicolas PERU | SonarSource

Senior Developer
http://sonarsource.com

https://twitter.com/benzonico

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/1d190470-9e9d-4a10-9c54-91685689222f%40googlegroups.com.

[plaxmi...@gmail.com]

Hi,

I upgraded my local instance to SonarQube 4.5.5 and now IT coverage is showing 0% for the same project. I ran the analysis with the same jacoco binaries (generated using 0.7.1.xx jacoco agent). I tried with Java plugin 2.4 and also 2.1 (along with jacoco plugin 2.1). This is a multi-module maven project and it seems to be finding the sources, the class files and the jacoco binary correctly, but coverage is now 0%. Here is an excerpt from the sonar analysis log:

[INFO] --- sonar-maven-plugin:2.6:sonar (default-cli) @ team-sonar-selenium ---

[INFO] SonarQube version: 4.5.5

INFO: Default locale: "en_US", source code encoding: "UTF-8"

INFO: Work directory: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/team-sonar/target/sonar

INFO: SonarQube Server 4.5.5

[INFO] [09:56:20.271] Load global referentials...

[INFO] [09:56:20.367] Load global referentials done: 98 ms

[INFO] [09:56:20.392] User cache: /Users/lakshmi/.sonar/cache

[INFO] [09:56:20.397] Install plugins

[INFO] [09:56:20.620] Install JDBC driver

[INFO] [09:56:20.624] Create JDBC datasource for jdbc:h2:tcp://localhost/sonar

[INFO] [09:56:21.447] Initializing Hibernate

[INFO] [09:56:22.519] Load project referentials...

[INFO] [09:56:22.658] Load project referentials done: 139 ms

[INFO] [09:56:22.658] Load project settings

[INFO] [09:56:23.258] Loading technical debt model...

[INFO] [09:56:23.272] Loading technical debt model done: 14 ms

[INFO] [09:56:23.274] Apply project exclusions

[INFO] [09:56:23.488] -------------  Scan sonar-selenium sonar-selenium-track-reporting

[INFO] [09:56:23.490] Load module settings

[INFO] [09:56:23.743] Loading rules...

[INFO] [09:56:24.153] Loading rules done: 410 ms

[INFO] [09:56:24.172] Configure Maven plugins

[INFO] [09:56:24.279] Compare to previous analysis (2015-08-21)

[INFO] [09:56:24.291] Compare over 30 days (2015-07-22, analysis of 2015-08-21 09:52:12.324)

[INFO] [09:56:24.292] No quality gate is configured.

[INFO] [09:56:24.403] Initializer FindbugsMavenInitializer...

[INFO] [09:56:24.404] Initializer FindbugsMavenInitializer done: 1 ms

[INFO] [09:56:24.404] Base dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-selenium

[INFO] [09:56:24.404] Working dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-selenium/target/sonar

[INFO] [09:56:24.404] Source paths: pom.xml, src/main/java

[INFO] [09:56:24.404] Test paths: src/test/java

[INFO] [09:56:24.404] Binary dirs: target/classes

[INFO] [09:56:24.404] Source encoding: UTF-8, default locale: en_US

[INFO] [09:56:24.404] Index files

[INFO] [09:56:24.734] 227 files indexed

[INFO] [09:56:25.557] Quality profile for java: Sonar way

[INFO] [09:56:25.615] Sensor JavaSquidSensor...

[INFO] [09:56:25.721] Java Main Files AST scan...

[INFO] [09:56:25.724] 117 source files to be analyzed

[INFO] [09:56:38.062] 117/117 source files analyzed

[INFO] [09:56:38.085] Java Main Files AST scan done: 12364 ms

[INFO] [09:56:38.089] Java bytecode scan...

[INFO] [09:56:38.564] Java bytecode scan done: 475 ms

[INFO] [09:56:38.564] Java Test Files AST scan...

[INFO] [09:56:38.564] 110 source files to be analyzed

[INFO] [09:56:39.062] Java Test Files AST scan done: 498 ms

[INFO] [09:56:39.062] 110/110 source files analyzed

[INFO] [09:56:39.088] Package design analysis...

[INFO] [09:56:39.179] Package design analysis done: 91 ms

[INFO] [09:56:39.813] Sensor JavaSquidSensor done: 14198 ms

[INFO] [09:56:39.814] Sensor QProfileSensor...

[INFO] [09:56:39.819] Sensor QProfileSensor done: 5 ms

[INFO] [09:56:39.819] Sensor InitialOpenIssuesSensor...

[INFO] [09:56:40.141] Sensor InitialOpenIssuesSensor done: 322 ms

[INFO] [09:56:40.141] Sensor ProjectLinksSensor...

[INFO] [09:56:40.148] Sensor ProjectLinksSensor done: 7 ms

[INFO] [09:56:40.148] Sensor VersionEventsSensor...

[INFO] [09:56:40.161] Sensor VersionEventsSensor done: 13 ms

[INFO] [09:56:40.161] Sensor FileHashSensor...

[INFO] [09:56:40.172] Sensor FileHashSensor done: 11 ms

[INFO] [09:56:40.172] Sensor SurefireSensor...

[INFO] [09:56:40.172] parsing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-selenium/target/surefire-reports

[INFO] [09:56:40.267] Sensor SurefireSensor done: 95 ms

[INFO] [09:56:40.267] Sensor JaCoCoItSensor...

[INFO] [09:56:40.270] Analysing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/team-sonar/target/sonar-merged-jacoco.bin

[INFO] [09:56:40.634] No information about coverage per test.

[INFO] [09:56:40.634] Sensor JaCoCoItSensor done: 367 ms

[INFO] [09:56:40.634] Sensor JaCoCoOverallSensor...

[INFO] [09:56:40.750] Analysing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-selenium/target/sonar/jacoco-overall.exec

[INFO] [09:56:40.941] No information about coverage per test.

[INFO] [09:56:40.941] Sensor JaCoCoOverallSensor done: 307 ms

[INFO] [09:56:40.941] Sensor Maven dependencies...

[INFO] [09:56:41.973] Sensor Maven dependencies done: 1032 ms

[INFO] [09:56:41.973] Sensor JaCoCoSensor...

[INFO] [09:56:41.975] Analysing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-selenium/target/jacoco.exec

[INFO] [09:56:42.081] No information about coverage per test.

[INFO] [09:56:42.081] Sensor JaCoCoSensor done: 108 ms

[INFO] [09:56:42.081] Sensor CPD Sensor (wrapped)...

[INFO] [09:56:42.082] JavaCpdEngine is used for java

[INFO] [09:56:42.090] Cross-project analysis disabled. Not supported on project branches.

[INFO] [09:56:42.349] Sensor CPD Sensor (wrapped) done: 268 ms

[INFO] [09:56:42.652] Execute decorators...

[INFO] [09:56:44.760] -------------  Scan sonar Test Tools sonar-selenium-track-reporting

[INFO] [09:56:44.762] Load module settings

[INFO] [09:56:44.820] Configure Maven plugins

[INFO] [09:56:44.822] Compare to previous analysis (2015-08-21)

[INFO] [09:56:44.823] Compare over 30 days (2015-07-22, analysis of 2015-08-21 09:52:12.324)

[INFO] [09:56:44.823] No quality gate is configured.

[INFO] [09:56:44.848] Initializer FindbugsMavenInitializer...

[INFO] [09:56:44.848] Initializer FindbugsMavenInitializer done: 0 ms

[INFO] [09:56:44.848] Base dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-test-tools

[INFO] [09:56:44.848] Working dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-test-tools/target/sonar

[INFO] [09:56:44.848] Source paths: pom.xml, src/main/java

[INFO] [09:56:44.849] Test paths: src/test/java

[INFO] [09:56:44.849] Binary dirs: target/classes

[INFO] [09:56:44.849] Source encoding: UTF-8, default locale: en_US

[INFO] [09:56:44.849] Index files

[INFO] [09:56:44.909] 120 files indexed

[INFO] [09:56:45.466] Quality profile for java: Sonar way

[INFO] [09:56:45.474] Sensor JavaSquidSensor...

[INFO] [09:56:45.485] Java Main Files AST scan...

[INFO] [09:56:45.486] 75 source files to be analyzed

[INFO] [09:56:53.788] 75/75 source files analyzed

[INFO] [09:56:53.797] Java Main Files AST scan done: 8312 ms

[INFO] [09:56:53.799] Java bytecode scan...

[INFO] [09:56:53.964] Java bytecode scan done: 165 ms

[INFO] [09:56:53.964] Java Test Files AST scan...

[INFO] [09:56:53.964] 45 source files to be analyzed

[INFO] [09:56:54.040] Java Test Files AST scan done: 76 ms

[INFO] [09:56:54.040] 45/45 source files analyzed

[INFO] [09:56:54.044] Package design analysis...

[INFO] [09:56:54.061] Package design analysis done: 17 ms

[INFO] [09:56:54.116] Sensor JavaSquidSensor done: 8642 ms

[INFO] [09:56:54.117] Sensor QProfileSensor...

[INFO] [09:56:54.117] Sensor QProfileSensor done: 0 ms

[INFO] [09:56:54.117] Sensor InitialOpenIssuesSensor...

[INFO] [09:56:54.179] Sensor InitialOpenIssuesSensor done: 62 ms

[INFO] [09:56:54.179] Sensor ProjectLinksSensor...

[INFO] [09:56:54.183] Sensor ProjectLinksSensor done: 4 ms

[INFO] [09:56:54.183] Sensor VersionEventsSensor...

[INFO] [09:56:54.185] Sensor VersionEventsSensor done: 2 ms

[INFO] [09:56:54.185] Sensor FileHashSensor...

[INFO] [09:56:54.187] Sensor FileHashSensor done: 2 ms

[INFO] [09:56:54.187] Sensor SurefireSensor...

[INFO] [09:56:54.187] parsing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/commons-test/sonar-test-tools/target/surefire-reports

[INFO] [09:56:54.196] Sensor SurefireSensor done: 9 ms

[INFO] [09:56:54.196] Sensor JaCoCoItSensor...

[INFO] [09:56:54.197] Analysing /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/team-sonar/target/sonar-merged-jacoco.bin

[INFO] [09:56:54.226] No information about coverage per test.

[INFO] [09:56:54.226] Sensor JaCoCoItSensor done: 30 ms

[INFO] [09:56:54.226] Sensor Maven dependencies...

[INFO] [09:56:54.451] Sensor Maven dependencies done: 225 ms

[INFO] [09:56:54.452] Sensor CPD Sensor (wrapped)...

[INFO] [09:56:54.452] JavaCpdEngine is used for java

[INFO] [09:56:54.454] Cross-project analysis disabled. Not supported on project branches.

[INFO] [09:56:54.536] Sensor CPD Sensor (wrapped) done: 84 ms

[INFO] [09:56:54.693] Execute decorators...

[INFO] [09:56:55.296] -------------  Scan sonar Selenium sonar-selenium-track-reporting

[INFO] [09:56:55.298] Load module settings

[INFO] [09:56:55.326] Configure Maven plugins

[INFO] [09:56:55.327] Compare to previous analysis (2015-08-21)

[INFO] [09:56:55.328] Compare over 30 days (2015-07-22, analysis of 2015-08-21 09:52:12.324)

[INFO] [09:56:55.328] No quality gate is configured.

[INFO] [09:56:55.349] Initializer FindbugsMavenInitializer...

[INFO] [09:56:55.350] Initializer FindbugsMavenInitializer done: 1 ms

[INFO] [09:56:55.350] Base dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/team-sonar

[INFO] [09:56:55.350] Working dir: /Users/lakshmi/my-jenkins/my/work/workspace/sonar-selenium-reporting/team-sonar/target/sonar

[INFO] [09:56:55.350] Source encoding: UTF-8, default locale: en_US

[INFO] [09:56:55.351] Sensor InitialOpenIssuesSensor...

[INFO] [09:56:55.365] Sensor InitialOpenIssuesSensor done: 14 ms

[INFO] [09:56:55.366] Sensor ProjectLinksSensor...

[INFO] [09:56:55.373] Sensor ProjectLinksSensor done: 7 ms

[INFO] [09:56:55.373] Sensor VersionEventsSensor...

[INFO] [09:56:55.376] Sensor VersionEventsSensor done: 3 ms

[INFO] [09:56:55.376] Sensor FileHashSensor...

[INFO] [09:56:55.376] Sensor FileHashSensor done: 0 ms

[INFO] [09:56:55.376] Sensor Maven dependencies...

[INFO] [09:56:55.377] Sensor Maven dependencies done: 1 ms

[INFO] [09:56:55.377] Sensor CPD Sensor (wrapped)...

[INFO] [09:56:55.377] Sensor CPD Sensor (wrapped) done: 0 ms

[INFO] [09:56:55.455] Execute decorators...

[INFO] [09:56:55.476] Store results in database

[INFO] [09:56:57.093] ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard/index/com.mycompany:team-sonar-selenium:sonar-selenium-branch-reporting

[INFO] [09:56:57.131] Executing post-job class org.sonar.plugins.core.issue.notification.SendIssueNotificationsPostJob

[INFO] [09:56:57.145] Executing post-job class org.sonar.plugins.core.batch.IndexProjectPostJob

[INFO] [09:56:57.168] Executing post-job class org.sonar.plugins.dbcleaner.ProjectPurgePostJob

[INFO] [09:56:57.174] -> Keep one snapshot per day between 2015-07-24 and 2015-08-20

[INFO] [09:56:57.174] -> Keep one snapshot per week between 2014-08-22 and 2015-07-24

[INFO] [09:56:57.174] -> Keep one snapshot per month between 2010-08-27 and 2014-08-22

[INFO] [09:56:57.174] -> Delete data prior to: 2010-08-27

[INFO] [09:56:57.177] -> Clean sonar Selenium sonar-selenium-branch-reporting [id=696]

[INFO] [09:56:57.179] <- Clean snapshot 7893

[INFO] [09:56:57.500] -> Clean sonar-selenium sonar-selenium-branch-reporting [id=697]

[INFO] [09:56:57.502] -> Clean sonar Test Tools sonar-selenium-branch-reporting [id=698]

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO] 

[INFO] sonar Tools .................................. SKIPPED

[INFO] sonar-selenium .................................... SKIPPED

[INFO] sonar Selenium .................................... SUCCESS [41.866s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 42.806s

[INFO] Finished at: Fri Aug 21 09:56:57 PDT 2015

[INFO] Final Memory: 31M/265M

[INFO] ------------------------------------------------------------------------

Sonar analysis completed: SUCCESS

Finished: SUCCESS

Can you please let me know if you notice anything wrong from this log? I ran it with -Dsonar.verbose=true but that log is huge and has all our class names and packages, so I'm not sure if I can paste it here.

We are using jdk 1.6 for the builds:

      <plugin>

        <artifactId>maven-compiler-plugin</artifactId>

        <version>2.3.2</version>

        <configuration>

          <source>1.6</source>

          <target>1.6</target>

        </configuration>

      </plugin>

Any help will be greatly appreciated.

Thanks,

Lakshmi

[sonih...@gmail.com]

Is there any update on this thread ? Is it resolved and if yes then how..Beacuse I am also facing the same issue.

Thanks!

Hardik

[Nicolas Peru]

Hi, 

Given that this thread is quite old and a lot has happen since then, can you please open a new thread describing your problem without forgetting to mention which version of SonarQube and SonarJava you are using. Please also try to share logs and specify which scanner you use to run the analysis in order to help us understand your issue.

Thanks.

--

You received this message because you are subscribed to the Google Groups "SonarQube" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonarqube+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sonarqube/60e2021e-ff98-4dbb-8911-e1796672e0ba%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Nicolas Peru | SonarSource

Senior Developer

https://twitter.com/benzonico

https://sonarsource.com