Code coverage report seems incorrect. Why?

[rell...@gmail.com]

Hello. I use Jacoco with ant and then use sonar to reuse jacoco.exec to show the result. But the result seems incorrect. What's wrong with it? Who knows?

The following is my result:

The test code is:


We can see that the method getUser has been tested and the getRecord.append statements have been covered. But sonar showed the coverage status is red and "0". Only try catch statement is green. 

Meanwhile, I convert jacoco.exec into xml:

 <method name="getSmbUser" desc="(I)Lcom/hp/saas/smbusermgmt/vo/SMBUserBean;" line="241">

<counter type="INSTRUCTION" missed="11" covered="0"/>

<counter type="BRANCH" missed="2" covered="0"/>

<counter type="LINE" missed="3" covered="0"/>

<counter type="COMPLEXITY" missed="2" covered="0"/>

<counter type="METHOD" missed="1" covered="0"/></method>

 

<method name="getSmbUser" desc="(IZ)Lcom/hp/saas/smbusermgmt/vo/SMBUserBean;" line="260">

<counter type="INSTRUCTION" missed="76" covered="0"/>

<counter type="BRANCH" missed="4" covered="0"/>

<counter type="LINE" missed="19" covered="0"/>

<counter type="COMPLEXITY" missed="3" covered="0"/>

<counter type="METHOD" missed="1" covered="0"/></method>

 

The line coverage result is the same as Sonar.

My ant build file about Jacoco is:

<property name="sonar.dynamicAnalysis" value="reuseReports" /> <property name="sonar.java.coveragePlugin" value="jacoco" /> <property name="sonar.jacoco.reportPath" value="target/jacoco.exec" /> <target name="sonar-ut">     <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">                 <classpath path="/home/build/nara/apache-ant-1.7.0/lib/jacocoant.jar" />     </taskdef>     <jacoco:coverage enabled="true" destfile="target/jacoco.exec" xmlns:jacoco="antlib:org.jacoco.ant">          <junit dir="${junit.src.dir}" fork="yes" printsummary="yes" haltonfailure="no" >         <classpath>                  <path refid="junittest.path" />             <pathelement location="${junit.classes.dir}" />              <path refid="sonar.classpath" />                  <fileset refid="sonar.ut.fileset"/>             <pathelement path="${basedir}/MPP/WEB-INF/classes" />             <pathelement path="${basedir}/MPP/WEB-INF/classes/mappingsHPSA" />             <pathelement path="${basedir}/MPP/WEB-INF/i18resources" />             <pathelement location="${basedir}/MPP/WEB-INF/junittests/UnitTest-Resources/spring-conf" />             <pathelement location="${sonar.libraries}" />                  <path refid="junittest.path" />         </classpath>         <formatter type="xml" />         <batchtest fork="yes" todir="${junit.sonar.reports.dir}">                   <fileset dir="${junit.src.dir}" includes="**/*Test.java,**/*test.java" defaultexcludes="no"/>         </batchtest>         </junit>     </jacoco:coverage>  </target>

 

[Henrik Horneber]

Without having looked too closely, this sounds like a case of sonar seeing different classes than jacoco did.   Different compilers, aspects, asm, something like that.

Hope that helps
Henrik

Am 31.10.2013 07:22 schrieb <rell...@gmail.com>:
>
> Hello. I use Jacoco with ant and then use sonar to reuse jacoco.exec to show the result. But the result seems incorrect. What's wrong with it? Who knows?
>
> The following is my result:
>
>
>
>

> The test code is:

> --
> You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[rell...@gmail.com]

Indeed, in Jacoco xml format report which is generated by the same Jacoco.execc, the result is the same as Sonar UI.

I suppose this issue has nothing to do with Sonar.

[Marc R. Hoffmann]

Hi,

1) Do other methods in the same class show any coverage?
2) Can you find your class listed on the "Sessions" page of the HTML report? Is the entry linked?

Best regards,
marc

[rell...@gmail.com]

1) Do other methods in the same class show any coverage?

Yes. The other methods show coverage. But they have the same issues. Out of Try-catch statements all shows no coverage. But indeed, these statements are all on the main path.

2) Can you find your class listed on the "Sessions" page of the HTML report? Is the entry linked?

Yes. I can find the class listed on the "Sessions" page just like the below.

[Marc R. Hoffmann]

> Out of Try-catch statements all shows no coverage.

So your methods do throw exceptions? In this case please check the FAQ  http://www.eclemma.org/jacoco/trunk/doc/faq.html:

Code with exceptions shows no coverage. Why?

JaCoCo determines code execution with so called probes. Probes are inserted into the control flow at certain positions. Code is considered as executed when a subsequent probe has been executed. In case of exceptions such a sequence of instructions is aborted somewhere in the middle and not marked as executed.

Cheers,
-marc

[rell...@gmail.com]

I don't understand clearly. 

My method is defined throw Exception and I have two unit test cases: normal case and exception-thrown case. The two tests are all green.

We also analyzed the coverage by eclipse with EclEmma. The result is right. 

EclEmma is based on Jacoco. Why the results are not the same?

[Marc R. Hoffmann]

This is strange. Are you sure the normal case is also executed when running outside Eclipse? E.g. can you insert a log statement in the very last line of the method?

[rell...@gmail.com]

Yes. I am sure. Another method just like this:

Object getXXXDetails(String name) Throw XXXException {

    statement 1;

    try {

         statement 2;

    } catch (XXException e) {

         statement 3;

    }

Code coverage shows:

1. statement 1 and statement 3 are uncovered. 

2. statement 2 is covered.

I think if statement 3 is uncovered, it means it is a normal case. So statement 1 and statement 2 should be covered. 

But why statement 1 shows uncovered?

[Marc R. Hoffmann]

What is the exact version (full qualified) of JaCoCo you're using?

Can you provide a minimal reproducer?

[rell...@gmail.com]

I am sorry for late. We use the latest version of Jacoco.

We did some test and wanted to reproduce this issue. 

We found if we only run the test cases for this class, the code coverage result seems right. 

But if we run all of test cases for the whole project, as you see, the result seems wrong.

Do you any reason would lead to the result?

[Marc R. Hoffmann]

Can your please answer the question I asked before: "What is the exact version (full qualified) of JaCoCo you're using?"

I'm asking because there are several forks out there and I want wo ensure we're talking about the same code base.

Thanks,
-marc

[rell...@gmail.com]

Jacoco ant task version, is it all right?

0.63.201306030806.

[rell...@gmail.com]

Sorry, 0.6.3.201306030806

[Marc R. Hoffmann]

The version looks good, this is the latest official JaCoCo release.

Just to double check: If you create a native HTML report directly in your build do the lines appear covered?

If there is any chance to provide a reproducer this would help a lot.

Cheers,
-marc