Jacoco code coverage is affected by AspectJ

[zhz shi]

Hi,

We're using AspectJ in our project and also Jacoco for test coverage report, currently we're facing an issue that due to AspectJ changed the byte code during compiling phase, which makes the code coverage report not correct. One example is due to AspectJ adds extra if-else statement, then the branch coverage shows something like 1/4 but actually there's no condition branch in the source code. Is there some good way to tell Jacoco to ignore all code generated by AspectJ?

I also created an issues 335 on Github and posted an question on SO. 

Thanks a lot.

[Marc Hoffmann]

Hi,

JaCoCo works on class files only. It reports whatever it sees in the
class files which get instrumented and analyzed. Future versions might
come with the option to filter certain constructs, but for AspectJ this
will probably not work as you can inject any code.

To solve this you have to make sure that the class files get instrumeted
by JaCoCo *before* AspectJ kicks in. If you use the AspectJ agent it has
to be configured after the JaCoCo agent. If you want to pre-compile your
classes you can use JaCoCo offline instrumentation first:
http://www.eclemma.org/jacoco/trunk/doc/offline.html

Regards,
-marc

> I also created an issues 335 [1] on Github and posted an question on
> SO [2].
>
> Thanks a lot.
>
> --
> You received this message because you are subscribed to the Google
> Groups "JaCoCo and EclEmma Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jacoco+un...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jacoco/e96a08b4-8c44-469a-b32b-52f013e57bbd%40googlegroups.com
> [3].
> For more options, visit https://groups.google.com/d/optout [4].
>
>
> Links:
> ------
> [1] https://github.com/jacoco/jacoco/issues/335
> [2]
> http://stackoverflow.com/questions/32001843/jacoco-code-coverage-is-affected-by-aspectj
> [3]
> https://groups.google.com/d/msgid/jacoco/e96a08b4-8c44-469a-b32b-52f013e57bbd%40googlegroups.com?utm_medium=email&utm_source=footer
> [4] https://groups.google.com/d/optout

[zhz shi]

Thanks @marc for your information. I'm trying to use the jacoco offline instrumentation with the following configuration:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.5.201505241946</version>
    <executions>
        <execution>
            <id>default-instrument</id>
            <goals>
                <goal>instrument</goal>
            </goals>
        </execution>
        <execution>
            <id>default-restore-instrumented-classes</id>
            <goals>
                <goal>restore-instrumented-classes</goal>
            </goals>
        </execution>
        <execution>
            <id>default-report</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12.2</version>
    <configuration>
        <systemPropertyVariables>
            <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
    </configuration>
</plugin>

and also with dependency:

<dependency>
    <groupId>org.jacoco</groupId>
    <artifactId>org.jacoco.agent</artifactId>
    <classifier>runtime</classifier>
    <version>0.7.5.201505241946</version>
</dependency>

but all tests failed due to org/jacoco/agent/rt/internal_b0d6a23/Offline can not be found. Is there anything wrong with my configration?

[Marc R. Hoffmann]

Looks like for whatever reason the JaCoCo runtime is not on the classpath or cannot be loaded in your testsetup.

You might check with the -X Maven option how the command line for your test runtime looks like.

Also here you can find a working example: http://www.eclemma.org/jacoco/trunk/doc/examples/build/pom-offline.xml

Regards,
-marc

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/088f94d1-850a-4c7b-a420-ef39147daf80%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[zhz shi]

Thanks Marc.

For 'To solve this you have to make sure that the class files get instrumeted by JaCoCo *before* AspectJ kicks in. ',  is it possible to make this happen if we're using AspectJ Compile Time Weaving, because I see ajc directly compile java source file to class files. 

[Marc Hoffmann]

Hi,

you might diskuss this question with the ApectJ project, we can't
provide support here. I think they have load time weaving options.

Regards,
-marc

On 2015-08-17 05:44, zhz shi wrote:
> Thanks Marc.
>
> For '_TO SOLVE THIS YOU HAVE TO MAKE SURE THAT THE CLASS FILES GET
> INSTRUMETED BY JACOCO *BEFORE* ASPECTJ KICKS IN. _', is it possible

> to make this happen if we're using AspectJ Compile Time Weaving,
> because I see ajc directly compile java source file to class files.
>
> On Saturday, August 15, 2015 at 12:38:25 AM UTC+8, Marc R. Hoffmann
> wrote:
>
>> Looks like for whatever reason the JaCoCo runtime is not on the
>> classpath or cannot be loaded in your testsetup.
>>
>> You might check with the -X Maven option how the command line for
>> your test runtime looks like.
>>
>> Also here you can find a working example:
>>
> http://www.eclemma.org/jacoco/trunk/doc/examples/build/pom-offline.xml

>> [1]

>> http://www.eclemma.org/jacoco/trunk/doc/offline.html [2]

>>> [1] https://github.com/jacoco/jacoco/issues/335 [5]
>>> [2]
>>>
>>
> http://stackoverflow.com/questions/32001843/jacoco-code-coverage-is-affected-by-aspectj
>> [6]
>>> [3]
>>>
>>
> https://groups.google.com/d/msgid/jacoco/e96a08b4-8c44-469a-b32b-52f013e57bbd%40googlegroups.com?utm_medium=email&utm_source=footer
>> [7]
>>> [4] https://groups.google.com/d/optout [4]

>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "JaCoCo and EclEmma Users" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to jacoco+un...@googlegroups.com.
>> To view this discussion on the web visit
>>

> https://groups.google.com/d/msgid/jacoco/088f94d1-850a-4c7b-a420-ef39147daf80%40googlegroups.com
>> [8].

>> For more options, visit https://groups.google.com/d/optout [4].
>

> --
> You received this message because you are subscribed to the Google
> Groups "JaCoCo and EclEmma Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jacoco+un...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/jacoco/0c4de744-f40b-4d9b-b6f9-ec7362d6ce68%40googlegroups.com
> [9].

> For more options, visit https://groups.google.com/d/optout [4].
>
>
> Links:
> ------
> [1]

> http://www.eclemma.org/jacoco/trunk/doc/examples/build/pom-offline.xml
> [2] http://www.eclemma.org/jacoco/trunk/doc/offline.html
> [3]
> https://groups.google.com/d/msgid/jacoco/e96a08b4-8c44-469a-b32b-52f013e57bbd%40googlegroups.com
> [4] https://groups.google.com/d/optout
> [5] https://github.com/jacoco/jacoco/issues/335
> [6]
> http://stackoverflow.com/questions/32001843/jacoco-code-coverage-is-affected-by-aspectj
> [7]
> https://groups.google.com/d/msgid/jacoco/e96a08b4-8c44-469a-b32b-52f013e57bbd%40googlegroups.com?utm_medium=email&utm_source=footer
> [8]
> https://groups.google.com/d/msgid/jacoco/088f94d1-850a-4c7b-a420-ef39147daf80%40googlegroups.com?utm_medium=email&utm_source=footer
> [9]
> https://groups.google.com/d/msgid/jacoco/0c4de744-f40b-4d9b-b6f9-ec7362d6ce68%40googlegroups.com?utm_medium=email&utm_source=footer

[zhz shi]

Thanks Marc again :-)

[Alexander Kriegisch]

You have two options with AspectJ if you want to avoid it compiling from source:

1) Use LTW with the weaving agent.

2) Move your aspects into a separate Maven module. Compile your Java modules with the normal Maven Compiler Plugin and the aspect module with AspectJ Maven. Then create another module which just uses AspectJ Maven in order to do binary weaving on a Java module, using both previously created artifacts as dependencies. In this scenario you need to make sure that JaCoCo offline instrumentation is bound to a phase before binary weaving is done.

The easiest way out, though, would be to test your aspects in isolation and also the Java code without aspects and measure coverage there without any issues.