Question about Jacoco and Maven incremental builds

[Elizabeth Sirkova]

Hi guys,

(I am not sure this is the right place to ask, but at this moment it's a bit unclear to me where our problem comes from, so I am hoping someone can point me in the right direction / or if someone has experienced the same to share their knowledge...)

We are looking into using the Maven build cache extension for incremental builds in order to speed up our CI and local builds. In our CI pipeline we have a step where we run tests and generate test coverage via Jacoco (which is later used by Sonar). Unfortunately, we experience issues with having the incremental builds/cache work when we generate code coverage reports.

For instance, whenever we run `mvn jacoco:prepare-agent test jacoco:report` the remote/local cache is never used. However, if we remove the `jacoco` parts from the command, then the cache is utilized.

We use:
* Java   - 17.0.9 (OpenJDK)
* Maven  - 3.9.6
* Jacoco - 0.8.11

For running tests we use the `maven-surefire-plugin`, which is configured like this:
```
<properties>
<argLine></argLine>
<surefireArgLine></surefireArgLine>
...
</properties>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<!-- https://maven.apache.org/surefire/maven-surefire-plugin/faq.html#late-property-evaluation -->
<configuration>
<argLine>@{argLine} ${surefireArgLine}</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${maven-jacoco-plugin.version}</version>
</plugin>
```

Our `maven-build-cache-config.xml` for the incremental builds is the following:

```
<?xml version="1.0" encoding="UTF-8" ?>
<cache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/BUILD-CACHE-CONFIG/1.0.0"
   xsi:schemaLocation="http://maven.apache.org/BUILD-CACHE-CONFIG/1.0.0 https://maven.apache.org/xsd/build-cache-config-1.0.0.xsd">
<configuration>
<enabled>true</enabled>
<hashAlgorithm>XX</hashAlgorithm>
<remote enabled="true" id="xxxx">
<url>xxxx</url>
</remote>
</configuration>
<executionControl>
<runAlways>
<executions>
<execution groupId="me.qoomon" artifactId="maven-git-versioning-extension">
<execIds>
<execId>git-versioning</execId>
</execIds>
</execution>
</executions>
<goalsLists>
<goalsList artifactId="maven-install-plugin">
<goals>
<goal>install</goal>
</goals>
</goalsList>
</goalsLists>
</runAlways>
<reconcile>
<plugins>
<plugin artifactId="maven-surefire-plugin" goal="test">
<reconciles>
<reconcile propertyName="skip" skipValue="true"/>
<reconcile propertyName="skipExec" skipValue="true"/>
<reconcile propertyName="skipTests" skipValue="true"/>
<reconcile propertyName="testFailureIgnore" skipValue="true"/>
</reconciles>
</plugin>
</plugins>
</reconcile>
</executionControl>
</cache>
```

Do you have any advise/suggestions how to make the Maven incremental builds + Jacoco code coverage work together?

Kind regards,

Elizabeth

[Marc Hoffmann]

Hi Elizabeth,

I had a quick look at the documentation and it says that there is a problem when using extra command line options: As these are not part of the effective POM they will break the cache key. Maybe they disable caching when additional goals are given for that reason. I would try the following:

1) Try adding another goal instead of JaCoCo, for example versions:display-dependency-updates. Does the cache work in this case?

2) Enable code coverage directly in your POM and see whether the cache works if you run the build without extra goals on the command line.

Regards,

-marc

--
You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/35d92af4-e0d4-4b06-aea5-5312807e65fen%40googlegroups.com.

[Elizabeth Sirkova]

Hi Marc,

Thank you for looking into this! You are right about the issue being the extra command line options

> 1) Try adding another goal instead of JaCoCo, for example versions:display-dependency-updates. Does the cache work in this case?

In my case the caching didn't work

> 2) Enable code coverage directly in your POM and see whether the cache works if you run the build without extra goals on the command line.

I think this actually works. I configured Jacoco like this in order to execute it always:

<plugin>
  <groupId>org.jacoco</groupId>
  <artifactId>jacoco-maven-plugin</artifactId>

  <executions>
   <execution>
     <id>pre-test</id>
      <goals>
         <goal>prepare-agent</goal>
      </goals>
   </execution>
   <execution>
     <id>post-test</id>
     <phase>test</phase>
     <goals>
         <goal>report</goal>
     </goals>
   </execution>
  </executions>
</plugin>

So now when running the `maven install` I see that the jacoco coverage is generated and that the cache is populated. Repeating `maven install` is actually reusing the cache as it shows:

```

[INFO] Local build found by checksum cb9673d03a972fd4
[INFO] Found cached build, restoring com.relay42.apps.routing:routing-app from cache by checksum cb9673d03a972fd4
[INFO] Skipping plugin execution (cached): enforcer:enforce
[INFO] Skipping plugin execution (cached): jacoco:prepare-agent

```

Now I will be trying to see if it's necessary for the test coverage to be included in the cached artifact (perhaps via Attached outputs) to make it work with Sonar.

Thank you again for the help!

Cheers,

Elizabeth

[Marc Hoffmann]

Hi Elizabeth,

Thanks for sharing your results here! Great that you found the cause!

Regards,

-marc 

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/0a2c558e-01e0-446e-9d95-9892cff1f778n%40googlegroups.com.

[Elizabeth Sirkova]

Hi, 

Just to follow up on including the test coverage as part of the cached artifact. It looks like it's indeed possible by configuring attachedOutputs for the maven-build cache extension configuration. For instance, this seems to work for me:

<configuration>
  <enabled>true</enabled>
  <hashAlgorithm>XX</hashAlgorithm>
  <remote enabled="true" id="xxx">

    <url>xxx</url>
  </remote>
  <attachedOutputs>
    <dirNames>
      <dirName>site/jacoco</dirName>
    </dirNames>
  </attachedOutputs>

</configuration>

 

Again, thank you for the help!

Best regards,

Elizabeth

[Abhishek Singh]

Hi Guys, recently I started working on integrating Maven Build Cache extension into our Multi Module Project
I'm also caching test reports to the build can be faster, however I'm facing with this issue
"Unsupported phase: sonar:sonar" and due to which cache is not utilized.

Some Analysis :- It always fails with the above error If I use maven 3.9.0 so I upgraded my local maven and runner maven to 3.9.6 however it fails on gitlab runner but gets passed on local system

Java Version used -: 17.0.9
Maven Version 3.9.6

Maven command :- mvn clean package sonar:sonar -Dsonartokens urls and etc


Local build found by checksum d73234c6f0575ab192af2177fc929534192b35f4c43b232ec33ab550
[INFO] Found cached build, restoring project from cache by checksum d73234c6f0575ab192af2177fc929534192b35f4c43b232ec33ab550
[ERROR] Failed to restore project
java.lang.IllegalArgumentException: Unsupported phase: sonar:sonar
    at org.apache.maven.buildcache.LifecyclePhasesHelper.isLaterPhase (LifecyclePhasesHelper.java:135)
    at org.apache.maven.buildcache.LifecyclePhasesHelper.isLaterPhaseThanBuild (LifecyclePhasesHelper.java:120)
    at org.apache.maven.buildcache.LifecyclePhasesHelper.getCachedSegment (LifecyclePhasesHelper.java:205)
    at org.apache.maven.buildcache.CacheControllerImpl.analyzeResult (CacheControllerImpl.java:265)
    at org.apache.maven.buildcache.CacheControllerImpl.findLocalBuild (CacheControllerImpl.java:234)
    at org.apache.maven.buildcache.CacheControllerImpl.findCachedBuild (CacheControllerImpl.java:185)
    at org.apache.maven.buildcache.BuildCacheMojosExecutionStrategy.execute (BuildCacheMojosExecutionStrategy.java:114)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:568)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
[INFO] Local build was not found by checksum d73234c6f0575ab192af2177fc929534192b35f4c43b232ec33ab550 for project

Regards,
Abhishek Singh

“This email and it’s contents are confidential and intended for the addressee. If you receive this message by mistake, please inform the sender immediately and destroy all copies of this email. It is prohibited to share this email with any third party without the written consent of the sender. Recipients should check the email for threats as the sender accepts no liability for any damage suffered by viewing the contents of this email.”