Test coverage from running spring-boot application not recorded

[Martin Dietze]

I am working on a spring-boot webservice using maven and the spring-boot-maven-plugin. As usual, unit and integration tests are executed separately.

For integration tests I use the spring-boot-maven-plugin where I run 'start' in the phase 'pre-integration-test' to start the spring boot application and then 'stop' in the pase 'post-integration-test' to shut it down again.

There is no issue with the unit tests, however when running integration tests, the coverage from tests sending requests to the running webservice application is not recorded. I don't see any error message.

My pom.xml contains these relevant sections:

      <plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <version>${jacoco.version}</version>
        <configuration>
          <excludes>
            <exclude>com/mycompany/generated/**/*</exclude>
          </excludes>
        </configuration>
        <executions>
          <execution>
            <id>jacoco-initialize-ut</id>
            <phase>initialize</phase>
            <goals>
              <goal>prepare-agent</goal>
            </goals>
          </execution>
          <execution>
            <id>jacoco-initialize-it</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>prepare-agent-integration</goal>
            </goals>
          </execution>
          <execution>
            <id>jacoco-report</id>
            <phase>verify</phase>
            <goals>
              <goal>report</goal>
              <goal>report-integration</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>${springboot.version}</version>
        <executions>
          <execution>
            <id>pre-integration-test</id>
            <goals>
              <goal>start</goal>
            </goals>
            <configuration>
              <arguments>
                <argument>--integration-test</argument>
              </arguments>
              <maxAttempts>3</maxAttempts>
              <wait>10000</wait>
              <fork>true</fork>
            </configuration>
          </execution>
          <execution>
            <id>post-integration-test</id>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <configuration>
          <groups>${integrationTestGroups}</groups>
          <includes>
            <include>**/*.java</include>
          </includes>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

As seen here I am sticking pretty much to the defaults.

Any idea what I am doing wrong?

[Evgeny Mandrikov]

As stated in documentation about "prepare-agent" mojo at page http://www.jacoco.org/jacoco/trunk/doc/prepare-agent-mojo.html :

Prepares a property pointing to the JaCoCo runtime agent that can be passed as a VM argument to the application under test.

By default it will set "argLine" property, but spring-boot-maven-plugin doesn't take into account this property by default and you're not passing it explicitly , hence spring-boot-maven-plugin starts JVM without JaCoCo agent.

[Martin Dietze]

Am Mittwoch, 14. Juni 2017 20:49:27 UTC+2 schrieb Evgeny Mandrikov:

By default it will set "argLine" property, but spring-boot-maven-plugin doesn't take into account this property by default and you're not passing it explicitly , hence spring-boot-maven-plugin starts JVM without JaCoCo agent.

Thank you for your reply. I tried to set this in both the failsafe and spring-boot maven plugin, but it had no effect. Is there an example that I could take a look at?

[Evgeny Mandrikov]

Personally I'm not aware of such example and not being a daily user of spring-boot don't want to spend time on building one, however:

If you can provide minimalistic example, then I can have a look at it.

Also setting of JVM arguments for spring-boot-maven-plugin should be as easy as described in their documentation - https://docs.spring.io/spring-boot/docs/current/maven-plugin/examples/run-debug.html

So something like:

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<executions>

  <execution>

    <id>jacoco-initialize-it</id>

    <phase>pre-integration-test</phase>

    <goals>

      <goal>prepare-agent</goal>

    </goals>

    <configuration>

      <propertyName>jacocoAgent</propertyName>

    </configuration>

  </execution>

and then

<groupId>org.springframework.boot</groupId>

<artifactId>spring-boot-maven-plugin</artifactId>

<configuration>

  <jvmArguments>${jacocoAgent}</jvmArguments>

</configuration>

And so question is - is it what you did?

Regards,

Evgeny

[Martin Dietze]

Am Donnerstag, 15. Juni 2017 15:20:34 UTC+2 schrieb Evgeny Mandrikov:

Also setting of JVM arguments for spring-boot-maven-plugin should be as easy as described in their documentation - https://docs.spring.io/spring-boot/docs/current/maven-plugin/examples/run-debug.html

I tried this:

[jacoco plugin]

     <execution>

      <id>jacoco-initialize-it</id>

      <phase>pre-integration-test</phase>

      <goals>

        <goal>prepare-agent-integration</goal>

      </goals>

    </execution>

 

i.e. No plugin name explicitly set.

[spring-boot plugin]

    <execution>

      <id>pre-integration-test</id>

      <goals>

        <goal>start</goal>

      </goals>

      <configuration>

        <arguments>

          <argument>--integration-test</argument>

        </arguments>

        <maxAttempts>3</maxAttempts>

        <wait>10000</wait>

        <fork>true</fork>

        <jvmArguments>${argLine}</jvmArguments>

      </configuration>

    </execution>

I.e. use the standard property name.

While the spring boot app was up I took a look at the process list and found that indeed my webservice was running with JVM-args like this: -javaagent:/path/to/jacoco.agent-0.7.9-runtime.jar=destfile=...

Hence the property obviously got passed successfully. 

This has brought me one step further: My webservice tests are recorded now, but only when I eliminate all other TestNG tests that are not sending webservice requests (e.g. testing services inside a test spring context etc.).

Is there any way to get this together?

[Martin Dietze]

Am Donnerstag, 15. Juni 2017 16:48:12 UTC+2 schrieb Martin Dietze:

 

i.e. No plugin name explicitly set.

Correction: should be: "no property name explicitly set". 

[Evgeny Mandrikov]

On Thursday, June 15, 2017 at 4:48:12 PM UTC+2, Martin Dietze wrote:

Is there any way to get this together?

Excuse me for repetition, but: if you can provide minimalistic example demonstrating your issue, then I can have a look at it. Otherwise, not being a daily user of spring-boot, cost to build something similar to your setup is pretty high (web services, spring-boot, testng, etc) and without guarantees to see the exact same behavior as you experience.

Presence of example/reproducer always speeds-up things - as examples: https://github.com/jacoco/jacoco/pull/462 and https://github.com/jacoco/jacoco/issues/394

[Martin Dietze]

Am Donnerstag, 15. Juni 2017 17:06:59 UTC+2 schrieb Evgeny Mandrikov:

Excuse me for repetition, but: if you can provide minimalistic example demonstrating your issue, then I can have a look at it. Otherwise, not being a daily user of spring-boot, cost to build something similar to your setup is pretty high (web services, spring-boot, testng, etc) and without guarantees to see the exact same behavior as you experience.

Presence of example/reproducer always speeds-up things - as examples: https://github.com/jacoco/jacoco/pull/462 and https://github.com/jacoco/jacoco/issues/394

It looks like it is working now with the configuration I posted above. Thank you very much!

[Evgeny Mandrikov]

Glad that your issue resolved.