Maven Site generation is not working with JaCoCo reports

[David Hoffer]

I am using jacoco-maven-plugin v0.8.0 to generate test code coverage reports (and checks) to the build. This is working fine for normal dev builds, e.g. mvn clean install.

However we also need to include these reports in the Maven site that is generated and published that part is not working.

The build is a multi-module build and I see you have a link to some ways of creating an aggregated report, https://github.com/jacoco/jacoco/wiki/MavenMultiModule, however I am not to the part yet. Right now I'm just trying to get it to publish the individual reports in the site build.

The problem is that the site build is including JaCoCo but its labeled as JaCoCo Aggregate on each of the modules (and parent) and each of these point to 'target/site/jacoco-aggregate/index.html' however that folder does not exist. The actual generated report is at target/site/jacoco-ut/index.html.

How can I configure JaCoCo so that it reports the correct name and location for each report? I have tried various configuration options in the reporting section but to no avail. It seems hard coded to use ReportAggregateMojo when it should use ReportMojo and I need to provide the outputDirectory when used indirectly via the site plugin. I will include my config below.

build...

<plugin>
        <groupId>org.jacoco</groupId>
        <artifactId>jacoco-maven-plugin</artifactId>
        <executions>
            <!--
    Prepares the property pointing to the JaCoCo runtime agent which
    is passed as VM argument when Maven the Surefire plugin is executed.
-->
            <execution>
                <id>jacoco-pre-unit-test</id>
                <goals>
                    <goal>prepare-agent</goal>
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution data. -->
                    <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
                    <!--
                        Sets the name of the property containing the settings
                        for JaCoCo runtime agent.
                    -->
                    <propertyName>surefireArgLine</propertyName>
                </configuration>
            </execution>

            <!--
    Ensures that the code coverage report for unit tests is created after
    unit tests have been run.
-->
            <execution>
                <id>jacoco-post-unit-test</id>
                <phase>test</phase>
                <goals>
                    <goal>report</goal>
                    <!--<goal>report-aggregate</goal>-->
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution data. -->
                    <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                    <!-- Sets the output directory for the code coverage report. -->
                    <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
                </configuration>
            </execution>

            <!--Ensures that the build has a minimum level of test coverage-->
            <execution>
                <id>jacoco-check</id>
                <phase>test</phase>
                <goals>
                    <goal>check</goal>
                </goals>
                <configuration>
                    <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                    <haltOnFailure>true</haltOnFailure>
                    <rules>
                        <rule implementation="org.jacoco.maven.RuleConfiguration">
                            <!--BUNDLE, PACKAGE, CLASS, SOURCEFILE or METHOD-->
                            <element>BUNDLE</element>
                            <limits>
                                <limit implementation="org.jacoco.report.check.Limit">
                                    <!--INSTRUCTION, LINE, BRANCH, COMPLEXITY, METHOD, CLASS-->
                                    <counter>INSTRUCTION</counter>
                                    <!--TOTALCOUNT, COVEREDCOUNT, MISSEDCOUNT, COVEREDRATIO, MISSEDRATIO-->
                                    <value>COVEREDRATIO</value>
                                    <minimum>${jacoco.percentage.instruction}</minimum>
                                </limit>
                                <limit implementation="org.jacoco.report.check.Limit">
                                    <!--INSTRUCTION, LINE, BRANCH, COMPLEXITY, METHOD, CLASS-->
                                    <counter>BRANCH</counter>
                                    <!--TOTALCOUNT, COVEREDCOUNT, MISSEDCOUNT, COVEREDRATIO, MISSEDRATIO-->
                                    <value>COVEREDRATIO</value>
                                    <minimum>${jacoco.percentage.branch}</minimum>
                                </limit>                                
                            </limits>
                        </rule>
                    </rules>
                </configuration>
            </execution>

            <!-- The Executions required by unit tests are omitted. -->
            <!--
                Prepares the property pointing to the JaCoCo runtime agent which
                is passed as VM argument when Maven the Failsafe plugin is executed.
            -->
            <execution>
                <id>jacoco-pre-integration-test</id>
                <phase>pre-integration-test</phase>
                <goals>
                    <goal>prepare-agent</goal>
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution data. -->
                    <destFile>${jacoco.it.execution.data.file}</destFile>
                    <!--
                        Sets the name of the property containing the settings
                        for JaCoCo runtime agent.
                    -->
                    <propertyName>failsafeArgLine</propertyName>
                </configuration>
            </execution>
            <!--
                Ensures that the code coverage report for integration tests after
                integration tests have been run.
            -->
            <execution>
                <id>jacoco-post-integration-test</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>report</goal>
                    <!--<goal>report-integration</goal>-->
                </goals>
                <configuration>
                    <!-- Sets the path to the file which contains the execution data. -->
                    <dataFile>${jacoco.ut.execution.data.file}</dataFile>
                    <!-- Sets the output directory for the code coverage report. -->
                    <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
                </configuration>
            </execution>

        </executions>
    </plugin>

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.15</version>
        <configuration>
            <!-- Sets the VM argument line used when unit tests are run. -->
            <argLine>${surefireArgLine}</argLine>
            <!-- Skips unit tests if the value of skip.unit.tests property is true -->
            <skipTests>${skip.unit.tests}</skipTests>
            <!-- Excludes integration tests when unit tests are run. -->
            <excludes>
                <exclude>**/IT*.java</exclude>
            </excludes>
        </configuration>
    </plugin>

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.15</version>
        <executions>
            <!--
                Ensures that both integration-test and verify goals of the Failsafe Maven
                plugin are executed.
            -->
            <execution>
                <id>integration-tests</id>
                <goals>
                    <goal>integration-test</goal>
                    <goal>verify</goal>
                </goals>
                <configuration>
                    <!-- Sets the VM argument line used when integration tests are run. -->
                    <argLine>${failsafeArgLine}</argLine>
                    <!--
                        Skips integration tests if the value of skip.integration.tests property
                        is true
                    -->
                    <skipTests>${skip.integration.tests}</skipTests>
                </configuration>
            </execution>
        </executions>
    </plugin>

    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-site-plugin</artifactId>
        <version>3.7</version>
    </plugin>

reporting...

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.0</version>
</plugin>

Note I'm not running IT tests yet.  But ideally when we do I'd like to be able to publish both sets of reports in the site.

My command line to generate the site is mvn clean package site

-Dave

[David Hoffer]

I see that the jacoco-maven-plugin has 3 reporting Mojos, ReportMojo, ReportITMojo & ReportAggregateMojo so to state my question another way...how do I configure the site phase to use ReportMojo and/or ReportITMojo ?  It seems it currently is using ReportAggregateMojo and it's not at all clear how to get it to use either of the others.  How can I configure it to use the others?

-Dave

[Evgeny Mandrikov]

As stated in documentation - http://www.jacoco.org/jacoco/trunk/doc/maven.html :

Usage of plugin together with maven-site-plugin without explicit selection of reports might lead to generation of redundant aggregate reports. Specify reportSets explicitly to avoid this

In attachment you'll find example of project, for which "mvn clean verify && mvn site:site" executes only ReportMojo producing "a/target/site/project-reports.html" and "b/target/site/project-reports.html" with a links to individual reports for modules "a" and "b" respectively:

[David Hoffer]

Thanks for the reply.  Yes yesterday I was finally able to achieve what you described.  However there are several issues/problems with this behavior.

1. According to the Maven reporting documentation https://maven.apache.org/plugins/maven-site-plugin/examples/configuring-reports.html all reporting goals for plugins are supposed to be rendered once.  So since this plugin has 3 reporting goals it should have rendered 3 reports (w/o any report sets).

2. Also as report sets are specified one should be able to configure each as necessary as the same link states.  Also I believe the Maven spec states that if no configuration is specified (per report) then it should get its configuration from the same plugin in the build section.  However that is not the case here.  Your sample (as did I) had to add a global configuration in the reporting section else it does not work.  So there are two parts to this, this should be specifiable via report as when I run the other two report goals it will require a different configuration, second it should get its configuration from the build section if not specified in reporting.  If the plugin followed these conventions there would have been no confusion on how this works.

3. I'd like to suggest this documentation and examples be more front and center on the website, I did eventually find the link you provided but not without much difficulty.  This is a great plugin and having it follow standard conventions and putting the docs more up front would have really helped and I think my use case is pretty standard.

Thanks much for your help.

-Dave

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/VAhaG3zJqVE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jacoco+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/35dca1b7-6f9f-496d-9e13-352fd41fea3e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Evgeny Mandrikov]

Hi Dave,

On Wednesday, January 31, 2018 at 4:03:40 PM UTC+1, David Hoffer wrote:

Thanks for the reply.  Yes yesterday I was finally able to achieve what you described.  However there are several issues/problems with this behavior.

2. Also as report sets are specified one should be able to configure each as necessary as the same link states. Also I believe the Maven spec states that if no configuration is specified (per report) then it should get its configuration from the same plugin in the build section.  However that is not the case here.

Where this is stated in Maven spec?

https://maven.apache.org/guides/mini/guide-configuring-plugins.html#Configuring_Reporting_Plugins states:

"

Configuring a reporting plugin in the <reporting> or <build> elements in the pom does NOT have the same behavior!
mvn site
It uses only the parameters defined in the <configuration> element of each reporting Plugin specified in the <reporting> element, i.e. site always ignores the parameters defined in the <configuration> element of each plugin specified in <build>.

"

1. According to the Maven reporting documentation https://maven.apache.org/plugins/maven-site-plugin/examples/configuring-reports.html all reporting goals for plugins are supposed to be rendered once.  So since this plugin has 3 reporting goals it should have rendered 3 reports (w/o any report sets).

Yes - in absence of explicit selection all 3 mojos are executed - you can see their appearance in "mvn site:site -X"

However because as explained above ReportMojo and ReportITMojo are not configured, they can't find customized exec file. In absence of exec they can generate report, but it will be showing zero coverage for all classes of module. And AFAIR actually this was the case in past and in response to previous requests from users was decided to not generate such misleading report, produce two following messages instead, and therefore to not produce broken links:

[INFO] --- maven-site-plugin:3.3:site (default-cli) @ b ---

[INFO] configuring report plugin org.jacoco:jacoco-maven-plugin:0.8.1-SNAPSHOT

[INFO] Skipping JaCoCo execution due to missing execution data file.

[INFO] Skipping JaCoCo execution due to missing execution data file.

ReportAggregateMojo generates absolutely empty report in such case, which is easier to diagnose than zeros.

Indeed we can improve above messages, however we have not thought about this given the amount of people talking about "mvn site" in this mailing list (and under "jacoco" tag on StackOverflow) in past two years compared to other questions.

 

  Your sample (as did I) had to add a global configuration in the reporting section else it does not work.  So there are two parts to this, this should be specifiable via report as when I run the other two report goals it will require a different configuration, second it should get its configuration from the build section if not specified in reporting.  If the plugin followed these conventions there would have been no confusion on how this works.

In addition to what was said above - we don't have any code that prevents these goals from taking configuration from "build" section. If you able to find such, then please let us know ;)

 

3. I'd like to suggest this documentation and examples be more front and center on the website, I did eventually find the link you provided but not without much difficulty.  This is a great plugin and having it follow standard conventions and putting the docs more up front would have really helped and I think my use case is pretty standard.

AFAIK http://www.jacoco.org/jacoco/ is specified as JaCoCo homepage everywhere where we can specify and then one click is needed to reach documentation at http://www.jacoco.org/jacoco/trunk/doc/ with a plain list of topics.

If by "more front and center" you mean that http://www.jacoco.org/ currently opens Eclipse EclEmma project homepage, then I agree that we should improve this - this is legacy of past where JaCoCo was a tiny and not so popular as today sub-project of EclEmma and even didn't had its own domain. Please bear in mind that list of active contributors - is a very small team of volunteers (2 as of today) that maintain and develop JaCoCo and EclEmma projects in spare time, when they have it. I personally dream that one day I will have enough time to invest into modernization of site, not counting many other dreams and low priority tasks ;)

If you refer to something else, then please elaborate. In any case - we'll appreciate if you can provide annotated screenshots with explanations/suggestions.

Regards,

Evgeny