Is it possible to generate a jacoco report from an existing jacoco exec file?

[vin...@xwiki.com]

Hi guys,

I have the following setup:
- github repo 1 (https://github.com/xwiki/xwiki-commons)
- github repo 2 (https://github.com/xwiki/xwiki-rendering)

I'd like to have a report that combines all the coverage generated by all tests from all modules (including coverage generated by modules on other modules) for both repos.

So I execute the following:
- on repo1: mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=false
- on repo2: mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=true

So this should generate a combined jacoco.exec that contains everything.

However I can't find a way to generate a report out the exec file.

If I run "mvn jacoco:report -Djacoco.destFile=/tmp/jacoco.exec -N" then I get "Skipping JaCoCo execution due to missing execution data file.".

This suggest that I may not be using this properly.

However I can't find a way to do what I want.

Any idea?

PS: Thanks a lot for this great tool!

[Evgeny Mandrikov]

Hi,

Name of a property for "report" is "jacoco.dataFile" and not "jacoco.destFile" - see http://www.eclemma.org/jacoco/trunk/doc/report-mojo.html#dataFile

However this property was added in version 0.7.8 that is not yet released (presented only in snapshot version) - see http://www.eclemma.org/jacoco/trunk/doc/changes.html

But prior to 0.7.8 you can define your own property as following:

<plugin>

  <groupId>org.jacoco</groupId>

  <artifactId>jacoco-maven-plugin</artifactId>

  <configuration>

    <dataFile>${jacoco.dataFile}</dataFile>

  </configuration>

</plugin>

HTH,

Evgeny

[Vincent Massol]

Hi Evgeny,

> On 02 Dec 2016, at 17:30, Evgeny Mandrikov <mand...@gmail.com> wrote:
>
> Hi,
>
> Name of a property for "report" is "jacoco.dataFile" and not "jacoco.destFile" - see http://www.eclemma.org/jacoco/trunk/doc/report-mojo.html#dataFile
> However this property was added in version 0.7.8 that is not yet released (presented only in snapshot version) - see http://www.eclemma.org/jacoco/trunk/doc/changes.html
> But prior to 0.7.8 you can define your own property as following:
> <plugin>
> <groupId>org.jacoco</groupId>
> <artifactId>jacoco-maven-plugin</artifactId>
> <configuration>
> <dataFile>${jacoco.dataFile}</dataFile>
> </configuration>
> </plugin>

Thanks a lot for your answer. I’ve tried your solution and it kind of worked since I was able to generate a report but the report is empty. Is it because the project where the report is generated from also needs to contain the sources matching the generated data in jacoco.exec? If so, then I don’t see how I could achieve the use case I described.

Would you have any idea about how to achieve it?

Thanks again
-Vincent

[Evgeny Mandrikov]

Maybe I don't understand something, but as long as class files on disk match class files used at runtime (see http://www.eclemma.org/jacoco/trunk/doc/classids.html) you should be able to achieve what you want as following:

cd repo1 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=false 

cd repo2 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=true

cd repo1 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec

cd repo2 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec

[Vincent Massol]

Hi Evgeny,

> On 3 Dec 2016, at 06:22, Evgeny Mandrikov <mand...@gmail.com> wrote:
>
> Maybe I don't understand something, but as long as class files on disk match class files used at runtime (see http://www.eclemma.org/jacoco/trunk/doc/classids.html) you should be able to achieve what you want as following:
>
> cd repo1 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=false
> cd repo2 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=true
> cd repo1 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec
> cd repo2 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec

Thanks but my goal is to generate a single aggregated Jacoco report for the 2 repos.

Also note that when I mention repo1 and repo2, they each contain multiple maven projects and the sources are thus scattered in various maven modules.

In short, there are tests located in repo2 that end up calling code whose sources are in repo1 and I’d like the test coverage to take that into account.

More precisely, when I build the maven projects from repo1, the binaries are put in my local maven repo. When I build the maven projects from repo2, they have maven dependencies which are honored from my local repo.

If I understand correctly, the jacoco exec file contains classids and this looks correct since the classes should have the same id when loaded from tests in repo1 or from tests in repo2.

However the issues seems to be to find out how to tell jacoco:report where to find the java sources to construct the report. Right?

Any idea?

Thanks again for your help, very appreciated!
-Vincent

[Vincent Massol]

> On 6 Dec 2016, at 16:18, Vincent Massol <vin...@xwiki.com> wrote:
>
> Hi Evgeny,
>
>> On 3 Dec 2016, at 06:22, Evgeny Mandrikov <mand...@gmail.com> wrote:
>>
>> Maybe I don't understand something, but as long as class files on disk match class files used at runtime (see http://www.eclemma.org/jacoco/trunk/doc/classids.html) you should be able to achieve what you want as following:
>>
>> cd repo1 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=false
>> cd repo2 && mvn clean jacoco:prepare-agent install -Djacoco.destFile=/tmp/jacoco.exec -Djacoco.append=true
>> cd repo1 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec
>> cd repo2 && mvn jacoco:report -Djacoco.dataFile=/tmp/jacoco.exec
>
> Thanks but my goal is to generate a single aggregated Jacoco report for the 2 repos.
>
> Also note that when I mention repo1 and repo2, they each contain multiple maven projects and the sources are thus scattered in various maven modules.
>
> In short, there are tests located in repo2 that end up calling code whose sources are in repo1 and I’d like the test coverage to take that into account.
>
> More precisely, when I build the maven projects from repo1, the binaries are put in my local maven repo. When I build the maven projects from repo2, they have maven dependencies which are honored from my local repo.
>
> If I understand correctly, the jacoco exec file contains classids and this looks correct since the classes should have the same id when loaded from tests in repo1 or from tests in repo2.
>
> However the issues seems to be to find out how to tell jacoco:report where to find the java sources to construct the report. Right?
>
> Any idea?

Note that using report-aggregate wouldn’t work for me since it means having a single maven reactor (just checked the source code), which I don’t have.

Also I mentioned repo1 and repo2 in my original mail as an example, but this for the xwiki project and I have 4 large repos in practice, and even if I could have a single reactor job would never have enough memory (close to ten hours of overall build…).

Last but not least, it’s not possible to build the whole of XWiki as a single reactor since there are some maven voodoo in some places that don’t work in a single reactor execution...

So I need to find a solution that allows to accumulate the test coverage data in a single exec file and then somehow have jacoco find all the sources to be able to construct the report.

I’m not overly confident that this can be achieved and that’s sad. Right now I’m able to do this with Clover (http://dev.xwiki.org/xwiki/bin/view/Community/Testing#HUsingClover-1) but I’d have really liked to use an open source tool and be able to do it with jacoco :)

Thanks
-Vincent

[Evgeny Mandrikov]

Unfortunately jacoco-maven-plugin is of no help in your scenario.

"report-aggregate" as of today is a best trade off that we can provide considering the way Maven works, information that is needed for generation of report and most common usage scenarios.

I'd be glad to understand how you manage to do this with clover-maven-plugin, but unfortunately links that you provide do not contain enough technical information about the way it actually works. Also digging in this direction is complicated by the fact that Clover is commercial product.

Having said all this bad news, there are still some good ones: I believe that there is a way for you to proceed with integration of JaCoCo. Generation of report requires 3 components - exec file, class files and source files. Shouldn't be hard to collect all this information knowing the layout of your projects and invoke JaCoCo Ant Task for report generation via maven-antrun-plugin.

Hope this helps.

Regards,

Evgeny

[Vincent Massol]

Hi Evgeny,

> On 9 Dec 2016, at 22:44, Evgeny Mandrikov <mand...@gmail.com> wrote:
>
> Unfortunately jacoco-maven-plugin is of no help in your scenario.
>
> "report-aggregate" as of today is a best trade off that we can provide considering the way Maven works, information that is needed for generation of report and most common usage scenarios.
>
> I'd be glad to understand how you manage to do this with clover-maven-plugin, but unfortunately links that you provide do not contain enough technical information about the way it actually works. Also digging in this direction is complicated by the fact that Clover is commercial product.

I’ve blogged about it here:
https://t.co/uvEKwgspRC

In short you tell the Clover plugin where to store the data and you store them in the same location for all reactor builds (same as what I did with jacoco with the -Djacoco.destFile=/tmp/jacoco.exec).

I think the difference is that Clover saves full data in that location (and not just class ids) which allows to run "mvn clover:report” on that location without requiring the sources to be present at that time since they’re already in the saved data...

> Having said all this bad news, there are still some good ones: I believe that there is a way for you to proceed with integration of JaCoCo. Generation of report requires 3 components - exec file, class files and source files. Shouldn't be hard to collect all this information knowing the layout of your projects and invoke JaCoCo Ant Task for report generation via maven-antrun-plugin.

Thanks for the idea. I’ve tried to implement it and I’m very close I think since I’m able to generate a jacoco report.

I have got this this:
https://gist.github.com/vmassol/87199d6697ba6ba57894b62be72fb618

However the sources are not in the report so I’m not referencing them properly. However I’m not sure how to do that generically for all maven modules (I cannot specify them one by one since I have between 200-300 java modules in total…).

I’m referring to the <sourcefiles> element in:

...
<sourcefiles>
<fileset dir="${commons.directory}">
<include name="**/src/main/java/**/*.*"/>
</fileset>
</sourcefiles>...

Any idea?

Thanks
-Vincent

[Evgeny Mandrikov]

On Sun, Dec 11, 2016 at 9:33 PM Vincent Massol <vin...@massol.net> wrote:

However the sources are not in the report so I’m not referencing them properly. However I’m not sure how to do that generically for all maven modules (I cannot specify them one by one since I have between 200-300 java modules in total…).

I’m referring to the <sourcefiles> element in:

...
                      <sourcefiles>
                        <fileset dir="${commons.directory}">
                          <include name="**/src/main/java/**/*.*"/>
                        </fileset>
                      </sourcefiles>...

Any idea?

Well to be honest I'm not an Ant guru, but my wild guess will be that you probably should use multirootfileset ( https://ant.apache.org/manual/Types/multirootfileset.html ) instead of fileset.

HTH,

Evgeny