Figures in summary

[Hynek Mlnařík]

Hi,

first of all, thanks for all your efforts in bringing JaCoCo to Jenkins!

In our project, we have difficulties with total number of classes/methods/lines/etc. They number in sumary is twice as large as the actual count. For example, there are 4 classes (each of them is displayed as 1/0 - not covered), but the number in summary is eight, see attached picture. Is this a bug in the plugin or somewhere in my setup? JaCoCo XML contains only four classes so I rather suspect the plugin.

Could you please help me to get correct values displayed?

Thanks

--Hynek

[H Mlnarik]

One addendum which might help - there are three jacoco.xml files processed at once. Only one of them contains the package that contains the classes in question.

Regards,

--Hynek

[Ognjen Bubalo]

Hi Hynek,

Thank you for the feedback! I am pretty sure that this is a bug in the plugin. I'll  fix it asap.

Hmm, why are 3 xml files generated? Are there 3 modules?

Br,
Ogi

[Mickael Istria]

On 06/20/2012 01:57 PM, Ognjen Bubalo wrote:

Hmm, why are 3 xml files generated? Are there 3 modules?

I think that's because current implementation of jacoco-maven-plugin:reports generates one report file for each module.

--
Mickael Istria
Eclipse developer at JBoss, by Red Hat
My blog - My Tweets

[H Mlnarik]

Hi Ogi,

thanks for quick reply. Reason for three xml files is that there are three modules in our project for which we gather statistics separately; still we want to aggregate the results from the files and display it in one graph.

Best,

--Hynek

[Ognjen Bubalo]

Hello!

Marc could you help me in this please?

I thought I found the problem and fixed it. I tried it on my dummy project and it looked ok, but when I saw the failing testcases I asked Jonathan to be sure I am on the right way. He enlightened me that maybe I will have to merge the listed covered lines.

So before I go further:

In the jacoco.xml there are counter lines after every source:  E.g. <counter covered="22" missed="3" type="LINE"/>

This means that the sum of covered lines is 22? So if I have tcs which cover same lines it is counted as 1 here? 

Thank you,

Ogi

[Marc R. Hoffmann]

Ho Ogi,

JaCoCo does not count the times a line, method, etc. has been hit.
Therefore the summaries do not depend on the number of test cases, it's
the total number of lines that have been hit at leat once.

Merging coverage numbers from the xml reports is tricky: You can't
simply add the numbers if the reports refer to the same source code. For
example if the same line has been hit in both reports it's still a
single covered line.

I wouldn't try to solve this in the plugin. JaCoCo perfectly merges
execution data from multiple test runs and can create reports on it. We
only need to find a good solution for multi module reports in Maven.

Here is a description the JaCoCo coverage counters:
http://www.eclemma.org/jacoco/trunk/doc/counters.html

Cheers,
-marc

[Ognjen Bubalo]

Hello Marc,

We discussed about this with Jonathan and I would like to ask you for your opinion.

So if we have multiple exec files, the JaCoCo could create a combined exec (or read all of them in the memory). Is it better for us to have everything in the memory and use JaCoCo API to retrieve the data we need, or still to create an XML file and parse it?

Also I would use the JaCoCo API to create the highlighted sourcefile HTML pages. What do you think about that?

Thank you!

Best Regards,

Ogi

[Marc Hoffmann]

Hi Ogi,

combining exec files is a simple operation that can be easily performed
with JaCoCo APIs.

But I don't think this is the critical point here. The interesting part
is report creation (html, xml, whatever). For this you need beside the
exec file(s) all classes and source files in the correct structure. This
leads us back to the question whether analysis/reporting is part of the
build or part of the Jenkins plugin.

> Also I would use the JaCoCo API to create the highlighted sourcefile
> HTML pages. What do you think about that?

Currently this is all internal implementation, the public API can only
be used to create a complete report (from exec, class and source files).
Of course we can discuss to extract some public APIs. In addition we can
think about a parser which creates IBundleCoverage instances from XML
reports. With this instances you could directly drive the JaCoCo
reporting APIs.

Cheers,
-marc

[Ognjen Bubalo]

Hello Marc,

Thank you for the response. First I would like to do the line highlighting in source files feature. I am wondering what would be the best to do. How does JaCoCo do this? I mean is uses the exec for creating the html site right?

Thank you,

Ogi

[Marc R. Hoffmann]

Hi Ogi,

the input to JaCoCo for report generation is

1. exec file(s)
2. class files
3. corresponding source files (optional).

You might take a look at this API usage example:

http://www.eclemma.org/jacoco/trunk/doc/examples/java/ReportGenerator.java

Internally the report generator uses ISourceFileCoverage instances for
source highlighting.

In addition the API and the Ant task allows to specify any hierarchical
structure of (class files, source files) tuples to create an structured
report like this one:

http://www.eclemma.org/jacoco/trunk/coverage/index.html

Cheers,
-marc

On 25.06.12 01:29, Ognjen Bubalo wrote:
> Hello Marc,
>

[Ognjen Bubalo]

Hello Hynek,

Could you please check your Jenkins instance for a JaCoCo update? Jonathan made the release (thank you). Now the results should be valid.

Could you give a feedback please? Thanks.

Best Regards,

Ogi

[Rob Johnson]

Hi Ogi,

I was experiencing the same problem that Hynek was until today's release. The coverage summary for a particular package seems to be ok, but the overall coverage summary is bugged; in the project that I am working on the coverage percentages are all 0 and the totals are far below what they should be:

Keep up the good work!

Rob

[Ognjen Bubalo]

Hi Rob,

I reproduced the problem. I am fixing it.

Thank you for your response!

Ogi

[Ognjen Bubalo]

Hi Rob,

I released a new plugin version (1.0.2), it should be visible in the next 12 hours. Please check if the results are valid.

Thank you!

Br, Ogi

[Mickael Istria]

On 07/01/2012 06:39 PM, Ognjen Bubalo wrote:

I released a new plugin version (1.0.2), it should be visible in the next 12 hours. Please check if the results are valid.

Does it contain a fix for that issue: https://issues.jenkins-ci.org/browse/JENKINS-14159 ?

[Rob Johnson]

Hi Ogi,

Still doesn't seem right I'm afraid. In the overall summary it seems to have chosen just one of the listed packages and shown the stats for that. I am afraid that I can't upload a picture showing you this now from my work PC, but I can do so from home later tonight. If it helps, the Jenkins job I am running is a multi-module build.

Thanks,

Rob

[Ognjen Bubalo]

Hi Rob,

Yes, the plugin does not support yet a multi-module report. Please describe me how does your project look like? You have modules, and each module creates a jacoco.xml?

Thank you,

Ogi

[Mickael Istria]

Hi Ogi,

I find the discussions on the mailing-list are a bit difficult to follow. You are using the mailing-list for issue tracking and related discussions, whereas IMHO, it's not the most convenient for people following this mailing-list, and it's not the most efficient for tracking.
As a follower of this mailing-list, I'd rather read here announce of new releases with release notes, and more functional discussion about new features or usage issues.
I suggest you should use the Jira as much as possible since it will provide you better tracking, better separation of tasks, and more contextual discussions. https://issues.jenkins-ci.org/browse/JENKINS/component/16541 . Also, you should suggest to people who report defects on this mailing-list to open a Jira ticket for further discussion. Your users community and you will be happier that way ;)

Cheers,

[Ognjen Bubalo]

Hi Mickael,

Thank you for your advice! You are definitely right.

I have all the current issues registered on JIRA, I'll redirect the reporters there.

Br,
Ogi

[Rob Johnson]

Hi Ogi, sorry for the delay. He's the picture of the project.

The structure for the project is a standard Maven multi-module project; there is an aggregator POM file that is at the same level as the projects that it manages. i.e:

-[root folder]
---[project A folder]
---[project B folder]
---[project C folder]
---[project D folder]
---pom.xml