Eclemma vs Jacoco

[kv.f...@gmail.com]

Hi , I am using latest version of Eclemma in my VM. But our built is on Jacoco. They have 2 different reports. While I was checking the difference of them, I found that Eclemma is using the Jacoco and report should be the same. Why I am having different reports?

[Evgeny Mandrikov]

Hi,

Without knowing which exact difference you observe and which metrics you compare, one can only guess. Among possible reasons:

• Latest EclEmma 3.1 uses JaCoCo 0.8.1, but maybe your build uses another JaCoCo version?
  
• While EclEmma uses JaCoCo, result (coverage report) can be different because JaCoCo performs analysis of bytecode, which can be different between two cases.
  
• In particular in case of different compilers, because they produce different bytecode - when you use EclEmma, I'm pretty sure that you rely on compilation in Eclipse, which uses ECJ (Eclipse Compiler for Java), while when you use JaCoCo you rely on command-line build which most likely uses javac from JDK. As concrete basic example - ECJ usually removes obvious dead code, while javac does not.
  

My personal practical observation - is that latest versions of compilers are more close to each other. Also in latest versions of JaCoCo and hence EclEmma there are filters for artifacts generated by different compilers and their versions, they help to reduce differences in reports. My personal practical observation based on one of closed-source projects - is that filters helped to remove differences between javac versions from 8 to 10. And while we do our best in implementation of these filters for different compilers and their versions (which is quite hard in itself), complete removal of all differences is impossible in general.

If you will provide minimal, yet complete and runnable example, then we might investigate your particular case deeper.

Regards,

Evgeny

[farrukh Bek]

Thanks for the quick respond. So I have a maven project. I am attaching the xml code how I installed it. At the same time , I have updated Eclemma( which probably should get latest version of Jacoco). 

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.8.1</version>

<configuration>

<destFile>${basedir}/target/coverage-reports/jacoco-unit.exec</destFile>

<dataFile>${basedir}/target/coverage-reports/jacoco-unit.exec</dataFile>

</configuration>

<executions>

<execution>

<id>jacoco-initialize</id>

<goals>

<goal>prepare-agent</goal>

</goals>

</execution>

<execution>

<id>jacoco-site</id>

<phase>package</phase>

<goals>

<goal>report</goal>

</goals>

</execution>

</executions>

</plugin>

 Basically I am not expecting identical report, But at least close to each other. Currently I have difference 20 to 25 percent Eclemma and Jacoco. Jacoco is showing less code coverage than Eclemma. Which one will be correct in this case? How can I increase jacocoe coverage and to make it close to Eclemma? 

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/5XsKEdGlqek/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jacoco+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/d02ef460-3d7d-4a8c-b8c1-9b553d93dd88%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Evgeny Mandrikov]

On Wednesday, July 25, 2018 at 10:35:25 PM UTC+2, farrukh Bek wrote:

Thanks for the quick respond. So I have a maven project. I am attaching the xml code how I installed it.

 

Just snippet of pom.xml is definitely not complete and runnable example. Not counting JaCoCo version, is completely useless in absence of source code.

At the same time , I have updated Eclemma( which probably should get latest version of Jacoco). 

 Basically I am not expecting identical report, But at least close to each other.

 

Currently I have difference 20 to 25 percent Eclemma and Jacoco.

Percent of a which metric? instructions? lines? branches?

 

Jacoco is showing less code coverage than Eclemma.

Which one will be correct in this case?

Without concrete and complete example from you, with probability as high as strange this might sound to you - both are correct.

Here is example from one of our tests - https://github.com/jacoco/jacoco/blob/v0.8.1/org.jacoco.core.test/src/org/jacoco/core/test/filter/targets/Finally.java#L38

Line with comment "$line-example.0$" will be shown as empty when compiled with javac and fully covered when compiled with ECJ. And both cases are absolutely correct.

 

How can I increase jacocoe coverage and to make it close to Eclemma? 

 

Once again, without concrete and complete example from you, one can only guess and make random suggestions - maybe simply use same compiler in both cases?

[farrukh Bek]

sorry for the confusion. When I said less, I assumed in all branches. All branches are less in jacoco comparing to Eclemma. Can you clarify what is  complete and runnable example ? Are you meaning the report details? I have a two reports, one is generating in eclipse and anothier one is generating as maven source where I can find the report details. I have compared the report details and I found that the tests executing and showing with green color in Eclemma, is showing with red color and showing as not executed in Jacoco report. 

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/43f0ed31-933a-4870-8d8b-18ec982f50c4%40googlegroups.com.

[Evgeny Mandrikov]

On Thursday, July 26, 2018 at 3:50:38 PM UTC+2, farrukh Bek wrote:

Can you clarify what is  complete and runnable example ? Are you meaning the report details?

 

complete =  all information necessary to reproduce the problem is included

runnable = allows others to reproduce, observe and investigate the same problem that you observe

Source code that can be built and executed on our side.

[farrukh Bek]

here is one of the test which used to run and increase the coverage previously. Now it is showing as red.

public class OlgaWrongNameOntology {

private final ByteArrayOutputStream errContent = new ByteArrayOutputStream();

@Before

public void setUpStreams() throws UnsupportedEncodingException {

PrintStream printStream = new PrintStream(errContent, false, "UTF-8");

System.setErr(printStream);

errContent.reset();

}

@After

public void cleanUpStreams() {

System.setErr(null);

}

@Test

public void testWrongName() {

File resourcesDirectory = new File("src/test/resources/simple/wrongname.owl");

OLGA.main(new String[] { "--version", "01.99.45", "--code", "cs", "--library", "trinity", "--name",

"WrongNameOntology", "--path", resourcesDirectory.getAbsolutePath() });

assertTrue(OLGA.getResult().contains("Fail"));

}

}

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/5XsKEdGlqek/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jacoco+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/6db56200-085d-4027-92f9-43798be9dbbd%40googlegroups.com.

[Evgeny Mandrikov]

Are you kidding???

How anyone except you can guess what is "OLGA"???

How anyone except you can compile and run this???

complete =  all information necessary to reproduce the problem is included

[farrukh Bek]

For that I need to send entire project. Is that what you mean?

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/0544cb58-1fd9-4afa-bdd3-0f75b31bcb7c%40googlegroups.com.

[farrukh Bek]

Kidding will be outside , we are in a professional world. Make clear your idea and I will send you right thing.

[Evgeny Mandrikov]

On Thursday, July 26, 2018 at 5:02:19 PM UTC+2, farrukh Bek wrote:

For that I need to send entire project. Is that what you mean?

It can be entire project or just its part or just a separate example, but it should demonstrate the problem, be complete, compilable and executable not just by you. I have no idea how else explain to you term "reproducer" well-know for software engineers.

[farrukh Bek]

Unfortunately not everybody can share there source code because of company confedentiality. If you expect it, then your approach is wrong here. Anyways, I was thinking about some configuration problems which you listed solution for those. But it didn't work. It might be possible bug for you that you need to consider about it. Is there anything you can add in terms of configuration part other than you have listed above?

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/ca7ca3e6-edcf-4c29-8833-46f684ead0cb%40googlegroups.com.

[Evgeny Mandrikov]

On Thursday, July 26, 2018 at 6:28:41 PM UTC+2, farrukh Bek wrote:

Unfortunately not everybody can share there source code because of company confedentiality. If you expect it,

Let me highlight again what was said before - "it can be separate example", different from your code, but demonstrating the same problem/difficulty. So no one asks/forces you to share confidential information.

 

then your approach is wrong here.

Anyways, I was thinking about some configuration problems which you listed solution for those. But it didn't work. It might be possible bug for you that you need to consider about it.

If you believe that there is bug, then provide reproducer. As you said - "make clear your idea". Unfortunately in absence of details from you and so in presence of too many unknown factors, wield guesses are counterproductive. If you believe that others should be guessing, wasting their time on your confidential problem, then please reconsider your approach.

 

Is there anything you can add in terms of configuration part other than you have listed above?

Given the lack of details from you, IMO there is nothing what can be added.

Regards,

Evgeny

 

[farrukh Bek]

Is there anybody who can understand human language? If not you can close the discussion.

Thanks.

To unsubscribe from this group and all its topics, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/6db56200-085d-4027-92f9-43798be9dbbd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/5XsKEdGlqek/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/0544cb58-1fd9-4afa-bdd3-0f75b31bcb7c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/5XsKEdGlqek/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jacoco+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/ca7ca3e6-edcf-4c29-8833-46f684ead0cb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jacoco/5XsKEdGlqek/unsubscribe.

To unsubscribe from this group and all its topics, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/8d9579ba-8d07-470d-8139-90afb3aa0acd%40googlegroups.com.

[Mirko Friedenhagen]

Hello Farrukh,

sorry to inform you that probably most of this thread’s readers will be on Evgeny’s side, at least I am. 

Jacoco is an open source project and developed as well as supported via this mailing list by mostly two people in their spare time.

He told you several times that your example is not complete enough to reproduce let alone analyze your problem.

You reacted with insults and complaints about that your code is confidential so I assume you get paid to program here. Most of us work for money as well and understand the situation. 

Nonetheless YOU need to do the work of creating a complete Maven based project which you may put on e.g. github which reproduces the problem.

I do not know you personally but only from your postings on this list, but I assume would we have to work together I would misunderstand you often.

Best Regards

Mirko 

You received this message because you are subscribed to the Google Groups "JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jacoco+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jacoco/CAO3aaPpf4ArWQuiku1SfDQrjuYvM_cAdtCtwodm-jd1PaXaEJA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Regards Mirko
--
http://illegalstateexception.blogspot.com/
https://github.com/mfriedenhagen/
https://bitbucket.org/mfriedenhagen/