Jacoco report for Gherkin (Cucumber) integration tests

[Dima Z]

Hello guys!

Is there possibility to get Jacoco reports for Gherkin (Cucumber) tests that are running by using failsafe plugin?

Thanks! 

[Evgeny Mandrikov]

Hi,

JaCoCo is independent from testing framework.

[Dima Z]

Thanks!

The point here is that in Gherkin (cucumber) tests there are special runners.
The goal of these runners run feature files and have some special configuration.

For example:

package mypackage;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty", "html:target/cucumber"})
public class RunCukesTest { 

} 

When I am trying to run jacoco for failsafe plugin that are running these runners I have 0% of coverage.

So I am assuming that 0% in jacoco report its because these runners do not touch source code that is why jacoco can not calculate test coverage for source code. 

Am I correct? 
Or maybe jacoco generate report for coverage in different way?

Thanks! 

[Evgeny Mandrikov]

Once again: JaCoCo is independent from test runners and can collect coverage of Java class files even during manual execution.

The best and fastest way to get help on your issue - is to provide small example, so that to allow us to execute/play with it. The general reason for a such examples - is to allow people that are familiar with JaCoCo, but not familiar with "Gherkin (cucumber)" or some other framework, to help you without wasting too much time on studying framework, setup of "hello world" using it. To see the value of this (how fast and deep possible to go having good example) you can take a look on following bug report containing sample project - https://github.com/jacoco/jacoco/issues/394

Thank you for your understanding.

[Dima Z]

Totally agree with you Evgeny!

Its a very good way to investigate an issue to have hello-world example.

I've created such example and you can fork it from here [ https://github.com/dimazelinskyi/gherkin-and-jacoco-integration ].

I am really appreciate any helps from your side.

To run  the example just call: 

mvn clean install

Thanks for your time!

[Dima Z]

One more thing is that: you should check two reports:

gherkin-and-jacoco-integration/target/site/jacoco-it/index.html

and 

gherkin-and-jacoco-integration/target/site/jacoco/index.html

You will see that in jacoco-it coverege 0%.

Thanks!

[Dima Z]

Evgeny and one more thing :)

I should a bit improve my example and I will let you know asap when I finish it.

Thanks!

[Dima Z]

Evgeny, the example is ready for now.

I provided the basic idea of the issue what I faced.

Will wait for any updates from your side.

Thanks!

[Evgeny Mandrikov]

Your main code is executed in JVM that is started by cargo-maven2-plugin, but JVM argument with JaCoCo agent that is configured by "jacoco:prepare-agent-integration" is not passed to this JVM (see cargo.jvmargs at https://codehaus-cargo.github.io/cargo/Configuration+properties.html). And so you measure coverage in JVM that is started by maven-failsafe-plugin and executes only tests, but no main code. Also order is incorrect - "jacoco:prepare-agent-integration" should be before "cargo-maven2-plugin:start".

All in all - pull request with changes: https://github.com/dimazelinskyi/gherkin-and-jacoco-integration/pull/2

That gives:

Hope this helps.

[Dima Z]

Yes I see now the issue.

Thanks for your help Evgeny it was very needed :)