Maven-plugin + Integration test

[Iván Fernández Perea]

Hi there,

I think your tool is quite nice to deal with test reports but I'm facing some problems to make it work with maven 2.2.1 (this version is required), Spring 2.5.6 and integration test with JUnit (failsafe-plugin).

I have added the jacoco-maven-plugin to my pom this way:

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.5.3.201107060350</version>

<configuration>

<destfile>${basedir}/target/coverage-reports/jacoco-unit.exec</destfile>

<datafile>${basedir}/target/coverage-reports/jacoco-unit.exec</datafile>

</configuration>

<executions>

<execution>

<id>jacoco-initialize</id>

<goals>

<goal>prepare-agent</goal>

</goals>

<configuration>

<excludes>*_javassist_*</excludes>

</configuration>

</execution>

<execution>

<id>jacoco-site</id>

<phase>verify</phase>

<goals>

<goal>report</goal>

</goals>

<configuration>

<excludes>*_javassist_*</excludes>

</configuration>

</execution>

</executions>

</plugin>

What I'm doing is next, I use the jetty-maven-plugin to run my integration tests in the integration phase :). Everything works nicely, test are executed properly but I cannot generate jacoco reports from them. 

I have tried several things:

- Exclude from the prepare-agent goal those packages than don't contain any of my integration test, i.e, packages within src/main/java and include those that implement my integration test. Output: no report about integration classes is generated,  but jacoco.exec just contains some information (just the name of the integration classes) about my integration classes. The weird thing about it is that if I inspect the site folder I have information about any class within the src/main/java folder, but none about my integration classes.

pom.xml:

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<version>0.5.3.201107060350</version>

<configuration>

<destfile>${basedir}/target/coverage-reports/jacoco-unit.exec</destfile>

<datafile>${basedir}/target/coverage-reports/jacoco-unit.exec</datafile>

</configuration>

<executions>

<execution>

<id>jacoco-initialize</id>

<goals>

<goal>prepare-agent</goal>

</goals>

<configuration>

<excludes>*_javassist_*:packages_within_src/main/java/.*</excludes>

<includes>integration_test_package.*</includes>

</configuration>

</execution>

<execution>

<id>jacoco-site</id>

<phase>verify</phase>

<goals>

<goal>report</goal>

</goals>

<configuration>

<excludes>*_javassist_*:packages_within_src/main/java/.*</excludes>

<includes>integration_test_package.*</includes>

</configuration>

</execution>

</executions>

</plugin>

- Using a newer plugin version:

  1) With the excludes and include configuration as previously.

<plugin>

<groupId>org.jacoco</groupId>

<artifactId>jacoco-maven-plugin</artifactId>

<!--<version>0.5.3.201107060350</version>-->

  <version>0.5.10.201208310627</version> 

<configuration>

<destfile>${basedir}/target/coverage-reports/jacoco-unit.exec</destfile>

<datafile>${basedir}/target/coverage-reports/jacoco-unit.exec</datafile>

</configuration>

<executions>

<execution>

<id>jacoco-initialize</id>

<goals>

<goal>prepare-agent</goal>

</goals>

<configuration>

<excludes>*_javassist_*:packages_within_src/main/java/.*</excludes>

<includes>integration_test_package.*</includes>

</configuration>

</execution>

<execution>

<id>jacoco-site</id>

<phase>verify</phase>

<goals>

<goal>report</goal>

</goals>

<configuration>

<excludes>*_javassist_*:packages_within_src/main/java/.*</excludes>

<includes>integration_test_package.*</includes>

</configuration>

</execution>

</executions>

</plugin>

I get the following error:

[INFO] Failed to configure plugin parameters for: org.jacoco:jacoco-maven-plugin:0.5.10.201208310627

    (found static expression: '*_javassist_*: packages_within_src/main/java/.* ' which may act as a default value).

Cause: Cannot assign configuration entry 'excludes' to 'interface java.util.List' from '*_javassist_*: packages_within_src/main/java/.* .*', which is of type class java.lang.String

  2) With the includes configuration as previously. I get the same as before but refer to the includes configuration.

  3) Without includes and excludes configuration. I get this type of error in the integration tests: 

$$_javassist_15, method: <clinit> signature: ()V) Illegal local variable number

 4) Using this plugin configuration. I get this: 

Cause: Cannot assign configuration entry 'excludes' to 'interface java.util.List' from '*_javassist_*', which is of type class java.lang.String

After reading some posts I'm not sure if the jacoco maven plugin can be used along with maven 2.2.1 + integration tests. I'm also using failsafe to generate reports and works fine, but I'd like to use jacoco in order to use it with Hudson.

Thank you!!

[Evgeny Mandrikov]

Hi,

First of all - it would be hard to help you without having an access to your project, or at least to minimalistic, but similar example.

Secondly - we try to be really careful with supported versions, i.e. Maven 2.2.1 is perfectly supported.

And finally - here is an example with maven-failsafe-plugin, while will generate reports in directories "target/site/jacoco" (unit tests) and "target/site/jacoco-it" (integration tests) after execution of "mvn clean install" : https://github.com/Godin/sonar-experiments/tree/master/jacoco-examples/maven-failsafe-plugin-example

[Marc R. Hoffmann]

Hi,

the syntax for includes/excludes configuration has changed with 0.5.4.
See changelog for details:

http://www.eclemma.org/jacoco/trunk/doc/changes.html

Best regards,
-marc

> include those that implement my integration test. *Output*: no report

> --

[Iván Fernández Perea]

Hi Evgeny and Marc,

thank you for your answer, it was the  <propertyName>failsafe.argLine</propertyName> what it made the trick :). Anyway I'm just starting to understand how the plugin works and have several doubts.

1. Excludes and includes sections allow you  to analyze or not these classes, but in any case reports are generated from both. Is there any way to not generate reports from classes within the excludes section?.
2. Some of my integrations tests are testing Java interfaces but the plugin doesn't seem to be aware of. Example:

        public class MyTestIT {

        @Autowired.

         private MyInterface interfaceObject;  // this is an interface object.

         @Test

         public void FirstTest() {

              interfaceObject.method();   // this is the method I want to invoke. This method implementation is coded in a concrete class that implements the interface.

         }

}

Thank you!! 

El lunes, 17 de septiembre de 2012 21:32:38 UTC+2, Marc R. Hoffmann escribió:

Hi,

the syntax for includes/excludes configuration has changed with 0.5.4.
See changelog for details:

   http://www.eclemma.org/jacoco/trunk/doc/changes.html

Best regards,
-marc

[Evgeny Mandrikov]

Includes and Excludes should be respected in Report Mojo - this was fixed in version 0.5.4 (see http://www.eclemma.org/jacoco/trunk/doc/changes.html). If not, then please provide minimalistic example project to reproduce problem.

[Marc R. Hoffmann]

Interfaces do not contain executable code. No code -- no code coverage ;-)

Cheers,
-marc

On 18.09.12 12:46, Iv�n Fern�ndez Perea wrote:
>
> Hi Evgeny and Marc,
>
> thank you for your answer, it was the

> *<propertyName>failsafe.argLine</propertyName>* what it made the trick

> :). Anyway I'm just starting to understand how the plugin works and have
> several doubts.
>

> 1. Excludes and includes sections allow you to analyze or not these

> classes, but in any case reports are generated from both. Is there
> any way to not generate reports from classes within the excludes
> section?.

> 2. Some of my integrations tests are testing Java interfaces but the

> plugin doesn't seem to be aware of. Example:
>

> * public class MyTestIT {*
>
> * @Autowired.*
>
> * private MyInterface interfaceObject; // this is an
> interface object.*
>
> *
> *
>
> * @Test*
>
> * public void FirstTest() {*
>
> * interfaceObject.method(); // this is the method

> I want to invoke. This method implementation is coded in a

> concrete class that implements the interface.*
>
> * }*
>
> *}*

>
>
> Thank you!!
>
> El lunes, 17 de septiembre de 2012 21:32:38 UTC+2, Marc R. Hoffmann

> escribi�:

>
> Hi,
>
> the syntax for includes/excludes configuration has changed with 0.5.4.
> See changelog for details:
>
> http://www.eclemma.org/jacoco/trunk/doc/changes.html
> <http://www.eclemma.org/jacoco/trunk/doc/changes.html>
>
> Best regards,
> -marc
>

[Iván Fernández Perea]

Interfaces do not contain executable code. No code -- no code coverage ;-)  . I agree with you. But you can declare a object as an Interface type (although is a concrete class) and invoke a method from it (so that you are executing code).

El martes, 18 de septiembre de 2012 13:24:18 UTC+2, Marc R. Hoffmann escribió:

Interfaces do not contain executable code. No code -- no code coverage ;-)

Cheers,
-marc

> escribi�:

[Marc Hoffmann]

Correct. In this case you will see code coverage on the object
implementing the interface.

-marc

>>> http://www.eclemma.org/jacoco/trunk/doc/changes.html [1]
>>> <http://www.eclemma.org/jacoco/trunk/doc/changes.html [2]>

> --
>
>
>
> Links:
> ------
> [1] http://www.eclemma.org/jacoco/trunk/doc/changes.html
> [2] http://www.eclemma.org/jacoco/trunk/doc/changes.html

[Iván Fernández Perea]

That what was I thought, unfortunately it doesn't :(. I even debug my integration tests to be 100% sure that code from concrete classes was executed, nonetheless no reports of this methods were shown.

Thank you!!.

[Marc Hoffmann]

Hi,

what do you mean by "no reports of this methods were shown"? Do the
classes not show up in the reports at all or are they marked as not
covered?

You can also check the "Sessions" page (link on the upper right corner)
to check whether data has been sampled for these classes.

Best regards,
-marc

>>>>> http://www.eclemma.org/jacoco/trunk/doc/changes.html [1] [1]
>>>>> <http://www.eclemma.org/jacoco/trunk/doc/changes.html [2] [2]>

>>> [1] http://www.eclemma.org/jacoco/trunk/doc/changes.html [3]
>>> [2] http://www.eclemma.org/jacoco/trunk/doc/changes.html [4]

>
> --
>
>
>
> Links:
> ------
> [1] http://www.eclemma.org/jacoco/trunk/doc/changes.html
> [2] http://www.eclemma.org/jacoco/trunk/doc/changes.html

> [3] http://www.eclemma.org/jacoco/trunk/doc/changes.html
> [4] http://www.eclemma.org/jacoco/trunk/doc/changes.html

[Iván Fernández Perea]

Hi Marc,

thank you for your patience. They are marked as not covered. The constructor is covered, but none of the methods within the class are covered.

The Sessions page shows the class is included, but just the class constructor is covered :(.

Thank you!!

[Marc Hoffmann]

What is your execution environment? Is it distributed (client/server),
is it an EJB container?

-marc

[Evgeny Mandrikov]

Hi,

As I said earlier - it might much easy to investigate, if you can provide an example project.

[Iván Fernández Perea]

OK, thank you for your support. I'll write you an easy example of my project  (I'm also testing the cobertura plugin and happens the same).

Regards.

[Iván Fernández Perea]

Hi,

the project is quite large but I'm gonna try to explain how it works giving you some insights.

I'm developing some web services with Apache CXF along with Spring. So I have a maven project with 3 modules (domain, services and web). I have my service layer where I code my bussiness logic (for each one I have its interface and its implementation) . I put these services within the Spring Application context using the @Service annotation. I use the web module to publish the service so that I can see them an tested using Soap UI or my own java client.

@Service("MyService")

public class MyServiceImpl implements MyService {

@Autowired

protected  MyServiceImpl  (MyDAO myDAO) {

super(myDAO);

}

}

To test the services I have a test context where I use the <jaxws:client> to create clients to these services in the test phase and besides of that I run a jetty web container. I think that the key because code within web services are not been inspected is that when I get these clients they are of the type of the interface but also they are spring proxy object. I think that the fact that the object is a proxy avoid instrumentation to work.

Regards,

Iván.

[Marc Hoffmann]

Are you sure you collect execution data for the server part ("I run a
jetty web container")?

If you collect execution data for the test client only you will not see
any code coverage for the server components, right?

Cheers,
-marc

[Iván Fernández Perea]

Yes, I'm executing server code too . I'm calling it from client side :). In fact, as I'm debugging my application I can clearly inspect server side code from the debug console .

Thank you!!

[Evgeny Mandrikov]

There is a difference : 

• start client and server with JaCoCo agent
  
• start client with JaCoCo agent, but server without
  

In both cases client side will call server side, but in second case you will not have instrumentation and so dump for server code ;)

[Iván Fernández Perea]

I see, I'm starting jacoco agent in the pre-integration-test phase and include those packages where server code are. After run the tests I can see no cover has been made for those classes. I have also tried to start jacoco agent in a previous phase but it doesn't work either. How could I be sure that jacoco is aware of the server then?.

thank you!!

[Evgeny Mandrikov]

Once again - provide minimalistic example ( like https://github.com/Godin/sonar-experiments/tree/master/jacoco-examples ) , which mimics your project ! 

[leoy...@gmail.com]

Can someone please show me the correct syntax for includes and excludes for the report goal? I can't seem to able to get it working using com.* or com/* path syntax.

[Evgeny Mandrikov]

Hi,

Regarding "Cannot assign configuration entry":

It should be

<configuration>

  <excludes>

    <exclude>value</exclude>

  </excludes>

  <includes>

    <include>value</include>

  </includes>

</configuration>

instead of

<configuration>

  <excludes>value</excludes>

  <includes>value</includes>

</configuration>