Test coverage for Scala tests

[goo...@kathistutz.de]

Hi all,

I'm new to Scala and Scalatest. My team builds a Scala project using Maven, I wrote some tests for it with Scalatest and now I want to have a look at the test coverage of the project. The problem is that I can't get Cobertura or SCoverage to report the test coverage even for really simple example projects.

What I did: I disabled Surefire and added the Maven plugins for Scalatest, Cobertura and SCoverage to the build as well as the reporting section of my pom. In the end, it looks like this:

    <build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <testSourceDirectory>src/test/scala</testSourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                </configuration>
            </plugin>

            <!-- disable surefire -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.7</version>
                <configuration>
                    <skipTests>true</skipTests>
                </configuration>
            </plugin>

            <!-- enable scalatest -->
            <plugin>
                <groupId>org.scalatest</groupId>
                <artifactId>scalatest-maven-plugin</artifactId>
                <version>1.0</version>
                <configuration>
                    <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
                    <junitxml>.</junitxml>
                    <filereports>WDF TestSuite.txt</filereports>
                </configuration>
                <executions>
                    <execution>
                        <id>test</id>
                        <goals>
                            <goal>test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

             <!--enable cobertura-->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <version>2.7</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>cobertura</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.scoverage</groupId>
                <artifactId>scoverage-maven-plugin</artifactId>
                <version>1.1.1</version>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                    <aggregate>true</aggregate>
                </configuration>
            </plugin>

        </plugins>
    </build>
 
   <reporting>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.scalatest</groupId>
                <artifactId>scalatest-maven-plugin</artifactId>
                <version>1.0</version>
            </plugin>

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>cobertura-maven-plugin</artifactId>
                <version>2.7</version>
                <configuration>
                    <formats>
                        <format>xml</format>
                    </formats>
                </configuration>
            </plugin>

              <plugin>
                <groupId>org.scoverage</groupId>
                <artifactId>scoverage-maven-plugin</artifactId>
                <version>1.1.1</version>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                    <aggregate>true</aggregate>
                </configuration>
            </plugin>

        </plugins>
    </reporting>

If I try to use SCoverage with

mvn clean install scoverage:report

I see in the console output that the tests are run twice (which should be the case, as I understood), but in the end, it says

[INFO] --- scoverage-maven-plugin:1.1.1:report (default-cli) @ sandbox ---
[INFO] Skipping SCoverage report generation

and there are indeed no reports to be found.

If I try to use Cobertura with

mvn clean install cobertura:cobertura -Dcobertura.report.format=xml

I see that the tests are run only once, but in the end it states

[INFO] Cobertura Report generation was successful.

and, yes, I find a coverage report under target/site/cobertura/coverage.xml - but it always says that test coverage is 0%.

I know that I'm doing something fundamentally wrong here - but what is it? How do you guys check the coverage of your Scala tests?

I spent quite some time on this problem already and would be really grateful if someone could point me in the right direction.

Thanks a lot!

Kathi

[Grzegorz Słowikowski]

Hi Kathi

1. Remove (comment out) cobertura-maven-plugin from the pom.xml
2. Read
https://github.com/scoverage/scoverage-maven-plugin#adding-scoverage-report-to-site
and select one report out of tree available.
3. Check, what Surefire writes about executed tests (how many executed).

Write, if it helped.

Regards
Grzegorz Slowikowski

> [INFO]SkippingSCoveragereport generation

> |
>
> and there are indeed no reports to be found.
>
> If I try to use Cobertura with
>
> |
> mvn clean install cobertura:cobertura -Dcobertura.report.format=xml
> |
>
> I see that the tests are run only once, but in the end it states
>
> |

> [INFO]CoberturaReportgeneration was successful.

> |
>
> and, yes, I find a coverage report under
> target/site/cobertura/coverage.xml - but it always says that test
> coverage is 0%.
>
> I know that I'm doing something fundamentally wrong here - but what is
> it? How do you guys check the coverage of your Scala tests?
>
> I spent quite some time on this problem already and would be really
> grateful if someone could point me in the right direction.
>
> Thanks a lot!
>
> Kathi
>

> --
> You received this message because you are subscribed to the Google
> Groups "scalatest-users" group.
> To post to this group, send email to scalate...@googlegroups.com
> To unsubscribe from this group, send email to
> scalatest-use...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/scalatest-users?hl=en
> ScalaTest itself, and documentation, is available here:
> http://www.artima.com/scalatest
> ---
> You received this message because you are subscribed to the Google
> Groups "scalatest-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to scalatest-use...@googlegroups.com
> <mailto:scalatest-use...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Grzegorz Słowikowski]

Hi again

You can compare your project with this one:
https://github.com/scoverage/scoverage-maven-samples/tree/scoverage-maven-samples-1.1.1/different-test-frameworks/scalatest

Grzegorz

On 2015-12-18 16:14, goo...@kathistutz.de wrote:

> [INFO]---scoverage-maven-plugin:1.1.1:report (default-cli)@sandbox ---

> [INFO]SkippingSCoveragereport generation

> |
>
> and there are indeed no reports to be found.
>
> If I try to use Cobertura with
>
> |
> mvn clean install cobertura:cobertura -Dcobertura.report.format=xml
> |
>
> I see that the tests are run only once, but in the end it states
>
> |

> [INFO]CoberturaReportgeneration was successful.

> |
>
> and, yes, I find a coverage report under
> target/site/cobertura/coverage.xml - but it always says that test
> coverage is 0%.
>
> I know that I'm doing something fundamentally wrong here - but what is
> it? How do you guys check the coverage of your Scala tests?
>
> I spent quite some time on this problem already and would be really
> grateful if someone could point me in the right direction.
>
> Thanks a lot!
>
> Kathi
>

[Kathi Stutz]

Hi Grzegorz,

thanks a lot!

I actually had already tried all of this at some point in time, but to no avail. But the link to the example project did finally help me, because when I compared the pom there with mine I found that I was using the wrong Maven plugin for Scala. After I changed this dependency, I suddenly got useful coverage reports!

But now I ran into the next problem: Since I have a multi-module project, I would like to aggregate the reports. Unfortunately, it doesn't work. When I put the <aggregate> line in the properties or in the build section of the pom, the build hangs for some time after the line

[INFO] [scoverage] Generating scoverage HTML report...

and afterwards fails with

java.net.ConnectException: Connection refused

When I put the <aggregate> line in the reporting section of the pom, no target directory for the root module is created.

So, do you have any idea where my mistake here is? (I don't have a lot of Maven experience, so it's probably something stupid.)

Again, thanks a lot for your help!

Kathi

[Grzegorz Słowikowski]

Hi Kathi

It looks similarly to
https://github.com/scoverage/scoverage-maven-plugin/issues/27

Grzegorz

On 2016-01-22 16:45, Kathi Stutz wrote:
>
> Hi Grzegorz,
>
> thanks a lot!
>
> I actually had already tried all of this at some point in time, but to
> no avail. But the link to the example project did finally help me,
> because when I compared the pom there with mine I found that I was using
> the wrong Maven plugin for Scala. After I changed this dependency, I
> suddenly got useful coverage reports!
>
> But now I ran into the next problem: Since I have a multi-module
> project, I would like to aggregate the reports. Unfortunately, it
> doesn't work. When I put the <aggregate> line in the properties or in
> the build section of the pom, the build hangs for some time after the line
>
> |

> [INFO][scoverage]Generatingscoverage HTML report...

> |
>
> and afterwards fails with
>
> |
> java.net.ConnectException:Connectionrefused
> |
>
> When I put the <aggregate> line in the reporting section of the pom, no
> target directory for the root module is created.
>
> So, do you have any idea where my mistake here is? (I don't have a lot
> of Maven experience, so it's probably something stupid.)
>
> Again, thanks a lot for your help!
>
> Kathi
>
>
> Am Freitag, 18. Dezember 2015 18:44:29 UTC+1 schrieb Grzegorz Slowikowski:
>
> Hi Kathi
>
> 1. Remove (comment out) cobertura-maven-plugin from the pom.xml
> 2. Read
> https://github.com/scoverage/scoverage-maven-plugin#adding-scoverage-report-to-site
> <https://github.com/scoverage/scoverage-maven-plugin#adding-scoverage-report-to-site>
>
> and select one report out of tree available.
> 3. Check, what Surefire writes about executed tests (how many
> executed).
>
> Write, if it helped.
>
> Regards
> Grzegorz Slowikowski
>