cobertura-maven-plugin with GWTTestCase classes

[Matt Raible]

I've added the cobertura-maven-plugin[1] to my project and can
successfully generate code coverage reports for my JUnit tests.

However, it doesn't pick up classes that extend from GWTTestCase. Is
it possible to use the cobertura-maven-plugin for these classes and
the gwt-maven plugin?

Thanks,

Matt

[1] http://mojo.codehaus.org/cobertura-maven-plugin/

[Charlie Collins]

In short, nope.

To get code coverage with GWT you need a patch to GWT, and to make it
convenient to filter things you need my patch to the patch. It's a
long story:
http://code.google.com/p/google-web-toolkit/issues/detail?id=779

[Charlie Collins]

I should have also noted there (but you can see it in the issue and
the GWT docs), that even once you patch you don't get the ease of
choosing a coverage tool and the regular Maven way for reports, etc -
you have to use Emma.

I have successfully used Emma for quite some time, with GWT and Maven
via GWT-Maven, but it's not for the faint of heart (patch, patch,
etc).

GWT 1.6 should help with make this a lot easier, but it is my
understanding it will still be Emma only. (Cobertura won't work - has
to do with what code actually gets run during GWTTestCase tests, and
the way the classpath is constructed - even instrumenting classloaders
don't have real access, like I said, long story.)

On Jan 29, 7:00 pm, Matt Raible <mrai...@gmail.com> wrote:

[Matt Raible]

How would I go about integrating this patch with Maven (and the emma-
maven-plugin)? I can get the JAR from
http://code.google.com/p/google-web-toolkit/source/browse/#svn/tools/redist/emma.
Should I just use the ant-run plugin?

Thanks,

Matt

On Jan 30, 11:07 am, Charlie Collins <charlie.coll...@gmail.com>
wrote:

[Charlie Collins]

I haven't used that patch, so I am not sure.

Nevertheless, if you just put that jar (the patched emma) in your
classpath (as your dep) then it should work (and will fix what that
patch addresses, merged reports exception).

To use code coverage at all with GWT 1.5.3 (or less) you need to patch
GWT itself though, which is different than this patch of emma. (See
the previous note with the issue report.)

On Feb 3, 3:38 pm, Matt Raible <mrai...@gmail.com> wrote:
> How would I go about integrating this patch with Maven (and the emma-

> maven-plugin)? I can get the JAR fromhttp://code.google.com/p/google-web-toolkit/source/browse/#svn/tools/....

[Matt Raible]

I will try the maven-emma-plugin next. I just tried to add
gwtcoverage-1.4.61.jar to my build path in Eclipse and run Coverage
using the EclEmma plugin. I get the attached error.

Any ideas?

Thanks,

Matt

--
http://raibledesigns.com

[Matt Raible]

Moving up the gwtcoverage-1.4.61.jar seems to help a bit. However, now
I get the following error:

[ERROR] Failure to load module 'com.mycompany.myapp.Application.JUnit'
java.lang.UnsupportedClassVersionError: Bad version number in .class file
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:675)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374)
at com.google.gwt.dev.shell.mac.BrowserWidgetSaf$ExternalObject.gwtOnLoad(BrowserWidgetSaf.java:76)
at com.google.gwt.dev.shell.mac.BrowserWidgetSaf$GwtOnLoad.invoke(BrowserWidgetSaf.java:155)
at org.eclipse.swt.internal.carbon.OS.ReceiveNextEvent(Native Method)
at org.eclipse.swt.widgets.Display.sleep(Display.java:3801)

I'm using Java 1.5.0_16. Was this JAR compiled with 1.6?

I also tried using the emma-maven-plugin by installing the emma.jar
and gwtcoverage.jar as Maven dependencies and referencing them in my
pom.xml:

<dependency>
<groupId>com.mycompany.gwt</groupId>
<artifactId>gwtcoverage</artifactId>
<version>1.4.61</version>
</dependency>
<dependency>
<groupId>com.mycompany.emma</groupId>
<artifactId>emma</artifactId>
<version>2.0.5312</version>
</dependency>

However, code coverage #s seem to stay the same regardless. Next I'll
try adding them as plugin dependencies rather than as project
dependencies.

Matt

--
http://raibledesigns.com

[Matt Raible]

Looking at the following page, it appears that Emma will only work
with tests run by the surefire-plugin.

http://mojo.codehaus.org/emma-maven-plugin/usage.html

In my experience, tests that extend GWTTestCase can't be run by
surefire, only by the GWT Plugin. Is this true?

--
http://raibledesigns.com

[Charlie Collins]

Emma works with GWT, and with GWTTestCase tests, but it's
complicated. You need the patch I referred to in the first response
here to enable it, and then you have to patch the
"gwt.coverage.enable=true" system property to your JVM to enable it.

Here is an old discussion about it from back in the day, that might
help along with the patch discussion:
http://markmail.org/message/4ixvnzxffohpd6jk.

The GWT team made a patch to allow code coverage to work. I made a
patch to their patch to filter out certain classes. All of this was
in the GWT 1.4.x timeframe. I haven't tried it lately, but I am think
I recall that it also works with 1.5.x (I used it a long time ago, but
again, haven't verified that lately). In GWT 1.6 it looks like they
made ANOTHER patch to Emma itself for a different purpose - so it's
confusing, but there are multiple patches in play depending on what
version of GWT you are talking about and what you are trying to do.

I used gwt.coverage.enable with GWT 1.4.6x quite a lot and it worked
fine. I *think* I also recall using it with 1.5.x and it worked there
too. It WILL NOT WORK to just try to use regular Emma stuff, its'
complicated and messy. If you don't want to get into the mess, just
wait for the next release of GWT, which appears to have this stuff
included.

In GWT 1.6 it appears there is an entirely new approach (I was unaware
of this until I researched it today ;)):
http://code.google.com/p/google-web-toolkit/wiki/EmmaSupport.


In case it helps with 1.5, if you are brave enough to go there - here
is the relevant text discussing it from the 1.4 timeframe (which
should also work with 1.5) from GWT in Practice:
8.2.3 Code coverage
----------------------------------
If you’re a traditional tester, you’re probably quite familiar with
the concept of code
coverage (and you deserve kudos). Coverage is a metric of how much of
your code, line
by line, method by method, is actually exercised by your tests. Using
coverage information,
you can gauge whether your application is adequately tested, and you
can see
which areas need attention.

Obtaining coverage information with GWT is not a walk in the park. You
can obtain
code-coverage data, but it requires a bit of special understanding and
an extension to
the toolkit. (The code that enables coverage support may be included
as part of a
future release, but for now it’s a patch, which can also be included
via an extension
JAR file.)

To qualify that a bit, server-side code, which runs as standard Java,
is no different
from any other Java code when using GWT. Client-side code is
different. Client side
code is executed as Java bytecode in hosted mode, and this is where
things get
tricky in GWT coverage terms.

PROBLEM
We want to obtain code-coverage information for a GWTTestCase-derived
test.

SOLUTION
To get code coverage working with GWT, you have two options: you can
patch GWT
yourself with a code-coverage patch, or you can include an extension
JAR file that
already includes the patch on your classpath. The easier route is to
use the extension
JAR, so that’s what we’ll focus on here. That said, patching the
toolkit on your own is not
that difficult, and having the knowledge to do it may come in handy in
other situations
as well. If you want to build and patch GWT on your own, see the
“Making GWT
Better” guide in the GWT documentation for full instructions (http://
code.google.com/
webtoolkit/makinggwtbetter.html#workingoncode).

The GWT code-coverage extension JAR file that we’ll use is based on a
coverage
patch that the GWT team has provided (codecoverage-r676.patch). For
details on the
patch, see issue 779 in the GWT issue tracker (http://code.google.com/
p/google-webtoolkit/
issues/detail?id=779). We have also further modified the patch to
allow the code-coverage process to filter out
classes that you probably don’t want to instrument — details on the
modification are
also in the aforementioned code-coverage issue.

The code-coverage patch, and our modification to allow for filtering
which classes
do and do not get instrumented is available in the form of a JAR file
that you can add
to your classpath. Including this coverage JAR
(gwtcoverage-1.4.60.jar) in your classpath,
in front of the GWT dev jar for your platform, overrides the necessary
classes and
gives you code-coverage capability.

Once you have the coverage JAR file in your classpath, you also need
to include an
EMMA library JAR file for coverage support. EMMA is a popular open
source codecoverage
tool that GWT’s coverage support uses. You can obtain EMMA from the
EMMA project’s website (http://emma.sourceforge.net/). With these
prerequisites in
place, you simply need to pass the special system property
*gwt.coverage.enable=true*
to the JVM when invoking GWT tests to get coverage data output.

With coverage support enabled, GWT will output EMMA coverage data
files to the
current working directory. Those files can then be processed with the
reporting facilities
of EMMA to generate a code-coverage report. Figure 8.4 shows the top-
level page
of such a report, obtained when running the tests from the GWTTestMe
project we
have used in this chapter.

DISCUSSION
It’s obviously not ideal to have to build and patch the toolkit just
to get code-coverage
data. This is why we created and provided the coverage support JAR
file, which should
ease the process somewhat. Even though using this JAR is not entirely
painless, and it
would be better if GWT simply included more robust coverage support to
begin with,
at least this solution makes it possible to obtain coverage
information. The good news
is that the GWT team is aware of this shortcoming and plans to include
the coverage
patch, and possibly more testing and coverage-related enhancements, in
a future
release. For now, we think including the coverage JAR to get the
coverage data is worth
the effort.

There are several reasons why GWT’s hosted mode coverage situation is
complicated,
which is what makes all of this necessary in the first place and
negates more traditional
code-coverage techniques. First, the JUnit support in GWT recompiles
code
from source. This means coverage tools that work offline, and
instrument classes, are
instrumenting classes that never get run by GWT. Second, the same
JUnit support has
its own CompilingClassLoader. This means that even on-the-fly
instrumenting classloaders
are knocked out of the picture (without some hacking of the toolkit).
This is
why there is special support in GWT for obtaining code-coverage data.
There are a few main points to take away from this discussion of GWT-
enabled code
coverage. The filtering of which classes are and are not instrumented
is only available
if you use the extension to the GWT patch that we have created. The
standard GWT
patch still works, but it instruments every class that passes through
the GWT compiler,
which means GWT classes, JUnit classes, test classes, and the like all
end up in your coverage
reports, rather than just the code you really care about—your own.
Also, GWT is
doing the instrumentation step for you, because of the
gwt.coverage.enable system
property. This means you do not need to manually run EMMA either on
the command
line or using a build tool as you might normally expect. All you need
to do is run the
tests with the system property set, and coverage data will be output
to the current
working directory.

. . .

On Feb 4, 4:05 pm, Matt Raible <mrai...@gmail.com> wrote:

[Matt Raible]

Thanks for the detailed explanation Charlie. I discovered today that I
don't have to patch or wait to get code coverage working. The secret?

Use Intellij IDEA. It's code coverage works for classes that extend GwtTestCase!

Cheers,

Matt

[Charlie Collins]

Thats good to know, IntelliJ is cool as heck - but of course that
doesn't really solve your issue because code coverage has to be part
of your regular automated build, it can't just be a one of thing in
your IDE ;).

Hopefully GWT 1.6 with straighten this out, and it sounds like from
the contrib thread and the Wiki and such that it will.

On Feb 5, 7:07 pm, Matt Raible <mrai...@gmail.com> wrote:
> Thanks for the detailed explanation Charlie. I discovered today that I
> don't have to patch or wait to get code coverage working. The secret?
>
> Use Intellij IDEA. It's code coverage works for classes that extend GwtTestCase!
>
> Cheers,
>
> Matt
>
> On Thu, Feb 5, 2009 at 7:38 AM, Charlie Collins
>

> ...
>
> read more »