Is using GWTTestCase really better than a plain-old-test-app?

[John Gunther]

Initially, I assumed that a GWTTestCase based test suite would OF
COURSE be better, but now I'm leaning the other way because:

o The dummy do-nothing test that junitCreator generates takes 20-30
seconds to run on my machine. That's about how long my entire test
application takes to run right now.

o I don't find the reports or the automatic execution of methods that
start with "test" very useful. Any detailed info I need is best
grabbed via the debugger, not via reports, and I can easily write a
simple method that invokes the tests I want to run sequentially.

o My current test program is an ordinary GWT application so it mimics
the environment I'm trying to test in perfectly. GWTTestCase creates a
"hidden browser" in some mysterious unspecified manner that seems very
different ("what does the auto-generated onModuleLoad actually do?")
from what happens in an ordinary GWT application.

This last is the most serious problem I have with GWTTestCase: in a
good test, the test must directly exercise the thing tested in a
clear, unambiguous, manner that can only have one outcome if the
program is correct. If you don't understand what the testing
infrastructure itself is doing, it is impossible to meet this
requirement.

But, a number of GWT projects on Google Code that I admire use
GWTTestCase and I therefore suspect I am missing and/or
misunderstanding something. So I ask you:

Is using GWTTestCase really better than a plain-old-test-app? What
makes it better (or worse)?

John

[dan]

Hi John,

For me, the best reason to write a TestCase is when I want an
automated, repeatable test ensuring that an object or module in my
system is behaving the way I expect. I use some other acceptance tool
like Selenium if I want an automated way of clicking through the
application and verifying that everything is hooked together as I
expect. In the onModuleLoad example you give, I think the tests
aren't intended to start up an entire module EntryPoint, but rather
test the smaller classes that are used by that module.

Like you said, the GWTTestCases themselves are slow; I use them
sparingly in my applications and only test things which really require
browser interaction, such as creating a widget, parsing JSON, or
running native Javascript code. The nice thing about GWT applications
being written in Java is that you can use plain old JUnit TestCases
(which GWTTestCases extend from) if you don't need any browser-related
functionality, and your tests run significantly faster (closer to 20
milliseconds as opposed to 20 seconds). Since they are so much
faster, I try first to make my tests extend TestCase, and change to
GWTTestCase when I know I have some logic in my class under test that
invokes the browser.

I liked this article from Bob Martin which explains more of the
process of using tests; it doesn't speak about GWTTestCases
specifically, but gives some insight into the motivation of writing
automated tests for a project:
http://blog.objectmentor.com/articles/2007/10/17/tdd-with-acceptance-tests-and-unit-tests

Dan

[John Gunther]

Thanks Dan, I read the linked-to article, and your remarks, especially
your point about using GWTTestCase only when TestCase won't work, were
helpful to me.

Suppose I have a conventional GWT test application that contains a
series of assertions--each replacing what would have been a test
within a GWTTestCase. No more 20 second delays, and what do I really
lose--a few reports? But the debugger is my tool of choice when one of
my tests fails, not a JUnit report. I agree with the thesis of the
article that automated test are great, I just don't see why--
considering all the pros and cons--using GWTTestCase to create them is
better than a conventional GWT app with a series of assertions in it.

John

On Nov 9, 11:49 pm, dan <dwell...@cyrusinnovation.com> wrote:
> Hi John,
>
> For me, the best reason to write a TestCase is when I want an
> automated, repeatable test ensuring that an object or module in my
> system is behaving the way I expect. I use some other acceptance tool
> like Selenium if I want an automated way of clicking through the
> application and verifying that everything is hooked together as I
> expect. In the onModuleLoad example you give, I think the tests
> aren't intended to start up an entire module EntryPoint, but rather
> test the smaller classes that are used by that module.
>
> Like you said, the GWTTestCases themselves are slow; I use them
> sparingly in my applications and only test things which really require
> browser interaction, such as creating a widget, parsing JSON, or
> running native Javascript code. The nice thing about GWT applications
> being written in Java is that you can use plain old JUnit TestCases
> (which GWTTestCases extend from) if you don't need any browser-related
> functionality, and your tests run significantly faster (closer to 20
> milliseconds as opposed to 20 seconds). Since they are so much
> faster, I try first to make my tests extend TestCase, and change to
> GWTTestCase when I know I have some logic in my class under test that
> invokes the browser.
>
> I liked this article from Bob Martin which explains more of the
> process of using tests; it doesn't speak about GWTTestCases
> specifically, but gives some insight into the motivation of writing

> automated tests for a project:http://blog.objectmentor.com/articles/2007/10/17/tdd-with-acceptance-...

[dan]

Hi John,

Ultimately it's about having some automated solution which helps
enforce that any change to the code doesn't cause unexpected side
effects. Whether that's a JUnit test suite, a custom test
application, or an automated click-through script (like Selenium), I
think they all serve the same goals.

If you want to give the JUnit style a try, I'd suggest writing some
straight up TestCases, but not GWTTestCases. For example, try testing
some calculation logic from your application, some state changing
code, or even some method which calculates what text to display in
some widget (but doesn't actually instantiate the Widget, because that
would require a GWTTestCase). Your IDE should be able to run all
these very quickly; sub-second or at most a few seconds. If you like
it, great, if not, consider it an experiment that proved that
TestCases weren't for you.

The current project I'm working on uses a mix of plain TestCases, much
fewer GWTTestCases for JSON parsing and a little bit of widget tests,
and a suite of Selenium automated tests. My first GWT application
used only TestCases and Selenium tests, and didn't use GWTTestCases at
all -- so certainly there's no one shoe that fits everything.

Dan

[mP]

GWTTestCases are always going to be slower than regular junit
TestCases because even in hosted mode the TestCase and other classes
must be compiled to javascript, downloaded from to an invisible hosted
mode browser from tomcat etc. Compiling, preparing tomcat and so on is
always going to cost some time. The more classes involved in your
GwtTestCase the longer everything is going to take...L(

[chris.f.jones]

One comment about the debugger stack being better than junit reports.

If you know what the expected values are through a debugger stack,
then you should know enough to write a good assertion. This may seem
like extra work if you're the only developer looking at your work and
its fairly fresh, but if you need to fix a test written by someone
else (or written by you a long time ago) then a good assertion is
worth a lot more.

A well written test will have meaningful assertions where you need
them and the failure message should be descriptive enough so that you
can solve the problem quickly. A bad/good example is:

assertTrue(3 == list.getItemCount());
results in: AssertionFailedError

vs.

assertEquals("My list didn't contain the elements I expected",
expectedMyList, actualMyList);
results in: My list didn't contain the elements I expected. Expected:
<[1, 2, 3]>, but was: <[1, 2]>

Personally, I couldn't handle keeping all those assumptions in my
brain... which is why I encode them properly as assertions and then
cleanse them from my thoughts ;)