Unit testing

[Si]

Hey all, what are your thoughts on testing frameworks? Coming from
PHPUnit I started with JUnit4. Having read through the cookbook and
played with it a little now I'm, er, a bit underwhelmed. It doesn't
appear to support mock objects and data providers for example (or I'm
blind).

Google revealed TestNG which seems far more feature rich. Is it worth
spending time looking at this instead? What do you guys use and does
it integrate well with Hudson?

Whilst on the subject of Hudson, I'm not too keen on paying $300 for
a single Clover license. Any pointers on alternatives that preferably
integrate with Hudson too?

Cheers,
Si

[Owain Lewis]

Hey Simon,

We're currently using the following setup:

Continuous Build Automation:
-Hudson

Testing:
-Junit + EasyMock (or jmockit)
-Selenium
-WANEM

Coverage:
-Cobertura
http://cobertura.sourceforge.net/

Owain.

--- On Tue, 8/12/09, Si <swes...@gmail.com> wrote:

[Ed Bowler]

I can heartily recomend testng ... we migrated to it from junit 4 one
afternoon. It supports data providers, running tests multiple times
in different thread pools, and loads of other goodness. For mock
objects, use the mockito library. Mock objects are a fair bit more
complicated in java than php, but mockito's api is pretty simple (and
a lot easier to use than easymock or jmock - the other two I've played
with ;), expressive and works with the static nature of java. You'll
also want to look at the hamcrest api which all of these frameworks
use. Hamcrest is a set of matchers for your code, so you can say
things like:

assertThat(myList, hasItem(equalTo(myElement))));

which should introduce you to one of the java 5 features I use the
most, static imports.

links:

hamcrest - http://code.google.com/p/hamcrest/
mockito - http://mockito.org/
testng - http://testng.org/
static imports -
http://java.sun.com/j2se/1.5.0/docs/guide/language/static-import.html
(I know it says use sparingly, but I don't think they realised how
bloody useful they were going to be ;)

All these (and more) available through maven, just add some
dependancies to your pom, and you're away. Maven is the easiest way
to integrate these things with something like hudson, or teamcity. We
also use cobertura, but if you're relying on a coverage tool, you
probably need to go back to programmer school and learn tdd. Maven
has plugins to generate all kinds of reports based on your code,
including a website that houses it all for you. This is why all the
java project websites look identical ;)

In mockito, you create mock objects with:

List<Object> myCollection = mock(List.class);

and make it behave with:

when(myCollection.get(anyInt())).thenReturn(object);

and verify behaviour with:

verify(myCollection).get(anyInt());

which will complain if the get method hasn't been called.

There are more advanced apis for things like executing code when
things happen, that tend to use anonymous inner classes for
readability, and simplicity, and because java hasn't got closures yet
;)

is this enough information, or should I shut up ?

Ed

[James Jefferies]

We use junit4 mainly at the mo, with easymock.

My advice when it comes to metrics and code coverage, if you want any
kind of reporting would be to use Sonar, http://sonar.codehaus.org/
which is the bees knees.

Once you have sonar running, you can then add a little bit of maven
magic and it'll do the rest. Here is an example of sonar in action:
http://nemo.sonarsource.org/

btw, hudson is also available these days as an apt-get.. which is
nice.

[James Jefferies]

Should add as well that Spock (using groovy) and ScalaTest (using
scala) both looked really good at Devoxx. Not tried either in anger
though...

[Ed Bowler]

On Wed, Dec 9, 2009 at 11:31 AM, James Jefferies
<jamesje...@gmail.com> wrote:
> My advice when it comes to metrics and code coverage, if you want any
> kind of reporting would be to use Sonar, http://sonar.codehaus.org/
> which is the bees knees.
>
> Once you have sonar running, you can then add a little bit of maven
> magic and it'll do the rest. Here is an example of sonar in action:
> http://nemo.sonarsource.org/

never played with sonar, but it looks pretty comprehensive ... will
need to play more ;)

[Si]

Hey Owain, long time no see and all that :)

Cobertura works like a charm, just dropped in 4 line of config into my
pom and mvn site:site does the rest. I'm really liking the power of
Maven - can't help being a little jealous as a PHP dev by trade, our
tooling still has a long way to go...

Bookmarked WANEM too, I'm sure that'll be useful soon for something
unrelated.

On Dec 9, 10:16 am, Ed Bowler <ed.bow...@gmail.com> wrote:
> We
> also use cobertura, but if you're relying on a coverage tool, you
> probably need to go back to programmer school and learn tdd.

Not exactly sure what you mean by this. Used solely for measuring
quality, agreed. But as a tool for identifying code not covered by
automated tests, it adds a lot of value.

Si

[Ed Bowler]

On Wed, Dec 9, 2009 at 1:56 PM, Si <swes...@gmail.com> wrote:
> On Dec 9, 10:16 am, Ed Bowler <ed.bow...@gmail.com> wrote:
>> We also use cobertura, but if you're relying on a coverage tool, you
>> probably need to go back to programmer school and learn tdd.
>
> Not exactly sure what you mean by this. Used solely for measuring
> quality, agreed. But as a tool for identifying code not covered by
> automated tests, it adds a lot of value.

Yeah ... I've made a flippant comment on the internet again haven't I
... this is probably a bad discussion to have on a mailing list, but
here we go:

if you've written some code, then you know that it's tested, cos
before you wrote any code, you wrote a test, and you're only writing
enough code to make the test pass. So any code coverage tool is just
a crutch that people can use to avoid writing tests first, because
before you check in, you can just run the coverage report, and write
enough tests so that all the lines are covered. This tends to lead to
tests that don't really test functionality, but just run the code and
make sure it doesn't throw exceptions. Also, it often leads to dead
branches of code that aren't needed, because of failure to strictly
follow the test first, one test at a time, tdd like you mean it
practice. And providing a tool in any less than ideally disciplined
environment is just encouraging bad developer behaviour.

However, I'm pretty sure that there's a rather strong argument for
using these tools on a codebase that has low test coverage, to get the
coverage up, but it doesn't remove the questions about what tests to
write, and what the system should actually do, it just tells you what
code you can delete when you've finished ;)

Having said all that, I'm not exactly totally committed to that point
of view ... it all kinda depends on the environment you develop in ...
and the codebase you have to work with, and and and ..... so ignore me
and I'll go back to sleep ;)

by the way, http://www.infoq.com/presentations/Lessons-Learned-from-Java-EE
starts with a little history of the JEE world, so it might be worth a
watch if you wanted to start looking at things like jboss ;)