Hindrance to unit testing

[rakesh mailgroups]

Hi guys,

do you guys find creating objects inline (using new) a barrier to unit testing?

Would the code have less less new's if it was developed TDD?? Does that mean its LESS OO??

Opinions welcome!!!

R

[Peter Becker]

Hi Rakesh,

I'm not sure what exactly you mean with creating objects inline, but generally speaking any direct call to a constructor can easily turn your unit test into an integration test. A constructor call is probably the strongest dependency possible.

While I'm all in favour of less OO, that problem doesn't necessarily lead that way. The usual solutions would be either passing the lower layer as parameter into the constructor of the unit to test; or using a factory method/class to create either the runtime or the mock version of the lower layer class. To avoid a mess around singletons you need to pass a factory object around or -- the easy option -- make the factory method part of the unit under test and override it for testing, returning a mock object.

TDD will lead you to any of these solutions. The good thing about it is that any of these solutions hints at using better abstractions -- you can use an interface in either case (passed parameter or factory) instead of having a dependency on the concrete class.

HTH,
  Peter

[Kevin Wong]

If it suits your project, Dependency Injection is a nicer solution
than factories.

On Apr 28, 6:18 pm, "Peter Becker" <peter.becker...@gmail.com> wrote:
> Hi Rakesh,
>
> I'm not sure what exactly you mean with creating objects inline, but
> generally speaking any direct call to a constructor can easily turn your
> unit test into an integration test. A constructor call is probably the
> strongest dependency possible.
>
> While I'm all in favour of less OO, that problem doesn't necessarily lead
> that way. The usual solutions would be either passing the lower layer as
> parameter into the constructor of the unit to test; or using a factory
> method/class to create either the runtime or the mock version of the lower
> layer class. To avoid a mess around singletons you need to pass a factory
> object around or -- the easy option -- make the factory method part of the
> unit under test and override it for testing, returning a mock object.
>
> TDD will lead you to any of these solutions. The good thing about it is that
> any of these solutions hints at using better abstractions -- you can use an
> interface in either case (passed parameter or factory) instead of having a
> dependency on the concrete class.
>
> HTH,
> Peter
>
> On Mon, Apr 28, 2008 at 11:02 PM, rakesh mailgroups <
>

[Jeremy Ross]

This is just one of those areas that you can't make concrete rules
about. Other than built-in really simple things, like Strings, I've
gotten to where I generally don't want an object to new up another,
particularly things that will likely need to be stubbed or mocked,
such as service provider-type objects or concrete domain objects. A
common exception to this is a factory method, where a concrete object
knows how to new up a family of objects in it's factory method. This
type of coupling is by design, even though it can make a testing a
little harder since you may have to stub the concrete factory.

I'd say that using TDD would not result in less new() calls -- they'd
just be in different places. Instead of being buried in business
code, they're in service locators, DI containers, test harnesses,
etc. I don't think this makes code less OO in any way. If anything,
it probably makes it more OO. Whether that's beneficial is debatable,
although I would say it is.

Jeremy




On Apr 28, 8:02 am, "rakesh mailgroups" <rakesh.mailgro...@gmail.com>
wrote:

[Rick]

Don't be afraid of 'new'. It is very fast on modern VMs, faster than
malloc.

If putting 'new' into your code makes it hard to test, then perhaps
your testing is too fine grained.
Try testing larger chunks of functionality all in one go.

Integration is good and not to be put off or avoided. I'd much rather
find that the complete product wasn't working, than pass all the unit
tests (and in many cases I've seen these concerns are orthogonal).

Also, don't settle for unit tests, you should eyeball the software as
well. Have a look at those log files it is spitting out (if it is
headless), have a look at the UI - I recommend daily, or after any big
merge. I've had several projects where people would regularly check
code in which passed the unit tests, but which broke the bigger
picture. Letting them know (gently) that their latest fix was 'teh
suxor' saves a lot of time, because the later they try to address it,
the more they will have forgotten about it.

Mocking out all the different layers of your project is something I'm
not convinced of the value of. Imagine if you were paying someone to
build a car for you, and then found out they spent all their time
playing with matchbox toys.

I'd much rather run tests on real code than imaginary code.

On Apr 28, 11:02 pm, "rakesh mailgroups" <rakesh.mailgro...@gmail.com>
wrote:

[Peter Becker]

While I agree with your general sentiment, your testing of larger chunks of functionality probably doesn't classify as "unit testing" anymore. But in many cases unit testing can be too finegrained -- it really depends on what you are testing. If you have some independent bit of business logic operating on a model I find a unit test against a mock model quite helpful sometimes. Integration tests even against in-memory databases just take too long (I want results in a second or two).

  Peter

[Peter Becker]

I should have used that term. I have a bit of a buzzword-aversion, but passing the instance into the constructor is pretty much the core idea of dependency injection. That's unless you do setter-injection but that screws up even more bits of class invariance -- we don't have much to work with in Java anyway. :-)

And yes: usually it is the better idea to use DI (and maybe even a DI framework), but it requires changes on the client side while the factory method has much less impact (only the class under test is affected).

Factory objects are a potential road to hell -- you get the same question one level up, now wondering where to get the factory instance from.

  Peter

[Jess Holle]

Rick wrote:
> Integration is good and not to be put off or avoided. I'd much rather
> find that the complete product wasn't working, than pass all the unit
> tests (and in many cases I've seen these concerns are orthogonal).
>
> Also, don't settle for unit tests, you should eyeball the software as
> well. Have a look at those log files it is spitting out (if it is
> headless), have a look at the UI - I recommend daily, or after any big
> merge. I've had several projects where people would regularly check
> code in which passed the unit tests, but which broke the bigger
> picture. Letting them know (gently) that their latest fix was 'teh
> suxor' saves a lot of time, because the later they try to address it,
> the more they will have forgotten about it.
>
> Mocking out all the different layers of your project is something I'm
> not convinced of the value of. Imagine if you were paying someone to
> build a car for you, and then found out they spent all their time
> playing with matchbox toys.
>
> I'd much rather run tests on real code than imaginary code.
>

Agreed on all counts!

There seems to be an unreasonable obsession with unit tests these days.
They certainly have their place, but the time, energy, and thought spent
mocking and the effort spent avoiding and delaying testing fully
integrated real software seems absurd in many cases.

After all while unit tests are inherently faster and highly focused,
they're "low fidelity" -- providing no assurance whatsoever of quality
of the overall integrated software. They can be a helpful
quick-and-dirty (and inaccurate) check on one's work throughout the day
and a helpful tool in tracking who-done-it when the real software
breaks, but the thing that really matters is that your overall software
behave as expected, which unit tests can never assure.

--
Jess Holle

[Alexey Zinger]

--- On Tue, 4/29/08, Jess Holle <je...@ptc.com> wrote:

Agreed. I may invite some criticism with what I'm about to say, but in my own experience, a solid API design early on generally allows for good isolation of functionality, such that it can be tested with or without automation and subsequently 99% of bugs reside in higher level decisions and very occasionally in implementation details. It's been very rare in my career that it would have been great to have a unit testing harness ready to go in order to instantly check that modifying implementation details of something down the road is not introducing unexpected behavior. And generally, when that situation arises, it's not so bad to sit down and put together some test cases for the "before" version, make sure it passes, and then apply it to the "after" version. To me, it's completely acceptable to make that part of the debugging/QA process rather than rigidly grown your development process around it from the very beginning of the project.

Alexey
2001 Honda CBR600F4i (CCS)
1992 Kawasaki EX500
http://azinger.blogspot.com
http://bsheet.sourceforge.net
http://wcollage.sourceforge.net

____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

[Kevin Wong]

You were wise to anticipate criticism.

New definition: Legacy Code = Code without tests. After all the dev
community has learned over the last 5-10 years, I find it hard to
believe that the value of automated testing (unit/integration) is
still debated. Here's a case for unit tests:

Maintenance accounts for ~80% of the work involved with a piece of
code; unit testing eases code maintenance.
With good unit tests in place, developers can refactor aggressively,
as the tests ensure that they don't break anything. No longer do we
have to live with ugly code because no one dares touch it.

Unit testing has a beneficial impact on code design. Unit tests are
clients of the code being tested, so it gives the developer immediate
feedback on the usability and flexibility of their design. Unit
testing also encourages loosely-coupled code, because to be easily
unit testable, code cannot be hardwired to other classes. (One common
implication is the avoidance of the Singleton [anti]pattern.)

Coding is faster, as the developer doesn’t need to launch and interact
with an application to test their code.

Unit testing makes the path of implementation more obvious. The
developer first writes a set of tests, then codes until they all pass.

It is common for a developer to think of a test case while coding.
With unit testing the user has a convenient place to record the case
while it's still in his/her head.

Unit testing has an inherent regression aspect. Once a test has been
written for a particular case, it is guaranteed that the case will be
properly handled in perpetuity. In the case of a bug, the best-
practice is to first write a failing test, then code until it passes,
and the bug will never been seen again.

Unit tests can act as the code's specification. This is especially
useful in agile projects which often have no formal written spec.


In regards to integration tests in place of unit tests, why can't you
do both? Develop individual components using unit tests, then use
integration tests to test the whole. (BTW, it's easy to stub out
arbitrarily-grained chunks of your system when integration testing
when using dependency injection.) This, of course, entails some
redundancy, but that's not such a bad thing if quality is a concern.
I've used this strategy with much success.

Poor experiences w/ unit testing might be resultant of a failure to
follow best practices, e.g.:
- Test using public APIs only. This allows implementation refactoring
without the need to change the tests.
- Isolate the target code. (Again, DI helps here.)
- Use a code coverage tool, e.g., Emma
- Use a continuous integration tool, e.g., CruiseControl, Hudson
- Use a mocking tool, e.g., EasyMock, Unitils

> 1992 Kawasaki EX500http://azinger.blogspot.comhttp://bsheet.sourceforge.nethttp://wcollage.sourceforge.net

[Alexey Zinger]

I'm not against unit testing per se. Most of my relatively short career until very recently has been around startups. Time was essential, but so was good architecture. Often times, our technology choices were limited because we were small in size and on the bleeding edge of tech and standards. I'm sure most of the Posse listeners are at least familiar with those kinds of scenarios.

All I can say is that in situations like this, our biggest headaches came not from failed integrations or inability to refactor to our liking, but from difficult architecture problems, unreliably reproducible bugs, or quite simply from overly aggressive demo schedules. Yes, I've been in a position, where unit tests raised alarms. I remember a particular case, where I had joined a project that was under a lot of scheduling pressures and it was quite tough for anyone on the team to allocate sufficient time to properly introduce me into the inner workings of the code base. The system was highly modular with lots of daily changes fixing and breaking stuff. Without unit tests in place, it would have been impossible for me to even know if I'd set up the dev environment correctly.

I'm a firm believer in the right tool for the job/circumstances. All I'm saying is that I've definitely seen circumstances, where we survived just fine without automated fine grained testing, at least at specific stages of development and with developing fine grained testing for earlier written code as part of QA and debugging processes.

Alexey
2001 Honda CBR600F4i (CCS)

1992 Kawasaki EX500
http://azinger.blogspot.com
http://bsheet.sourceforge.net
http://wcollage.sourceforge.net

--- On Tue, 4/29/08, Kevin Wong <kevin.pe...@gmail.com> wrote:

[David Linsin]

Alberto Savoia calls such tests "Characterization Tests" and wrote a
great article about it here:

http://www.artima.com/weblogs/viewpost.jsp?thread=198674

with kind regards,

David Linsin
- - - - - - - - - - - - - - - - - - - - - - - -
email: dli...@gmail.com
blog: http://dlinsin.blogspot.com

[Jeremy Ross]

> Factory objects are a potential road to hell -- you get the same question
> one level up, now wondering where to get the factory instance from.
>
> Peter

I usually DI the factory object. Then it creates concrete objects
which belong to its "family". Not everything has to be dependency
injected, and what does and what doesn't is sometimes hard to grapple
with.

[Rick]

Call me old fashioned, but I find that in general the instance of
Class which represents the class is the best factory object for
instances of that class. :-p

[Rick]

Unit tests have two "killer features"

(1) Refactoring
(2) Test Driven Design

In the early days of refactoring the tools were perhaps a bit hit and
miss, and so even basic refactorings like global renaming of a
variable could potentially break the code in interesting ways.
Arguably you could catch most of the brokenness at the next compile,
but that is time consuming.

So then we have this interesting cost/benefit. Doing refactoring
without extensive unit tests has a time cost. But then creating the
extensive unit tests in the first place also requires a large time
investment.

Presumably there is some kind of tipping point where if you do enough
wholesale refactoring you recoup your time investment.

I think now though that the refactoring tools are getting better, so
the time saved by unit testing drops off. However, since you have to
do some kind of testing anyway that means that the cost is reduced as
well.

For (#2 TDD) this has always struck me as a great trick for overcoming
'coders block'. Coding a big chunk of work might be too overwhelming,
so you just think of a tiny little test that is real easy to
implement, and then another and before you know it, hey presto! Your
mega-code is written. Divide and conquer algorithm strikes again!

As noted here TDD lends itself towards unit testing, since with TDD
you are writing extremely fine grained tests anyway.

Also, some methodologies hold that "The Design" is an emergent
property of the development process, so TDD will drive (sic) you
towards that. And in a situation where you don't have a good designer
you might be better off doing it this way. A good designer however
will allow for changes down the line, and will build in flexibility
into the design. You cannot know the unknown in advance, but you can
make allowances for it in the design, and you can even do a bit of
thinking in advance about what changes are likely to come down the
pipe later on.

But then we run into the "just do barely enough" school of thought.
On the one hand this is appealing, because you are at any one time
only doing a minimum amount of work, so therefore this must be the
path of least resistance, and so therefore the "just do barely enough"
wins on efficiency, right? Because you haven't built in any
unnecessary flexibility.

I don't agree with that view. I suspect there is an implicit cost in
that of the continual redesign/refactoring. I would use the example
of designing a form of transport. You might start saying "okay, the
first step is to go 1kph" so you build a unicycle. That works fine up
till 10kph, when you decide to add a second wheel and drive chain and
turn it into a bicycle (which involves work). That works fine up till
30kph, when you decide to add motor (which involves work), then you
find out you need a CD player and air conditioning, so you turn it
into a car (which involves more work). Which is great until you
discover you need to get from New York to London... oops. Turning a
car into an aeroplane is _surely_ going to be more expensive (and
result in a much worse design) than designing the aeroplane in the
first place.*

* ... unless you are the lads from Top Gear wherein turning a Reliant
into a rocketship is so massively off the coolness scale that it
outweighs mere pedestrian concerns such as practicality.

There are some other benefits to unit testing:

(3) Some methodologies use unit tests to capture requirements.

The problem here is trying to distinguish between requirements
capturing unit tests, and wallpaper/fluff unit tests. In practice, if
you make a change to the code (e.g. to fix bug #00271) and some random
unit breaks, what happens? Most programmers I know would 'fix' the
unit test. Unit tests become obstacles to changing the code.
Sometimes this is a good thing, sometimes this is annoying.

(4) It formalises your testing, which can make other people (e.g.
management) happy.

Keeping management happy leads to pay rises, pay rises lead to bling,
bling leads to bling bling, bling bling leads to pain and suffering...
oh wait, what was my point again?

(5) Unit Tests might help really bad programmers catch certain kinds
of errors.

But this is a chicken and egg problem, if the programmer is bad enough
to benefit from unit tests, how are they to be sure that their unit
tests are working properly?

[Peter Becker]

I guess the fact that Java treats "new" in such a special way distracts from this point of view. It also means that you can have only one constructor per parameter combination, which can be a limit -- in a way that makes avoiding all public constructors in favour of static factory methods a viable option :-)

  Peter

[Peter Becker]

I personally find unit testing useful in some cases, but use integration tests for a lot of other things.

TDD can be great if you design a new API -- it kind of binds the little sentences that your methods represent into a little story, thus checking if the vocabulary leads to a fluid conversation.

Sometimes they can be useful to catch regressions -- but that usefulness is really dependend on the ratio of the complexity of the unit under test and the effort it takes to separate it from the layers below. But I find that if your architecture allows for it a mock model can be easier to test than integration against a test database -- but you have to make sure your model is easy to mock.

The thing I hate is when people claim their unit tests are specifications. I have two major issues with that: first of all specifications should be intensional, all-quantified while tests are extentional, based on some samples. Secondly people do not treat the unit tests like specifications: as you mention the unit tests get "fixed" rather quickly if they break. Things are a bit better with tools like Quickcheck or Scalacheck, but even those should be written based upon specifications, not instead of them.

One thing I notice very often is that people really mean "integration tests" when they say "unit tests". And those do make much more sense, particularly as regression tests. They are the tests you really want to keep alive, while a unit test may not even worth the effort of maintaining it once the unit under test is fully developed. It can be a good tool during development of the harder parts of your code, though -- particularly if you can't integrate yet since the neighbourhood of the unit is not finished.

Just my 2c of course :-)

  Peter

[Kevin Wong]

BTW, if you're using Spring, there's a class listed in this thread
that might be of some use for integration testing:
http://forum.springframework.org/archive/index.php/t-46824.html

On Apr 28, 9:02 am, "rakesh mailgroups" <rakesh.mailgro...@gmail.com>
wrote: