[Boost-users] [test] boost::test equivalent of CppUnit's "protectors" ?
Tim Day
I've used both CppUnit and boost::test for C++ unittesting. Generally I
prefer boost::test, mainly because the auto-test macros minimise the
effort to setup tests. But there's one thing I really miss from CppUnit:
the ability to register your own custom "protectors", instances of which
automatically wrap all the run tests. (Technically, you install a test
"listener", and that can wrap each test run in a protector scope.
Multiple listeners and nested protectors are possible).
I've found these invaluable in the past for monitoring unittests for
unexpected side effects (e.g checking code hasn't changed the
floating-point unit state flags, or messed with other global state). I
can't see any equivalent in the boost::test documentation, although
BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a
CppUnit protector can return a bool to indicate pass/fail, it's less
clear to me how a fixture destructor should indicate failure).
Any suggestions for how to best achieve the same thing as CppUnit's
protectors in boost::test (and without updating the body of each test) ?
Thanks for any advice
Tim
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Gennadiy Rozental
>
>
> I've used both CppUnit and boost::test for C++ unittesting. Generally I
> prefer boost::test, mainly because the auto-test macros minimise the
> effort to setup tests. But there's one thing I really miss from CppUnit:
> the ability to register your own custom "protectors", instances of which
> automatically wrap all the run tests. (Technically, you install a test
> "listener", and that can wrap each test run in a protector scope.
> Multiple listeners and nested protectors are possible).
You can register test observers using framework::register_observer.
See the header test_observer.hpp and framework.hpp for interface definition.
> I've found these invaluable in the past for monitoring unittests for
> unexpected side effects (e.g checking code hasn't changed the
> floating-point unit state flags, or messed with other global state). I
> can't see any equivalent in the boost::test documentation, although
These are more on advanced usage side and are not coverred in current docs.
> BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a
> CppUnit protector can return a bool to indicate pass/fail, it's less
> clear to me how a fixture destructor should indicate failure).
What and where you want to indicate? How framework has to respond to this
indication?
Gennadiy
Tim Day
> Tim Day <timday <at> bottlenose.demon.co.uk> writes:
> > But there's one thing I really miss from CppUnit:
> > the ability to register your own custom "protectors", instances of which
> > automatically wrap all the run tests.
>
> See the header test_observer.hpp and framework.hpp for interface
> coverred in current docs.
Thanks for the pointer; those look exactly like what I was looking for.
> > BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a
> > CppUnit protector can return a bool to indicate pass/fail, it's less
> > clear to me how a fixture destructor should indicate failure).
>
> What and where you want to indicate? How framework has to respond to this
> indication?
I was just concerned that if I went down the route of trying to indicate
test(-suite) failure from BOOST_FIXTURE_TEST_CASE's destructor, pretty
much the only option would seem to be be to throw (not a nice thing to
do in a destructor) (hmmm... actually it occurs to me I have no idea how
things like BOOST_CHECK indicate to the framework that a test has
failed; it clearly isn't by throwing as, unlike BOOST_REQUIRE, execution
of the test continues. I really should look at the source more
closely). But failing a test within a test_observer::test_finish() call
seems like it ought to be a cleaner place to do it.
Tim
Tim Day
> But failing a test within a test_observer::test_finish() call seems
> like it ought to be a cleaner place to do it.
Hmmm having played with it a bit, I'm at a bit of a dead-end with regard
to how or even whether it's possible for a test_observer to express the
opinion that the test should actually be considered failed. It's easy
enough for the observer to log out its opinion as text, but attempting
to throw or use a BOOST_CHECK or similar (e.g direct call to
framework::assertion_result or test_unit_aborted) within
test_observer::test_finish() generally results in something like
"Boost.Test framework internal error: unknown reason." or an access
violation.
Any suggestions ?
(Previous experience suggests tests can spew logs and warnings all they
like, but it takes an actual failure to really get attention).
Gennadiy Rozental
> > > BOOST_FIXTURE_TEST_CASE maybe comes closest. (However, whereas a
> > > CppUnit protector can return a bool to indicate pass/fail, it's less
> > > clear to me how a fixture destructor should indicate failure).
> >
> > What and where you want to indicate? How framework has to respond to this
> > indication?
>
> I was just concerned that if I went down the route of trying to indicate
> test(-suite) failure from BOOST_FIXTURE_TEST_CASE's destructor, pretty
> much the only option would seem to be be to throw (not a nice thing to
> do in a destructor)
Yes. I think that you can do this. If you test stack unrolling using
uncaught_exception().
> closely). But failing a test within a test_observer::test_finish() call
> seems like it ought to be a cleaner place to do it.
It was not designed for this purpose. Maybe we can reconsider it.
Gennadiy
Gennadiy Rozental
>
> On Mon, 2009-05-11 at 11:50 +0100, Tim Day wrote:
> > But failing a test within a test_observer::test_finish() call seems
> > like it ought to be a cleaner place to do it.
>
> Hmmm having played with it a bit, I'm at a bit of a dead-end with regard
> to how or even whether it's possible for a test_observer to express the
> opinion that the test should actually be considered failed. It's easy
I can consider making this possible. For now try throwing exception from fixture.
In general you should try to make your test cases self consistent, so that you
would not need to test for side-effects outside of the test case.
Tim Day
> Tim Day <timday <at> bottlenose.demon.co.uk> writes:
> > Hmmm having played with it a bit, I'm at a bit of a dead-end with regard
> > to how or even whether it's possible for a test_observer to express the
> > opinion that the test should actually be considered failed. It's easy
>
> I can consider making this possible. For now try throwing exception from fixture.
>
> In general you should try to make your test cases self consistent, so that you
> would not need to test for side-effects outside of the test case.
Yes I'm now coming round more to thinking that if a test passed by it's
own rules, no external observer really has any business saying "no,
actually you failed". It probably does make more sense for such
observers to log their opinion "well you might think you passed, but I
really don't like the fact that you changed <whatever state observer is
monitoring>" somewhere sufficiently visible.
Tim