Design by Contract (DBC) and CppUTest

[Miro]

In my embedded work I often use assertions (Design by Contract) and I
wanted to be able to write tests for them. I understand that one
possible position is that consistent use of unit tests mitigates the
need for assertions. I don't think so. To me assertions are
complementary to unit testing, because, unlike the unit tests, they
*stay* in the production code and keep checking its proper execution
all the time. (Yes, I'm a big proponent of keeping assertions in the
production code.)

For example, in my work I often use assertions at the level of an
embedded real-time framework to catch errors in the application code
(which as the framework designer I can't control.) Often also, I've
been using real-time assertions that use embedded hardware to
determine timeliness of code execution. Such things are practically
untestable at the unit level.

Unfortunately, CppUTest does not support testing assertions at this
time. Please note that when an assertion fails the test cannot
continue, because executing the code past assertion failure makes no
sense. But it is not sufficient to place FAIL() in the assertion
handler, because sometimes you exactly want to hit the assertion, so
hitting it means actually success. Conversely, when you expect an
assertion and you don't hit one, the test should fail.

Anyway, I ended up writing a test plugin for assertions. The example
code with the plugin and small test is available for download from:

http://www.state-machine.com/attachments/AssertTest.zip

The test was exercised on Windows in Visual Studio 2008 (sorry, I'm
not Makefile guru). The VS project, which is located in the tests
subdirectory assumes that the CPP_U_TEST environment variable points
to the location of the CppUTest framework.

I think that DbC is important enough so that a plugin similar to my
AssertTestPlugin should be included in the standard CppUTest distro.

I'd love to hear opinions on this matter. I attach a snippet of code
with the tests of assertions based on the AssertTestPlugin:

include "CppUTest/TestHarness.h"

#include "mySqrt.h"
#include "AssertTestPlugin.h"

//------------------------------------------------------------------
TEST_GROUP(mySqrt) {
};
//..................................................................
TEST(mySqrt, SquareOfSqrt) {
//EXPECT_ASSERTION("mySqrt", 0); // uncomment to test unexpected
assertion
double x = 2.0;
double y = mySqrt(x);
DOUBLES_EQUAL(x, y*y, 1E-10);
}
//..................................................................
TEST(mySqrt, NegativeAsserts) {
EXPECT_ASSERTION("mySqrt", 0x11);
mySqrt(-1.0);
}
//==================================================================
void onAssert(char const *file, int line) {
AssertTestPlugin::assert(file, line);
}

[James Grenning]

Hi Miro

It looks like a great addition. I'm at the Agile conference this week. Vacation next. I'll be looking at this more carefully after that.
I look forward to seeing you in Sept.

thanks, James

--------------------------------------------------------------------------------------------
James Grenning Author of TDD for Embedded C
www.renaissancesoftware.net http://pragprog.com/titles/jgade/
www.renaissancesoftware.net/blog
www.twitter.com/jwgrenning
Cell: +1 847-630-0998 Office: +1 847 438 9942

[Terry Yin]

this reminds me a blog written by Bas:

http://blog.odd-e.com/basvodde/2011/01/c-assert-and-unit-tests.html

--

-terry

-------------------------
Blog: http://terryyin.blogbus.com/
twitter: http://twitter.com/terryyin

[Marco]

1) I still like to see some asserts in the "production" code. A lot of people won't look at your unit tests (human nature). This is especially true in layered software where something bad might slip through into a lower layer function. (yes it was probably bad design from the start but it is in production now and the boss doesn't want to change it).

2)  I prefer to use the term 'assertion' for internal tests and 'check' for external tests (since that's the way I learned it  - although the many xUnit tools use the term assert as external). My pet peeve.

[Miro Samek]

Please correct me if I am wrong, but I have a feeling that the TDD
community regards software contracts in the production code as a sort
of cover up for bad design. Frankly, I don't agree with this
assessment.

(It seems to me that the term "assertion" has a different meaning in
TDD than outside this community, so I would propose the term
"contract" to describe the function associated with the standard
assert() macro.)

To me contracts are not only complementary to TDD, but actually it
seems to me almost irresponsible to write production code without
contracts. Again please correct me if I am wrong, but is seems to me
that the central idea of TDD is to test-drive the development
primarily by *unit* tests. Now, unit tests are exactly good at
exercising interfaces, but fall short in ensuring that the interfaces
are used as intended at the *system* level. This is the job of
integration tests, which are typically not part of the TDD micro-
cycle.

And here is when software contracts come in. Preconditions, for
example, are exactly designed to catch incorrect interface use.

I have read the Bas' blog post "C assert and unit tests" (http://
blog.odd-e.com/basvodde/2011/01/c-assert-and-unit-tests.html) and I
don't feel that he explains how TDD makes the C-style assertions
unnecessary. I just don't follow the logic. Do you agree with that
blog post? Why?

[Bas Vodde]

Hi Miro,

Uhm, I think it is a wrong assessment to say that the TDD community regards asserts (contracts) as a cover-up for bad design :)
Let me try to explain my perspective and also some background.

Terminology-wise, I'll still use assertion. An assertion is just a statement that must be true at that point in time. Design by contract is a technique (created by Bertrand Meyer) that looks at the caller-callee relationship as a formal contract (ignoring class invariants for now) and that makes the responsibilities between them very clear. These contracts, in C, can be enforced using assertions (statements that must be true).

I consider DbC extremely important. Not using this during design tend to lead to classes that either do way too much checking as it assumes an empty pre-condition or classes that do not checking at all as it assumes a strong post-condition. So, in designing interfaces, thinking using DbC is very important.

Before I wrote a lot of unit tests (10 years ago), I used a lot of C assertions to document the contracts and to ensure that the software doesn't break it. After I started test-driving my code, I don't add these assertions in the code anymore as I find them less needed. I clarify a lot of the contract by deciding what unit tests to write and these unit test also document the excepted behavior. I usually didn't find a need to still add these asserts and stopped writing them.

Now, you do make a good comment related to integration of features. Though, when I write unit tests, I try to get them to also test between classes, just making sure there are only 2 classes involved. Though, production-code asserts can certainly still be useful for that, though, as mentioned in the blog post, I often find them to clutter the code and rather not add them.

Just to be clear, I'm not saying that adding asserts to your production code is bad and evil and should never be done. Instead, I'm sharing that I personally tend to not do it anymore after moving to full TDD-style development. If other people find it useful, then please *do* keep using it. Also, I don't *never* use them, there are definitively situations where I would use them, though, not as frequent as in the past.

Also, I'll probably integrate the DbC TestPlugin in the CppUTest Extensions in the future (on my todo list) as it does seem to provide useful functionality and does promote good design.

Hope this makes it slightly clearer?

Thanks!

Bas