Gradual document improvements

[Bas Vodde]

Hiya all,

I've been having some complaints about the CppUTest documentation (or the lack of documentation). Because of that, I'm gradually trying to improve that! People who would like to help are more than welcome.

Originally (and now still) the manual is at www.cpputest.org and then the short introduction was at cpputest.github.com/cpputest. The latter page was a Github page of the cpputest project. This is a bit hidden as it is on the gh_pages branch of the cpputest project (which feels like a hack from the github people).

It is also possible to have a "organisation-wide" github pages, so I moved the pages to there. Now they have their own repository at:
https://github.com/cpputest/cpputest.github.io

Also, I've changed the html file to markdown, so that it will be easier to edit. So, the same quick intro page as before is at:
http://cpputest.github.com

I've taken the first page from the previous manual and started converting that to Markdown also at:
http://cpputest.github.io/manual.html

Also, I've committed a first doxygen file and put the first output at:
http://cpputest.github.io/docs/

None of this is actually finished or looking good yet :) I wanted to share it with you guys so you know what is happening and it leaves the possibility of contributing!

It'll still take a while to get it in really good shape :)

Thanks,

Bas

[Martin Ertsaas]

Hi.

I've been thinking for a while about the possibility for testing if some
input makes the program die, say because of an assert.

It is an easy macro to write, however I'm not sure if it is suitable,
and feel I need input from you guys if it's a good idea or not.

My pros and cons is:

+ If you for some reason want some input to force the program/a library
to crash, you can actually test that it does that. This means you can
get feedback on erroneous input earlier (Hopefully before production).
+ Tests are documentation. If you write a library that crashes on
certain input, you can document that in your tests.

- The program itself should never crash, so it feels kind of like
testing only to increase reported coverage.
- Testing for crashes feel wrong somehow.

I know gtest has functionality for this, but I'm a bit on the fence to
if this really makes sense. I'm not going to do anything on it
before I'm convinced that it's a good idea, so input from you guys on
this would be great.

- Martin

[Bas Vodde]

Hi Martin,

Yeah, google test has "death tests" and I consider them a very bad idea. Like you, I've been thinking about this one for quite a while till I was over the fence and very firm in the "bad idea" camp :)

There were a couple of things that pushed me in the "bad idea" camp:
- Having tests that should crash sounds like a bad idea…
- Crashing seems to be just like any 3rd party behavior (interesting thought, but very important)
- These tests are fragile and slow and not very portable.

For checking assertions, it is better to record the assertions and use a test plugin. Someone submitted an assert checking library earlier and we've not yet integrated that.

For crashing. I've test-droven some crashing behaviour in the CppUTest memory allocation framework. To copy some code from there:

static void crashMethod()
{
cpputestHasCrashed = true;
}

void setup()
{
UtestShell::setCrashMethod(crashMethod);
cpputestHasCrashed = false;
}

TEST(MemoryLeakWarningGlobalDetectorTest, crashOnLeakWithOperatorNewArray)
{
crash_on_allocation_number(1);
char* memory = new char;
CHECK(cpputestHasCrashed);
delete memory;
}

I've simply stubbed out the crashing part ;P

The only LOC that is really untested is:

static void defaultCrashMethod()
{
UtestShell* ptr = (UtestShell*) 0x0; ptr->countTests();
}

I figured, you always want to be able to stub out the crashing part and thus then death tests are always useless. It is just poor test design.

Hope this pushes you off that fence :P

Bas

> --
> You received this message because you are subscribed to the Google Groups "cpputest" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cpputest+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Andreas Stenius]

Excellent work!

Well done, Bas :)

2013/4/24 Bas Vodde <ba...@odd-e.com>

[Martin Ertsaas]

This technique was quite nice. And tastes less stale than a EXPECT_DIES
call.
The setCrashMethod call was something I din't know about, so when we
have that I can ignore such a macro. I totally agree that
stuff like that smells bad, and should be avoided.

Thanks.

- Martin

On 04/24/13 10:24, Bas Vodde wrote:
> Hi Martin,
>
> Yeah, google test has "death tests" and I consider them a very bad idea. Like you, I've been thinking about this one for quite a while till I was over the fence and very firm in the "bad idea" camp :)
>
> There were a couple of things that pushed me in the "bad idea" camp:

> - Having tests that should crash sounds like a bad idea�

[Bas Vodde]

Hi,

Yeah, the when you *want* your program to crash, you just call a method that crashes your program :) There shouldn't be any situation in which you would want another method to crash, I hope :) (unless you use the design by contract assertions).

That method, you can then stub out. The setCrashMethod just switches a function pointer on a method that causes the crash.

Bas

On 24 Apr, 2013, at 4:56 PM, Martin Ertsaas <mart...@gmail.com> wrote:

> This technique was quite nice. And tastes less stale than a EXPECT_DIES
> call.
> The setCrashMethod call was something I din't know about, so when we
> have that I can ignore such a macro. I totally agree that
> stuff like that smells bad, and should be avoided.
>
> Thanks.
>
> - Martin
>
> On 04/24/13 10:24, Bas Vodde wrote:
>> Hi Martin,
>>
>> Yeah, google test has "death tests" and I consider them a very bad idea. Like you, I've been thinking about this one for quite a while till I was over the fence and very firm in the "bad idea" camp :)
>>
>> There were a couple of things that pushed me in the "bad idea" camp:

>> - Having tests that should crash sounds like a bad idea…