Wishing that withParameterOfType had a save-a-copy version

[Bryce Schober]

I'm cutting my teeth on CppUTest's mocking functionality after a few years of using its more simple functionality, and now I'm banging my head on the behavior of withParameterOfType().

I'm testing a manager object that does some handling on mydata objects that it gets from a producer object and hands off to multiple consumer objects. I've successfully implemented & test mock versions of the producer and consumer objects, and I've implemented the requisite mydata comparator.

But now when I'm trying end-to-end testing of the manager object, I'm having to create multiple objects to use for expectations on the consumption, because of buffering between the production and consumption phases. It seems that it would be pretty useful to have a version of withParameterOfType() that would allocate memory and make a copy of the provided value for later comparison. I could just use identical repetitive data, of course, but that doesn't seem like as good a test, and in some use cases wouldn't allow for full coverage.

Or maybe I'm not seeing the forest for the trees... 

<><  <><  <><

Bryce Schober

[Bas Vodde]

Hi Bryce,

Just to clarify. You mean the withParameterOfType as "expectation" and not as "actual", correct?

The problem you describe feels like a problem in constructing test data. Am I right?

Thanks,

Bas

[Bryce Schober]

Yes, I mean the process of setting up expectations, and specifically a series of different expectations. At the moment, I'm pushing them into a vector and expecting them from there, which keeps them in scope, at least. But it very much feels like the kind of things that belongs inside the framework, at least as a configuration option to the mocking functionality. It does its own allocating for other purposes already...

<><  <><  <><

Bryce Schober

[Bas Vodde]

Oki. Did you ever try a "Test Data Builder" ?

Usually it is a good way of structuring the test data and creating many objects. I'm not convinced yet that storing a copy in the withParameterOfType is the right solution to this problem. :)

You can find more about Test Data Builder at:
http://www.natpryce.com/articles/000714.html

Or on c2 wiki at:
http://c2.com/cgi/wiki?TestDataBuilder

Thanks,

Bas

[Bas Vodde]

Hi Bryce,

Any news on this? Did this help?

Thanks,

Bas

[Bryce Schober]

Hmm... Now that time has gone by...

I feel like the TestDataBuilder pattern is a solution for an orthogonal problem. It would be a good way to build test data in an understandable way, and I'm sure its similarity to the mocking interface is no coincidence.

On the other hand, my problem was with setting up a more detailed sequence of mocking expectations. I *could* build up a cleaner interface (than simply caching in a vector) to represent a sequence of expectations, but that feels more like working around a lack of mocking functionality than building up test data correctly.

Honestly, my initial impressions of mocking led me to believe that this caching of expected data wouldn't even be necessary, because it does *just*work* to expect multiple calls of *different* functions with various parameters before calling into the production code. So I naturally expected it to *just*work* when I wanted to expect multiple calls of the *same* function with different parameter values. ;-)

If I went further down the road I started, over several test suites, I would probably end up factoring out a better-looking interface for setting up a sequence of mock expectations. But the next natural step would be to once again wonder why it couldn't *just*work* in the same mocking interface that already exists.

<><  <><  <><

Bryce Schober

[Bas Vodde]

Hi Bryce,

I'm not 100% sure what couldn't work. Do you still have a wish?

If so, would it be possible to send some code :)

Thanks,

Bas

[Bryce Schober]

It sure would be nice, but honestly, I don't feel qualified to muddle with the internals of the CppUTest code... And it's not exactly clear how one would go about it, as CppUTest doesn't know anything about the sizes of objects. If it were me, I'd be tempted to write a template-style MockExpectedFunctionCall::withParameterOfType() method to allow the caller to do something like:

UserClass* test_data = new UserClassBuilder.withData("foo").build();

mock().expectOneCall("write").onObject(buffer)

.withParameterOfType<UserClass>( "data", test_data );

The idea being that as a template method, you would actually know the UserClass's size and be able to use its comparison operators (which would be required, of course).

Caveats:

1. I'm as much of a C++ newbie as you could probably imagine, so it's quite likely I've typed something above that is quite horrible or even impossible.

2. It seems like CppUTest has chosen not to use templates, even for very simple use-cases. I have no broad experience with how C++ compilers are at handling simple type-templates, so I have no such compunctions.

3. There may be some other elegant & simple way to make this easier, but now that I've written it, it looks *so*much* nicer than the currently required registered-object-comparator stuff.

<><  <><  <><

Bryce Schober

[Bas Vodde]

Hi Bryce,

> It sure would be nice, but honestly, I don't feel qualified to muddle with the internals of the CppUTest code... And it's not exactly clear how one would go about it, as CppUTest doesn't know anything about the sizes of objects. If it were me, I'd be tempted to write a template-style MockExpectedFunctionCall::withParameterOfType() method to allow the caller to do something like:
> UserClass* test_data = new UserClassBuilder.withData("foo").build();
> mock().expectOneCall("write").onObject(buffer)
> .withParameterOfType<UserClass>( "data", test_data );
> The idea being that as a template method, you would actually know the UserClass's size and be able to use its comparison operators (which would be required, of course).

Ok. Though, it isn't much different from withParameterOfTime("UserClass", "data", test_data) and if you already have the comparison operator, then implementing a comparator is about 1 line of code… (plus one more for installing it).

So, though I understand it looks a but nicer, the actual benefit of this (compared to the additional complexity) isn't that high.

> Caveats:
> 1. I'm as much of a C++ newbie as you could probably imagine, so it's quite likely I've typed something above that is quite horrible or even impossible.

Oh, it is so horrible :)

Nah, it is ok and it makes it more clear what you mean…

> 2. It seems like CppUTest has chosen not to use templates, even for very simple use-cases. I have no broad experience with how C++ compilers are at handling simple type-templates, so I have no such compunctions.

Yes, CppUTest is not using templates and that is probably why we won't be implementing the suggestion.

There are a couple of reasons for not using templates (and some other C++ features):
- CppUTest is quite often used for embedded systems and compilers and their C++ support isn't always that great.
- Templates sometimes make things easy… but often complicate things a lot too. For example, in the code example you gave, if you make a mistake you are likely to get a very puzzling confusing compiler error. I sometimes use GMock with CppUTest but the compiler errors that I sometimes get are really mind-blowing.
- When going down the template road, I'm afraid there is no return anymore :P

> 3. There may be some other elegant & simple way to make this easier, but now that I've written it, it looks *so*much* nicer than the currently required registered-object-comparator stuff.

Is it? Did you use the FunctionComparator yet? Then you don't need to use a subclass at all, you just declare one comparing function.

I'm interested to improving the comparator stuff, but not sure where the problem lies in yet. And I'm sure I probably wouldn't want to go the way of the template.

Thanks!

Bas (thinking about how to make the comparators easier to use)

[Bryce Schober]

On Wed, Dec 12, 2012 at 6:56 PM, Bas Vodde <ba...@odd-e.com> wrote:

Did you use the FunctionComparator yet? Then you don't need to use a subclass at all, you just declare one comparing function.

I didn't even know about it. The CppUTest documentation in in the manuals is quite a bit behind the "documentation" in the source...

Looking at it now, my instinct is to say that it still looks like a clunky work-around to the loveliness of *simple* template usage. I have to write a static function to wrap my operator and do some hacky casting that violates my finer sensibilities. And the larger issue is the discoverability of all these details in the absence of robust documentation of the mocking functionality.

If you'd just make some pragmatic and minimal compromises on very basic template usage, this would all be much, much easier, without any special adaptation of the source under test (other than requiring the comparison operator). How long has it been since you actually saw an embedded compiler puking up gobs of object code for a simple template method to use sizeof() and an equality operator? It would be interesting to see what actual data points exist to support this assertion.

You say "When going down the template road, I'm afraid there is no return anymore," but I'm not sure I buy that. It seems that the maintainers can be strict on the non-usage of templates, they can be equally strict on the minimization of template usage.

<><  <><  <><

Bryce Schober 

[Bas Vodde]

Hi Bryce,

> Did you use the FunctionComparator yet? Then you don't need to use a subclass at all, you just declare one comparing function.
>

> I didn't even know about it. The CppUTest documentation in in the manuals is quite a bit behind the "documentation" in the source…

Eh… yes. Our ability to document isn't fantastic. I'm sorry about that. Thinking about just writing a book on it someday and use part of that as documentation.

> Looking at it now, my instinct is to say that it still looks like a clunky work-around to the loveliness of *simple* template usage. I have to write a static function to wrap my operator and do some hacky casting that violates my finer sensibilities. And the larger issue is the discoverability of all these details in the absence of robust documentation of the mocking functionality.

Oki. Uhm, I'm still very hesitant to put a template there. It actually complicates things more than you expect. We wouldn't want the class to be templated. So therefore it would need to be a templated function in the class. Now… because templated functions can't be virtual, it will brin a whole bunch of additional complication. So… not a good idea.

Perhaps a better alternative is to wrap around the "static function to wrap my operator" in a small templated class. It is fairly easy to subclass a templated comparator that uses the operator == and register that one. That would hide the casts and the static function is a small utility class and registering one would be very simple. In fact, we can probably also make a macro for that. You would still need to register the type and still use strings as type, but you wouldn't need to use the do the additional work you mentioned.

> If you'd just make some pragmatic and minimal compromises on very basic template usage, this would all be much, much easier, without any special adaptation of the source under test (other than requiring the comparison operator). How long has it been since you actually saw an embedded compiler puking up gobs of object code for a simple template method to use sizeof() and an equality operator?

I've recently worked with some embedded compilers that didn't support C++ at all :P

> It would be interesting to see what actual data points exist to support this assertion.
>
> You say "When going down the template road, I'm afraid there is no return anymore," but I'm not sure I buy that. It seems that the maintainers can be strict on the non-usage of templates, they can be equally strict on the minimization of template usage.

Well, I've been down the road before and I see it quite frequently in code bases. The seemingly harmless suggestion isn't that harmless at all. As I mentioned, it would need to be a templated function rather than a templated class. That means it can't be virtual anymore. Because the recording is not comparing yet, there it will need to store the type info in a container of unknown types.

This means, if we go down that road we'll be in template meta-programming land before we know it. About 2 weeks ago, I've spend a week unravling a bunch of template metaprogramming code (and boost mpl code) and I'm more sure now than ever that I wouldn't want to do that way.

When you submit a *really simple* implementation, then perhaps. But even then, it might be better to just subclass from the CppUTest classes and add that functionality yourself :)

C++ is a wonderful language that tends to create more complexity than you would want.

Bas

>
> <>< <>< <><
> Bryce Schober

[Bryce Schober]

As I expected, I'm the newbie here, and of course nothing is as simple as it seems it should be, especially in C++. Thanks for explaining your position in more detail.

<><  <><  <><

Bryce Schober