gmock or amop?
ddd
http://code.google.com/p/amop/
Can anyone give some advice about which one to choose between gmock
and amop? Thanks.
Zhanyong Wan (λx.x x)
As an author of Google Mock, my opinion is biased. Therefore I
encourage other users to answer this.
Having said that, here's my take:
First, I think amop is a fine framework. The ABI-based auto-creation
of mocks is really handy. If you find it adequate for your testing
problems, go for it. My impression though is that it's not as
complete, polished, and well tested (as in both "unit test" and "field
test") as Google Mock, so you may find it limiting when you start to
use it in bigger projects. Also I disagree with some design choices
made there. Some examples:
I like Google Mock's syntax for setting expectations better. In
particular, the syntax
EXPECT_CALL(mock, Foo(5, Gt(10)));
resembles the actual call syntax. A reader can easily tell that Foo()
is expected to be called with 5 and a number > 10. Being a macro,
EXPECT_CALL also allows the error messages to contain source file
paths and line numbers, which IMHO is very useful.
In amop you'd write:
mock.Method(&MyInterface::Foo).Expect<0>(5).Expect<1>(Gt(10));
You have to write the name of the interface every time and there's
more boiler plate in general. The syntax becomes even more
overwhelming when the method is overloaded:
mock.Method((void (*)(int,
int))&MyInterface::Foo).Expect<0>(5).Expect<1>(Gt(10));
I think this is the consequence of exploring the ABI to automatically
create the mock. Since you only need to define a mock class once for
each interface, and will typically use it many times, I like Google
Mock's trade-off better.
Another feature of Google Mock that I haven't seen elsewhere is the
ability to easily extend it. If you ever find the built-in matchers
and actions to be not enough, you can easily define new ones using the
newly-implemented MATCHER* and ACTION* macros.
I plan to do a more complete comparison between Google Mock and other
mocking frameworks one day, but it will take time...
Hth,
--
Zhanyong
Vlad Losev
One advantage AMOP is mock auto-creation. To mock an interface in Google Mock, one has to create a mock class and override each abstract method and each non-abstract virtual method that the user need to intercept. For example, in the example above the user will have to write:
class MyInterfaceMock : public MyInterface {
protected:
MOCK_METHOD2(Foo, void(int, int));
// any other methods...
};
Granted, it only has to be done once per interface, but in AMOP, you don't have to do any of that.
Another note is on the richness of expectation conditions one may want to write. As written, AMOP does not support relational matchers like Gt. It has a notion of policy which allows to perform more general equality comparisons. Currently only two policies are implemented: one to compare a dereferenced pointer and another to compare a C-style array. Apart from that, you can only compare against a value. While, strictly speaking, matchers like Gt can be implemented via policies, those implementations (as currently designed) will be counter-intuitive to write and read.
Another feature of Google Mock that I haven't seen elsewhere is the
ability to easily extend it. If you ever find the built-in matchers
and actions to be not enough, you can easily define new ones using the
newly-implemented MATCHER* and ACTION* macros.
Another thing worth noticing is that Google Mock has been written to work only with Google Test. We have recently added support for other test frameworks via exceptions, but this is not optimal. In the future we plan to provide more seamless support for other frameworks, but to use Google Mock with them now you still have to jump through some hoops.
Overall, I agree with Zhanyong - the distinctive features of gmock are its intuitive syntax and easy extensibility. Oh, and another thing worth mentioning: Google Mock has better wiki documentation. This may be important if you are just starting. Although AMOP's authors seem to be quite responsive on AMOP's mailing list.
I plan to do a more complete comparison between Google Mock and other
mocking frameworks one day, but it will take time...
Hth,
--
Zhanyong
Regards,
Vlad
Zhanyong Wan (λx.x x)
Other consequences of the automatic mock creation technique used by
amop (as noted on amop's sourceforge page
http://amop.sourceforge.net/webtest/#limit):
Because of AMOP is based on ABI of C++, there are some limitations:
* Only supports __cdecl calling convention., which is the default
calling convention of almost all compliers
* Only supports Pure Virtual Classes ( No member variables, No non
virtual member funtions )
* Only supports Single Inheritance Interface.
* The maximum number of virtual functions of your interface should be
lower than MAX_NUM_VIRTUAL_FUNCTIONS. (The default is 50, you can
changed it in amop/Config.h, but it will slow down the compiler)
Also, my understanding is that amop (and many other mocking
frameworks) has some ad hoc restrictions on the type of the method.
For example, you may not be able to mock a method that returns a type
that has no public default constructor, or a method that returns a
reference to an uncopyable type.
Google Mock was designed to be a general solution, and has none of
these limitations.
One can argue that it's a good thing to only mock pure interfaces, but
I'm more pragmatic. Often you need to work with legacy code that you
cannot freely change, or sometimes it makes sense to have data members
in a class/interface you want to mock. Or your class may start as
pure abstract, but at one point need to contain some concrete
implementation. I don't like a mocking framework to impose such
decisions on the user, although we can recommend and encourage good
practices.
> Another feature of Google Mock that I haven't seen elsewhere is the
> ability to easily extend it. If you ever find the built-in matchers
> and actions to be not enough, you can easily define new ones using the
> newly-implemented MATCHER* and ACTION* macros.
Another aspect of Google Mock's extensibility is that since it imposes
no ad hoc restrictions on the base class or its methods you are
mocking, you are free to extend your design without worrying that it
may suddenly become GoogleMock-hostile and thus break all your tests
using Google Mock. I think that's important for large scale, real
world software development.
>
> I plan to do a more complete comparison between Google Mock and other
> mocking frameworks one day, but it will take time...
>
> Hth,
> --
> Zhanyong
>
--
Zhanyong
Zhanyong Wan (λx.x x)
> Another thing worth noticing is that Google Mock has been written to work
> only with Google Test. We have recently added support for other test
> frameworks via exceptions, but this is not optimal. In the future we plan to
Minor correction: it's not done yet, but should be ready very soon.
> provide more seamless support for other frameworks, but to use Google Mock
> with them now you still have to jump through some hoops.
Note that amop always uses exceptions to report errors, so the
integration story is the same here.
This leads to another important design choice Google Mock made: it
does not use or require exceptions when used with Google Test. I
believe relying on exceptions for error reporting is fundamentally
flawed, although (sadly) that's what most mocking frameworks do and
that's what Google Mock will initially do when we add support for
non-GoogleTest testing frameworks.
Why? Because a test that should fail may not fail, as a result of
using exceptions as the error reporting mechanism.
The problem with exceptions is that *any* function on the call stack
can catch and hide them. This is especially true in the context of
mocking, whether the mock function is usually called by production
code that you are testing. Since it's production code, the test
author has no control over it and cannot trust it. If the production
code eats the exception the mock function throws, you'll never know
the error happened.
--
Zhanyong
ddd
On 2月21日, 上午8时41分, Zhanyong Wan (λx.x x) <w...@google.com> wrote: