> Can anyone give some advice about which one to choose between gmock > and amop? Thanks.
As an author of Google Mock, my opinion is biased. Therefore I encourage other users to answer this.
Having said that, here's my take:
First, I think amop is a fine framework. The ABI-based auto-creation of mocks is really handy. If you find it adequate for your testing problems, go for it. My impression though is that it's not as complete, polished, and well tested (as in both "unit test" and "field test") as Google Mock, so you may find it limiting when you start to use it in bigger projects. Also I disagree with some design choices made there. Some examples:
I like Google Mock's syntax for setting expectations better. In particular, the syntax
EXPECT_CALL(mock, Foo(5, Gt(10)));
resembles the actual call syntax. A reader can easily tell that Foo() is expected to be called with 5 and a number > 10. Being a macro, EXPECT_CALL also allows the error messages to contain source file paths and line numbers, which IMHO is very useful.
You have to write the name of the interface every time and there's more boiler plate in general. The syntax becomes even more overwhelming when the method is overloaded:
I think this is the consequence of exploring the ABI to automatically create the mock. Since you only need to define a mock class once for each interface, and will typically use it many times, I like Google Mock's trade-off better.
Another feature of Google Mock that I haven't seen elsewhere is the ability to easily extend it. If you ever find the built-in matchers and actions to be not enough, you can easily define new ones using the newly-implemented MATCHER* and ACTION* macros.
I plan to do a more complete comparison between Google Mock and other mocking frameworks one day, but it will take time...
Zhanyong has beat me to the punch with his answer so I will just add my observations as comments to his. Don't forget, his bias warning applies to my comments as well.
On Fri, Feb 20, 2009 at 10:55 AM, Zhanyong Wan (λx.x x) <w...@google.com>wrote:
> > Can anyone give some advice about which one to choose between gmock > > and amop? Thanks.
> As an author of Google Mock, my opinion is biased. Therefore I > encourage other users to answer this.
> Having said that, here's my take:
> First, I think amop is a fine framework. The ABI-based auto-creation > of mocks is really handy. If you find it adequate for your testing > problems, go for it. My impression though is that it's not as > complete, polished, and well tested (as in both "unit test" and "field > test") as Google Mock, so you may find it limiting when you start to > use it in bigger projects. Also I disagree with some design choices > made there. Some examples:
> I like Google Mock's syntax for setting expectations better. In > particular, the syntax
> EXPECT_CALL(mock, Foo(5, Gt(10)));
> resembles the actual call syntax. A reader can easily tell that Foo() > is expected to be called with 5 and a number > 10. Being a macro, > EXPECT_CALL also allows the error messages to contain source file > paths and line numbers, which IMHO is very useful.
> You have to write the name of the interface every time and there's > more boiler plate in general. The syntax becomes even more > overwhelming when the method is overloaded:
> I think this is the consequence of exploring the ABI to automatically > create the mock. Since you only need to define a mock class once for > each interface, and will typically use it many times, I like Google > Mock's trade-off better.
One advantage AMOP is mock auto-creation. To mock an interface in Google Mock, one has to create a mock class and override each abstract method and each non-abstract virtual method that the user need to intercept. For example, in the example above the user will have to write:
class MyInterfaceMock : public MyInterface { protected: MOCK_METHOD2(Foo, void(int, int)); // any other methods...
};
Granted, it only has to be done once per interface, but in AMOP, you don't have to do any of that.
Another note is on the richness of expectation conditions one may want to write. As written, AMOP does not support relational matchers like Gt. It has a notion of policy which allows to perform more general equality comparisons. Currently only two policies are implemented: one to compare a dereferenced pointer and another to compare a C-style array. Apart from that, you can only compare against a value. While, strictly speaking, matchers like Gt can be implemented via policies, those implementations (as currently designed) will be counter-intuitive to write and read.
> Another feature of Google Mock that I haven't seen elsewhere is the > ability to easily extend it. If you ever find the built-in matchers > and actions to be not enough, you can easily define new ones using the > newly-implemented MATCHER* and ACTION* macros.
Another thing worth noticing is that Google Mock has been written to work only with Google Test. We have recently added support for other test frameworks via exceptions, but this is not optimal. In the future we plan to provide more seamless support for other frameworks, but to use Google Mock with them now you still have to jump through some hoops.
Overall, I agree with Zhanyong - the distinctive features of gmock are its intuitive syntax and easy extensibility. Oh, and another thing worth mentioning: Google Mock has better wiki documentation. This may be important if you are just starting. Although AMOP's authors seem to be quite responsive on AMOP's mailing list.
> I plan to do a more complete comparison between Google Mock and other > mocking frameworks one day, but it will take time...
>> Can anyone give some advice about which one to choose between gmock >> and amop? Thanks.
> As an author of Google Mock, my opinion is biased. Therefore I > encourage other users to answer this.
> Having said that, here's my take:
> First, I think amop is a fine framework. The ABI-based auto-creation > of mocks is really handy. If you find it adequate for your testing > problems, go for it. My impression though is that it's not as > complete, polished, and well tested (as in both "unit test" and "field > test") as Google Mock, so you may find it limiting when you start to > use it in bigger projects. Also I disagree with some design choices > made there. Some examples:
> I like Google Mock's syntax for setting expectations better. In > particular, the syntax
> EXPECT_CALL(mock, Foo(5, Gt(10)));
> resembles the actual call syntax. A reader can easily tell that Foo() > is expected to be called with 5 and a number > 10. Being a macro, > EXPECT_CALL also allows the error messages to contain source file > paths and line numbers, which IMHO is very useful.
> You have to write the name of the interface every time and there's > more boiler plate in general. The syntax becomes even more > overwhelming when the method is overloaded:
> I think this is the consequence of exploring the ABI to automatically > create the mock. Since you only need to define a mock class once for > each interface, and will typically use it many times, I like Google > Mock's trade-off better.
Because of AMOP is based on ABI of C++, there are some limitations:
* Only supports __cdecl calling convention., which is the default calling convention of almost all compliers * Only supports Pure Virtual Classes ( No member variables, No non virtual member funtions ) * Only supports Single Inheritance Interface. * The maximum number of virtual functions of your interface should be lower than MAX_NUM_VIRTUAL_FUNCTIONS. (The default is 50, you can changed it in amop/Config.h, but it will slow down the compiler)
Also, my understanding is that amop (and many other mocking frameworks) has some ad hoc restrictions on the type of the method. For example, you may not be able to mock a method that returns a type that has no public default constructor, or a method that returns a reference to an uncopyable type.
Google Mock was designed to be a general solution, and has none of these limitations.
One can argue that it's a good thing to only mock pure interfaces, but I'm more pragmatic. Often you need to work with legacy code that you cannot freely change, or sometimes it makes sense to have data members in a class/interface you want to mock. Or your class may start as pure abstract, but at one point need to contain some concrete implementation. I don't like a mocking framework to impose such decisions on the user, although we can recommend and encourage good practices.
> Another feature of Google Mock that I haven't seen elsewhere is the > ability to easily extend it. If you ever find the built-in matchers > and actions to be not enough, you can easily define new ones using the > newly-implemented MATCHER* and ACTION* macros.
Another aspect of Google Mock's extensibility is that since it imposes no ad hoc restrictions on the base class or its methods you are mocking, you are free to extend your design without worrying that it may suddenly become GoogleMock-hostile and thus break all your tests using Google Mock. I think that's important for large scale, real world software development.
> I plan to do a more complete comparison between Google Mock and other > mocking frameworks one day, but it will take time...
On Fri, Feb 20, 2009 at 1:43 PM, Vlad Losev <vladlo...@gmail.com> wrote: > Another thing worth noticing is that Google Mock has been written to work > only with Google Test. We have recently added support for other test > frameworks via exceptions, but this is not optimal. In the future we plan to
Minor correction: it's not done yet, but should be ready very soon.
> provide more seamless support for other frameworks, but to use Google Mock > with them now you still have to jump through some hoops.
Note that amop always uses exceptions to report errors, so the integration story is the same here.
This leads to another important design choice Google Mock made: it does not use or require exceptions when used with Google Test. I believe relying on exceptions for error reporting is fundamentally flawed, although (sadly) that's what most mocking frameworks do and that's what Google Mock will initially do when we add support for non-GoogleTest testing frameworks.
Why? Because a test that should fail may not fail, as a result of using exceptions as the error reporting mechanism.
The problem with exceptions is that *any* function on the call stack can catch and hide them. This is especially true in the context of mocking, whether the mock function is usually called by production code that you are testing. Since it's production code, the test author has no control over it and cannot trust it. If the production code eats the exception the mock function throws, you'll never know the error happened.
> On Fri, Feb 20, 2009 at 1:43 PM, Vlad Losev <vladlo...@gmail.com> wrote:
> > Another thing worth noticing is that Google Mock has been written to work
> > only with Google Test. We have recently added support for other test
> > frameworks via exceptions, but this is not optimal. In the future we plan to
> Minor correction: it's not done yet, but should be ready very soon.
> > provide more seamless support for other frameworks, but to use Google Mock
> > with them now you still have to jump through some hoops.
> Note that amop always uses exceptions to report errors, so the
> integration story is the same here.
> This leads to another important design choice Google Mock made: it
> does not use or require exceptions when used with Google Test. I
> believe relying on exceptions for error reporting is fundamentally
> flawed, although (sadly) that's what most mocking frameworks do and
> that's what Google Mock will initially do when we add support for
> non-GoogleTest testing frameworks.
> Why? Because a test that should fail may not fail, as a result of
> using exceptions as the error reporting mechanism.
> The problem with exceptions is that *any* function on the call stack
> can catch and hide them. This is especially true in the context of
> mocking, whether the mock function is usually called by production
> code that you are testing. Since it's production code, the test
> author has no control over it and cannot trust it. If the production
> code eats the exception the mock function throws, you'll never know
> the error happened.