ON_CALL or EXPECT_CALL?

[Zhanyong Wan (λx.x x)]

Hi, Google Mock users,

ON_CALL is likely the single most under-utilized construct in Google Mock.

There are basically two constructs for defining the behavior of a mock object: ON_CALL and EXPECT_CALL.  The difference?  ON_CALL defines what happens when a mock method is called, but doesn't imply any expectation on the method being called.  EXPECT_CALL not only defines the behavior, but also sets an expectation that the method will be called with the given arguments, for the given number of times (and in the given order when you specify the order too).

Since EXPECT_CALL does more, isn't it better than ON_CALL?  Not really.  Every EXPECT_CALL adds a constraint on the behavior of the code under test.  Having more constraints than necessary is baaad - even worse than not having enough constraints.

This may be counter-intuitive.  How could tests that verify more be worse than tests that verify less?  Isn't verification the whole point of tests?

The answer, lies in what a test should verify.  A good test verifies the contract of the code.  If a test over-specifies, it doesn't leave enough freedom to the implementation.  As a result, changing the implementation without breaking the contract (e.g. refactoring and optimization), which should be perfectly fine to do, can break such tests.  Then you have to spend time fixing them, only to see them broken again the next time the implementation is changed.

Keep in mind that one doesn't have to verify more than one property in one test.  In fact, it's a good style to verify only one thing in one test. If you do that, a bug will likely break only one or two tests instead of dozens (which case would you rather debug?).  If you are also in the habit of giving tests descriptive names that tell what they verify, you can often easily guess what's wrong just from the test log itself.

So use ON_CALL by default, and only use EXPECT_CALL when you actually intend to verify that the call is made.  For example, you may have a bunch of ON_CALLs in your test fixture to set the common mock behavior shared by all tests in the same group, and write (scarcely) different EXPECT_CALLs in different TEST_Fs to verify different aspects of the code's behavior.  Compared with the style where each TEST has many EXPECT_CALLs, this leads to tests that are more resilient to implementational changes (and thus less likely to require maintenance) and makes the intent of the tests more obvious (so they are easier to maintain when you do need to maintain them).

--
Zhanyong

[Martin Svensson]

Hi Zhanyong.

Your reasoning seems sensible, but why should I use ON_CALL() when I can do EXPECT_CALL().Times(AnyNumber)) to also get rid of warnings?

Isn't ON_CALL() just good for not needing to repeat action specifications?

/Martin

[Zhanyong Wan (λx.x x)]

Hi Martin,

Great question!  Glad that you asked it. :)

There is one significant difference between the two constructs:

ON_CALL is orthogonal to EXPECT_CALLs; EXPECT_CALL().Times(AnyNumber())) is not orthogonal to other EXPECT_CALLs.

What do I mean?

Suppose you have a group of related tests that all make use of the same mock object but need to set different expectations on it.  You don't want to repeatedly define what the mock should do when invoked (as it's largely the same for all your tests), so you define its default actions in your test fixture, and put different EXPECT_CALLs in different TEST_Fs:

class MyTest : public testing::Test {

 protected:

  MyTest() {

    ON_CALL(mock_bar_, Foo(_)).WillByDefault(Return(42));

    ...

  }

  MockBar mock_bar_;

};

TEST_F(MyTest, Quarks) {

  EXPECT_CALL(mock_bar_, Foo("a")).Times(2);

  ...

}

The test says that Foo() will be called twice, each time with argument "a".  If Foo() is called with argument "b", for example, it will fail as there's no matching expectation.

Now consider the same test code, except that we write EXPECT_CALL().Times(AnyNumber()) instead of ON_CALL:

class MyTest : public testing::Test {

 protected:

  MyTest() {

    EXPECT_CALL(mock_bar_, Foo(_))

        .Times(AnyNumber())

        .WillRepeatedly(Return(42));

    ...

  }

  MockBar mock_bar_;

};

TEST_F(MyTest, Quarks) {

  EXPECT_CALL(mock_bar_, Foo("a")).Times(2);

  ...

}

The semantics of the test has changed.  Now, if Foo() is called 3 times, with arguments "a", "a", and "b" respectively, the 3rd call will succeed, as it matches the "catch-all" EXPECT_CALL in the test fixture.  This may not be what you want.

So here's the moral:

Use ON_CALL unless you intend to affect the expectation on the method.

What about the annoying warnings about "uninteresting calls", then?  I agree that they are annoying when we don't really care about whether the method is called or not.  That's why I recommend to use nice mocks (https://code.google.com/p/googlemock/wiki/CookBook#Nice_Mocks_and_Strict_Mocks) by default (and only set expectations when we intend to verify the interaction).  That is, instead of

    MockFoo mock_foo_;

write

    NiceMock<MockFoo> mock_foo_;

To optimize for this common case, I plan to change the default behavior of mocks from "naggy" to nice (https://groups.google.com/forum/?fromgroups=#!topic/googlemock/FGLl-zoW5E0), such that you don't have to write NiceMock all the time.  That hasn't happened yet, as many users aren't ready for the change, but it's the direction I intend to drive googlemock to.

--
 
---
You received this message because you are subscribed to the Google Groups "Google C++ Mocking Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to googlemock+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Zhanyong