Using GMock in Arrange-Act-Assert mode

[Alain Désilets]

It seems that the default way of using GMock is to do Record and Playback testing, where we record the calls we expect our mock to receive, and GMock automatically verifies that those expectations are met.

But I tend to prefer a "lenient" Arrange-Act-Assert approach, where I setup the mock to simply record the calls that are made to it, and then at specific points in the test (usually at the end), I check what calls were made. For some reason, I find this more natural, and it has the advantage that, besides checking that such and such calls were made, I can make sure that they were made at specific points in the test.

Is there a way to do this? I know that there is NiceMock, which makes it possible to use a mock without having to record every call to every function that will be made. But there does not seem to be a way to check a posteriori whether such and such calls were made.

Thx.

Alain Desilets
www.alpacatechnologies.com

[Zhanyong Wan (λx.x x)]

Alain,

On Thu, Dec 19, 2013 at 12:43 PM, Alain Désilets <alainde...@gmail.com> wrote:

It seems that the default way of using GMock is to do Record and Playback testing, where we record the calls we expect our mock to receive, and GMock automatically verifies that those expectations are met.

I think that is a misunderstanding.  Did you get that impression from gmock documentation or somewhere else?  If the former, we should fix it.

gMock does not encourage EasyMock-style record-and-playback.  That leads to brittle tests that are coupled too tightly with the implementation.  Therefore, in gMock if a method is called and there's no EXPECT_CALL for it, it's not an error.  One should only set an EXPECT_CALL when he means to ensure that the call occurs, and one should pick his argument matchers and .Times() clauses judiciously to avoid over specifying the constraints.  See https://code.google.com/p/googlemock/wiki/CookBook#Knowing_When_to_Expect for more details.

But I tend to prefer a "lenient" Arrange-Act-Assert approach, where I setup the mock to simply record the calls that are made to it, and then at specific points in the test (usually at the end), I check what calls were made. For some reason, I find this more natural,

OTOH, this means invalid calls will not be caught right away.  When the error was reported later, its context is already gone, making it harder to debug the error.

With gMock, you set the expectations, and as soon as gMock detects an invalid call, it reports it.  If you run the test under a debugger and with the --gtest_break_on_failure flag (https://code.google.com/p/googletest/wiki/AdvancedGuide#Turning_Assertion_Failures_into_Break-Points), the debugger will catch the error and allow you to inspect the stack frames, etc, which can be very useful. 

 

and it has the advantage that, besides checking that such and such calls were made, I can make sure that they were made at specific points in the test.

Not sure I understand this.  gMock lets you specify ordering constraints if you need to.  What exactly are you trying to verify that cannot be expressed in gMock? 

Is there a way to do this? I know that there is NiceMock, which makes it possible to use a mock without having to record every call to every function that will be made. But there does not seem to be a way to check a posteriori whether such and such calls were made.

Thx.

Alain Desilets
www.alpacatechnologies.com

--
 
---
You received this message because you are subscribed to the Google Groups "Google C++ Mocking Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to googlemock+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/googlemock/f0ca02b6-57ed-4312-8bf0-a204f76c4587%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Zhanyong

[Zhanyong Wan (λx.x x)]

(adding back the list s.t. the discussion can benefit others)

On Thu, Dec 19, 2013 at 5:28 PM, Alain Désilets <alainde...@gmail.com> wrote:

On Thu, Dec 19, 2013 at 5:54 PM, Zhanyong Wan (λx.x x) <w...@google.com> wrote:

Alain,

On Thu, Dec 19, 2013 at 12:43 PM, Alain Désilets <alainde...@gmail.com> wrote:

It seems that the default way of using GMock is to do Record and Playback testing, where we record the calls we expect our mock to receive, and GMock automatically verifies that those expectations are met.

I think that is a misunderstanding.  Did you get that impression from gmock documentation or somewhere else?  If the former, we should fix it.

Yes, I got that impression from the documentation. It seems most of the examples I saw in there had EXPECT_CALL()s at the beginning of the test. So I assumed that with GMock you had to define all expected events upfront.

It is true that in gMock one has to define the expectations upfront.  However, that's not the same as record-and-playback, where the expectations are an exact trace of the actual events, which is bad for the reason I stated in my previous message.  We encourage gMock users to write expectations that match the test's intent (no more, no less), not the exact behavior the code-under-test happens to exhibit at the moment.  For example, if you don't care about an argument, use a _ matcher; if you don't care about whether a method is actually called, don't say anything about it; if it's not a bug to call a method more than once, say .Times(AtLeast(1)).

 

 

With gMock, you set the expectations, and as soon as gMock detects an invalid call, it reports it.  If you run the test under a debugger and with the --gtest_break_on_failure flag (https://code.google.com/p/googletest/wiki/AdvancedGuide#Turning_Assertion_Failures_into_Break-Points), the debugger will catch the error and allow you to inspect the stack frames, etc, which can be very useful. 

 

and it has the advantage that, besides checking that such and such calls were made, I can make sure that they were made at specific points in the test.

Not sure I understand this.  gMock lets you specify ordering constraints if you need to.  What exactly are you trying to verify that cannot be expressed in gMock? 

Basically, I am trying to test a class called WebMonitor. This class pings a particular web site, and if the site is down, it emails a notice to some people. I want to replace the EmailSender  by a mock implementation, so that I can tell that the appropriate messages are being sent at the appropriate times. In other words, I would like to write a test that looks like this:

---
TEST_F(WebMonitorTests, checkSite__SiteIsDown__ShouldSendEmailNotification)
{
    WebSiteMonitor *monitor = new WebSiteMonitor();
    monitor->toBeNotified("joe...@somewhere.com");

    monitor->emailSender = new EmailSenderMock();

    // Set the mock web agent so it reports that the site is down
    monitor->webAgent = WebAgentMock();
    ON_CALL(monitor->webAgent, pingURL(string url))
        .WillByDefault(Return(false));

    // Gard assertion: Check that no message has been sent yet
    assertLastEmailSent(monitor->emailSender, void, void);

    string url = "http://www.somesite.com/";
    monitor.checkSite(url);
    assertLastEmailSent(monitor->emailSender, "joe...@somewhere.com", "Site "+url+" is down")
}
---

In this example, assertLastEmailSent() would peek inside the mock EmailSender to see what was the last call made to sendMail(), and what the argument vaues. Notice how we are doing this check before and after the call to checkSite(), which tells us for sure that the call to sendMail() was done as a result of the checkSite() call. If I was to encode the expectations using EXPECT_CALL, then all I would know is that at some point, sendMail() was called with the expected arguments. But I wouldn't know for sure that the call was made as a result of me calling checkSite().

There are several ways you can do this in gMock.  One is to do the expect-verify cycle is several stages, separated by Mock::VerifyAndClearExpectations().  Another is to use checkpoints (https://code.google.com/p/googlemock/wiki/CookBook#Using_Check_Points).

Another advantage of this approach is that it defines the expected "events" at the points in the test where they are expected to be fulfilled.

The VerifyAndClearExpectations() approach lets you do this too.

 

I find this makes the test more readable compared to a style where you define all the expected events at the very beginning of the test.

Alain

--
Zhanyong

[Zhanyong Wan (λx.x x)]

(Please keep the discussion on the mailing list to benefit others.)

On Fri, Dec 20, 2013 at 10:51 AM, Alain Désilets <alainde...@gmail.com> wrote:

On Fri, Dec 20, 2013 at 1:24 PM, Zhanyong Wan (λx.x x) <w...@google.com> wrote:

(adding back the list s.t. the discussion can benefit others)

On Thu, Dec 19, 2013 at 5:28 PM, Alain Désilets <alainde...@gmail.com> wrote:

On Thu, Dec 19, 2013 at 5:54 PM, Zhanyong Wan (λx.x x) <w...@google.com> wrote:

Alain,

On Thu, Dec 19, 2013 at 12:43 PM, Alain Désilets <alainde...@gmail.com> wrote:

It seems that the default way of using GMock is to do Record and Playback testing, where we record the calls we expect our mock to receive, and GMock automatically verifies that those expectations are met.

I think that is a misunderstanding.  Did you get that impression from gmock documentation or somewhere else?  If the former, we should fix it.

Yes, I got that impression from the documentation. It seems most of the examples I saw in there had EXPECT_CALL()s at the beginning of the test. So I assumed that with GMock you had to define all expected events upfront.

It is true that in gMock one has to define the expectations upfront.  However, that's not the same as record-and-playback, where the expectations are an exact trace of the actual events, which is bad for the reason I stated in my previous message.  We encourage gMock users to write expectations that match the test's intent (no more, no less), not the exact behavior the code-under-test happens to exhibit at the moment.  For example, if you don't care about an argument, use a _ matcher; if you don't care about whether a method is actually called, don't say anything about it; if it's not a bug to call a method more than once, say .Times(AtLeast(1)).

OK. I was going by the definition of record-and-playback versus arrange-act-assert that is given in Roy Osherove's "The Art of Unit Testing". He describes  the difference as whether you define expectations before or after exercising the mock.

FYI, I was using the "record-and-playback" term as in EasyMock vs jMock: http://jmock.org/easymock-comparison.html

Whether the expectations are "strict" or not is a different concept for him (it's called leniant versus strict mock).

Anyways, the rest of your message tells me that it's possible to do arrange-act-assert in GMock using VerifyAndClearExpectations(). Thx for the clarification.

Alain

--
Zhanyong