Mocking Private or Protected Methods

[tfmcc...@verizon.net]

I am having trouble setting the default behavior of a derived class to use the protected base class method.

 

class



 WPApp : public WPModule

{

protected

:

 HINSTANCE pinst; // previous instance handle

public

:

 //...

 BOOL first() { return pinst == NULL; }

 //...

protected

:

 virtual void run();

 //...

};

namespace

 ut_WpApp {

 class Mock_WPApp : public WPApp

 {

 public:

 Mock_WPApp()

 {

 //...

 ON_CALL(*this, first())

 .WillByDefault(Invoke(this, &WPApp::first));

 ON_CALL(*this, run())

 .WillByDefault(Invoke(this, &WPApp::run));

 // error: cannot access protected membe }

 //...

 MOCK_METHOD0(first, BOOL ());

 public: // must be public for mock

 MOCK_METHOD0(run, void ());

 };

 }

}

(sigh. the code formatting doesn't seem to have helped here.)

From what I see you can't specify the base class member function as a  default action when it is not public. How would this be accomplished?

[Keith Ray]

Try this to redefine the access level in your subclass. I've used this for redefining the access of inherited member function.

...

class Mock_WPApp : public WPApp
{
public:

using WPApp::pinst;

Mock_WPApp()
{
//...

--
C. Keith Ray
* (650) 533-6535
* http://agilesolutionspace.blogspot.com/
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "Google C++ Mocking Framework" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to googlemock+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/googlemock/8ac0e63d-eea9-4e8b-8d49-f9f5e9a16fce%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[tfmccarthy_verizon_mail]

This doesn't seem to affect it.

I'm digging deeper to gather data....

[tfmccarthy_verizon_mail]

Clearly, others are able to get the using statement to work but I can't see
how.

My analysis indicates that "something is amiss." Some of this is taken from
TCPL 3rd. (§15.2 Pointers to Members).

A programmer should not be able to circumvent the type access model through
a pointer to memeber.

class std_interface
{
protected:
virtual void delay() = 0;
public:
virtual void suspend() = 0;
};
typedef void (std_interface::*pstd_mem)();

void f(std_interface* p)
{
pstd_mem s = &std_interface::suspend;
p->suspend(); // direct call
(p->*s)(); // call through pointer to member

//s = &std_interface::delay; // error: cannot access protected member

// ...or cast it away
//s = static_cast<bool (std_interface::*)()>(&std_interface::delay);
}

Derivation should not alter this

class text : public std_interface
{
protected:
void resume();
public:
void start();
};
typedef void (text::*pd_mem)();

void f(text * p)
{
void (std_interface::*pmi)() = &text::start; // error
void (text::*pmt)() = &std_interface::suspend; // ok
void (text::*pmt2)() = &std_interface::delay; // error

pd_mem sd = &text::start;
p->start(); // direct call
(p->*sd)(); // call through pointer to member

sd = &std_interface::delay; // error: cannot access protected member
sd = &text::resume; // error: cannot access protected member
}

How does a using statement alter this?

Probably the first question to answer is if this premise applies. The
problem statement is

Mock_WPMDIClient::Mock_WPMDIClient()
: WPMDIClient(0)
{
ON_CALL(*this, activeWin())
.WillByDefault(Invoke(this, &WPMDIClient::activeWin));
}

is an attempt to define the base class beahvior as the default behavior of
the mock class. However, WPMDIClient::activeWin is a protected function.

The ussage here is what Invoke(this, &WPMDIClient::activeWin) expands to.

template <class Class, typename MethodPtr>
PolymorphicAction<internal::InvokeMethodAction<Class, MethodPtr> > Invoke(
Class* obj_ptr, MethodPtr method_ptr) {
return MakePolymorphicAction(
internal::InvokeMethodAction<Class, MethodPtr>(obj_ptr, method_ptr));
}

We examine the template InvokeMethodAction and find the smoking gun...

template <class Class, typename MethodPtr>
class InvokeMethodAction {
public:
InvokeMethodAction(Class* obj_ptr, MethodPtr method_ptr)
: obj_ptr_(obj_ptr), method_ptr_(method_ptr) {}
//...

private:
Class* const obj_ptr_;
const MethodPtr method_ptr_; // error
//...
};

This is the ssame error as

void (text::*pmt2)() = &std_interface::delay; // error

My current solution is to define a member function to wrap the base class
function,

WPMDIChild * Mock_WPMDIClient::local_activeWin()
{
return WPMDIClient::activeWin();
}

This can now be used in the default specification in the mock class
constructor,

ON_CALL(*this, activeWin())
.WillByDefault(Invoke(this, &Mock_WPMDIClient::local_activeWin));

- Tim

[tfmccarthy_verizon_mail]

You learn something every day....

After I did the analysis...the simple solution "finally" came to me...

> Probably the first question to answer is if this premise applies. The
> problem statement is
>
> Mock_WPMDIClient::Mock_WPMDIClient()
> : WPMDIClient(0)
> {
> ON_CALL(*this, activeWin())
> .WillByDefault(Invoke(this, &WPMDIClient::activeWin));
> }
>
> is an attempt to define the base class beahvior as the default behavior of
> the mock class. However, WPMDIClient::activeWin is a protected function.

So don't qualify the pointer to member with the base class!

Mock_WPMDIClient::Mock_WPMDIClient()
: WPMDIClient(0)
{
ON_CALL(*this, activeWin())

.WillByDefault(Invoke(this, &Mock_WPMDIClient::activeWin));
}

Much better. The pointer to member function is valid for the derived class.

[Keith Ray]

Good for you. I was going to mention pointer-to-base-class (member is protected) vs pointer-to-derived-class (member is public because of "using" statement) but I didn't have a keyboard handy until now.

As to why "using" has this ability... One reason would be to cope with overloaded methods in a base class hidden by a method in the derived class with the same name (but not the same signature). 

e.g.:  class base { public: void foo(int); void foo(char); };

class derived : public base { public: void foo(float a, float b); };

derived d;

d.foo(2); // should fail to compile because hidden by foo(float,float).

Adding in using base::foo; should allow all the 3 foo methods to be visible to users of derived.

I was a C++ language lawyer until C++0x arrived. The language got a little insane after that. 

Objective-C is a lot nicer, and Swift doesn't look too bad (though Swift's lots-of-optional-syntax is moving toward's C++ complexity a little too much, IMO.)

--
C. Keith Ray
* 650-533-6535
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf
* http://agilesolutionspace.blogspot.com/

[tfmccarthy_verizon_mail]

Well, the bad news is it only compiles but doesn't work.
It's too late here now so I will have to come later.
I thinka full concrete example is in order.

Using directive: needs to provide the correct access.

class base
{
public:
void foo(int);
void foo(char);
};

class derived : public base
{

//using base::foo; // won't work
public:
using base::foo; // works

void foo(float a, float b);
};

void f()
{

derived d;
d.foo(2); // should fail to compile because hidden by foo(float,float).
}

- tim

----- Original Message -----
From: Keith Ray
To: tfmccarthy_verizon_mail
Cc: <googl...@googlegroups.com>
Sent: Tuesday, September 30, 2014 10:56 PM
Subject: Re: [googlemock: 2170] Mocking Private or Protected Methods

[tfmccarthy_verizon_mail]

Here is a full example:

The purpose of the test is to run characterization tests for a base class.
The issue is how to test the protected member function.

The base class is

class std_interface
{
protected:
bool suspend()
{
wcout << __FUNCTION__ << _T(": false\n");
return false;
}
public:
bool start()
{
wcout << __FUNCTION__ << _T(": true\n");
return true;
}
};

Create a mock class for it

class mock_std_interface : public std_interface
{
protected:
public:

MOCK_METHOD0(start, bool ());
MOCK_METHOD0(suspend, bool ());
};

We write the tests for this.

TEST(protected_member_test, std_interface_start)
{
mock_std_interface mstdi;
EXPECT_CALL(mstdi, start());

ASSERT_THAT(mstdi.start(), Eq(true));
}
TEST(protected_member_test, std_interface_suspend)
{
mock_std_interface mstdi;
EXPECT_CALL(mstdi, suspend());

ASSERT_THAT(mstdi.suspend(), Eq(false));
}

The outpout is surprising

1>[----------] 2 tests from protected_member_test
1>[ RUN ] protected_member_test.std_interface_start
1>protected_member_example::std_interface::start: true
1>[ OK ] protected_member_test.std_interface_start (0 ms)
1>[ RUN ] protected_member_test.std_interface_suspend
1>[ OK ] protected_member_test.std_interface_suspend (0 ms)
1>[----------] 2 tests from protected_member_test (0 ms total)

The protected member function (suspend) is not called...but the test passes.
The expectation is met but isn't what we think it is.

We change the mock class to explicitly specify using the base class function
as the default in the default constructor,

class mock_std_interface : public std_interface
{
protected:
public:

mock_std_interface()
{
ON_CALL(*this, start())
.WillByDefault(Invoke(this, &std_interface::start));
ON_CALL(*this, suspend())
.WillByDefault(Invoke(this, &std_interface::suspend)); // error; can't
access protected member
}
//...
};

We try the namespace directive...

class mock_std_interface : public std_interface
{
protected:
public:
using std_interface::suspend;

mock_std_interface()
{
ON_CALL(*this, start())
.WillByDefault(Invoke(this, &std_interface::start));
ON_CALL(*this, suspend())
.WillByDefault(Invoke(this, &std_interface::suspend)); // error; can't
access protected member
}
//...
};

The fully qualified pointer to member cannot specify the protected function.
Since the function has been pulled into the public scope, we use the mock
class in the specification,

class mock_std_interface : public std_interface
{
protected:
public:
using std_interface::suspend;

mock_std_interface()
{
ON_CALL(*this, start())
.WillByDefault(Invoke(this, &std_interface::start));
ON_CALL(*this, suspend())
.WillByDefault(Invoke(this, &mock_std_interface::suspend)); // ok
}

The result, however, is failure

1>[----------] 2 tests from protected_member_test
1>[ RUN ] protected_member_test.std_interface_start
1>protected_member_example::std_interface::start: true
1>[ OK ] protected_member_test.std_interface_start (0 ms)
1>[ RUN ] protected_member_test.std_interface_suspend
1>unknown file: error: SEH exception with code 0xc00000fd thrown in the test
body.
1>[ FAILED ] protected_member_test.std_interface_suspend (48 ms)
1>[----------] 2 tests from protected_member_test (48 ms total)

The SEH exception 0xc00000fd is a stack overflow. We have a recursive call.

We define local functions to wrap any (protected) base class member
functions

class mock_std_interface : public std_interface
{
protected:
public:
using std_interface::suspend;

mock_std_interface()
{
ON_CALL(*this, start())
.WillByDefault(Invoke(this, &std_interface::start));
ON_CALL(*this, suspend())
//.WillByDefault(Invoke(this, &std_interface::suspend)); // error
//.WillByDefault(Invoke(this, &mock_std_interface::suspend)); // ok: but
recursive
.WillByDefault(Invoke(this, &mock_std_interface::local_suspend));
}

MOCK_METHOD0(start, bool ());
MOCK_METHOD0(suspend, bool ());
private:
bool local_suspend() {return std_interface::suspend();}
};

1>[----------] 2 tests from protected_member_test
1>[ RUN ] protected_member_test.std_interface_start
1>protected_member_example::std_interface::start: true
1>[ OK ] protected_member_test.std_interface_start (0 ms)
1>[ RUN ] protected_member_test.std_interface_suspend
1>protected_member_example::std_interface::suspend: false
1>[ OK ] protected_member_test.std_interface_suspend (0 ms)
1>[----------] 2 tests from protected_member_test (0 ms total)

So the functions are actually called and the expectations are met (but see
the first results)

This is less than ideal but works. Makes creating the mock classes a PITA.
And private member functions don't look to be accessible at all.

[Keith Ray]

We're going down a rat-hole now. I suggest removing the whole idea of mocking from your test strategy. In the tests, you are mocking functions, and then calling those mocked functions: you are not testing your code, you are testing the mock code. 

Also, because your mocked functions don't have return values set in EXPECT_CALL, they are probably returning uninitialized memory/random values. See gmock's documentation on "return". 

If you think you need to test private or protected functions, you have a bad design--probably too many responsibilities in those classes. see "single-responsibility principle" and "SOLID". More on that here: 

http://lassekoskela.com/thoughts/24/test-everything-but-not-private-methods/

The purpose of mock objects is to "stub" out dependencies that you don't want tested, when you are testing something else. The mock object can ALSO verify that certain methods are called by the code-under-test. The mock can also verify that specific argument values were passed into the functions you stubbed out. (Emphasis on stubbed.) If the mock is going to call YOUR code, then get rid of the mock, because it isn't needed for your test.

Example... Your code calls a server, and you want to test that your code handles a connection failure correctly. Rather than unplugging the server when you run that test, you "inject" a mock/fake/stub object into the code-under-test, that replaces the server object: you set the stub object to return a connection failure error. You test verifies that YOUR CODE does the right thing: it logs the error, or throws an exception, or whatever. Your test reads the log, or catches the exception, or whatever. 

Other things you may want to stub out: functions that need the current date/time, functions that call servers that you may not have control over, databases that would behave differently if they persisted values from other tests or other users, objects that are hard or expensive to set up. File-system calls that are slow or suffer the same "global state" problems that databases have.

If you are doing TDD (test driven development) in a  "top-down" or "outside-in" style, you might mock out lower-level code that hasn't been written yet. People who TDD this way will replace those mocks with real objects, once those lower-level objects have been coded. They don't keep the mocks around for long, because mocks are fragile and can get in the way of refactoring.

--
C. Keith Ray
* 650-533-6535
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf
* http://agilesolutionspace.blogspot.com/

[Keith Ray]

BTW, if you want to test how a parent class interacts with a child class, use multiple inheritance; have your test fixture inherit from the class-under-test:

class BaseTests : public testing::Test, public YourBaseClass {... If you think multiple inheritance is bad, writing untestable code is worse.

--
C. Keith Ray
* 650-533-6535
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf
* http://agilesolutionspace.blogspot.com/

On Oct 1, 2014, at 8:41 AM, "tfmccarthy_verizon_mail" <tfmcc...@verizon.net> wrote:

[tfmccarthy_verizon_mail]

I'm getting the same feeling. Maybe my approach needs some validation.

 

The code is legacy code. (see the late Paul Dilascia's Windows++ code from 1995). I'm using it to learn/explore TDD with Google Mock after doing Jeff Langr's book (Modern C++ Programming with Test-Driven Development). So this is an exercise in using TDD with legacy code under Google Mock.

 

(Aside: wxWidgets is a similar library to Windows++ and uses CppUnit to test the library. So I wondered if I could do the same with Google Mock.) 

 

The preliminaries were to...

...get the code to compile and link successfully; check

...compile the code as UNICODE to be compatible with Google mock framework: check

 

The (naive) intention is write a set of tests to capture the current code behavior before making changes. These tests are the preliminary tests and didn't expect to spend a lot of time on them; I just want them to exercise the code on a surface level. So it might be best to test just the public interfaces. Some classes can be tested just through the public interface. A number of them rely on inheritance, as might be expected. These are mainly UI related classes (windows, dialogs, menus etc) which, as I understand it, can be particularly difficult to test.

 

So there's a hierarchy that looks like

class WPWin    {}

class WPMainWin : public WPWin {}

class WPDialog : public WPWin {}

.etc.

The dilemma was testing the public api means the classes would create "live" windows, which I wanted to avoid; but that may not be possible. (Aside: the wxWdigets tests do create live windows). I though I would mock the base class dependencies (WPWin) to test the derived class (WPMainWin).

All without having to change the code. I am starting to think this isn't realistic. I think this needs a test double and that fells like a significant amount of work. And some original code change, which I wanted to avoid.

I'm starting to think best way forward is to define an interface class and use test doubles. I have to work out how that would look.

I really appreciate your feedback.

Reading the links now.

Why did I think this wouldn't be too onerous?

 

----- Original Message -----

From: Keith Ray

To: tfmccarthy_verizon_mail

Cc: <googl...@googlegroups.com>

Sent: Wednesday, October 01, 2014 1:31 PM

Subject: Re: [googlemock: 2170] Mocking Private or Protected Methods

[tfmccarthy_verizon_mail]

Now that's a Good Idea (R).

I don't care for multiple inheritance but I really want to know how to do this properly.

 

----- Original Message -----

From: Keith Ray

To: tfmccarthy_verizon_mail

Cc: <googl...@googlegroups.com>

Sent: Wednesday, October 01, 2014 2:41 PM

Subject: Re: [googlemock: 2170] Mocking Private or Protected Methods

[Keith Ray]

Sometimes instead of multiple inheritance, you 'just' create a subclass that is only used by your tests. It can make protected functions and data public by providing new public functions, not necessarily using the "using" trick.

Your test cases implemented using TEST_F() are actually subclasses of your test fixture class; so sometimes multiple inheritance still doesn't give you the access you want.

There are several versions of CppUnit, the most common one requires too much boilerplate. I hope wxWidgets has modernized CppUnit. 

wxWidgets is fairly unit-test-unfriendly. It's easy to create a mess of hard-to-test/hard-to-mock dependencies when writing code for it. 

--
C. Keith Ray

* (650) 533-6535
* http://agilesolutionspace.blogspot.com/
* http://www.thirdfoundationsw.com/keith_ray_resume_2014_long.pdf

[tfmccarthy_verizon_mail]

Regarding wxWidgets...my _only_ thought there was that UI testing _could_ be done and that is one way. I feel there _must_ be a better way.

 

After reading the other articles, I'm seeing a different way forward for this.

 

I often wonder, "What would the tests for a legacy app look like if they were developed with TDD?" I'm starting to think that legacy private methods would have first been in a separate object/function (and tested) and then composed into the class. Inverting the process is tricky or repetitive but necessary before the class characterization tests can be complete; so the original code may have to change or use test doubles.

 

 

----- Original Message -----

From: Keith Ray

To: tfmccarthy_verizon_mail

Cc: googl...@googlegroups.com

Sent: Wednesday, October 01, 2014 3:34 PM

Subject: Re: [googlemock: 2170] Mocking Private or Protected Methods