MSVC Compiler exhausts heap for larger mock interfaces

[Randy Schott]

Hello group,

I was hoping somebody might have an idea on how I can work around
the following issue. I am unit testing some code that makes use of a
few Direct3D COM interfaces. One of these interfaces,
IDirect3DDevice9, has somewhere around 115 functions in it. Some of
these functions have no arguments, while others have 9, and all the
functions require STDMETHODCALLTYPE, so I'm using macros such as
MOCK_METHOD0_WITH_CALLTYPE.
The problem I'm seeing is that when I mock out this interface, and
then compile code that uses the mock, the MSVC compiler uses up as
much of my available memory as it can get (over 1.5 GB), slows my
machine to a crawl, and eventually exits with an error indicating it
ran out of heap space. The second I remove the include, compilation
uses up less than 100MB and is lightning fast.
From what little I understand of the details, google mock makes
heavy use of macros and templates, which will obviously require a lot
of work on the compiler's part, but 1.5 gigs of ram seems excessive,
and it isn't really going to fly with the other developers on my
team :-).
Does anybody have a suggestion on ways to reduce the amount of
resources the compiler needs while generating the mock class? I'm not
too strong on compiler theory, but I did notice that including the
mock class header from within another header seems to drive up the
resources, so I tried to avoid doing so. Aside from that, I'm out of
ideas.

Thanks for your time and help,
Randy

[Vlad Losev]

Hi Randy -

I have been able to compile a complete mock of IDirect3DDevice9 (see the attached file) in VC 2005 SP1, both in debug and release modes. In the release mode I used the following flags: /DWIN32 /DUNICODE /D_UNICODE /D_WINDOWS /DSTRICT /DWIN32_LEAN_AND_MEAN /DWINVER=0x0500 /D_WIN32_IE=0x0500 /D_RICHEDIT_VER=0x00100 /D_HAS_EXCEPTIONS=0 /nologo /W3 /WX /Gy /Gd /Oi /Zc:forScope /MT /O1 /Os /Oy. The compiled test has run successfully. It also compiled and run with /O2.

So apparently, you are facing some specific combination of source code, compiler version, and compiler options that causes your compiler to crash. Can you please provide enough details for us to be able to reproduce your problem?

Having said that, there are some steps you may take to work around your problem. One thing is to reduce the number of methods you mock. For example, if your tests do not use the CreatePixelShader method, you can define it as

virtual HRESULT CreatePixelShader(const DWORD*, IDirect3DPixelShader9**) {
  ADD_FAILURE() << "CreatePixelShader is not mocked";
}

Another method is to wrap your COM pointers into a C++ object that exposes only operations essential for testing and test using that mock. This may not work in all cases, of course.

Hope this helps you. If not, please reply with more details.

Regards,
Vlad.

[Zhanyong Wan (λx.x x)]

Nice report, Vlad.

Randy, one possibility is that you have a typo in the mock definition.
If somehow you missed a parenthesis somewhere, the compiler could
think the rest of the entire file is one macro argument, which could
make it seriously confused and do insane things. To catch such
errors, you might want to try binary search (i.e. commenting out
roughly half of the code and see if it compiles, repeating until you
find the problem).

Thanks,

--
Zhanyong

[Randy Schott]

Thank you both for the helpful suggestions.  I'm positive it is not a typo.  The code will eventually compile.  I tried Vlad's version, minus the gtest code, since we are using a different framework.  Eventually, it compiles just fine, it just uses way too much memory.  I have tried faking all methods I don't really need, which brought the count from 115+ down to just over 50 methods.  The compiler still eats up 800MB while compiling and takes quite a while.

  On top of that, if I include this header inside of any source file that has other includes (especially includes for other gmocked classes) the compiler eventually fails due to error C1128 (number of sections exceeded object file format limit).  

  For the sake of reference, we are testing a native c++ library that is part of a mixed-mode application.  We are using MS Test, so the unit tests are technically C++/CLI (managed), but they test native code, using native gmock objects.  The original cause of this problem was that I mocked the IDirect3D9 interface (as gmock_IDirect3D9) and then had the constructor of gmock_IDirect3D9 initializing an instance of gmock_IDirect3DDevice9 and setting the default action for gmock_IDirect3D9::CreateDevice() to return the instance of gmock_IDirect3DDevice9.  The end behavior is that a mocked IDirect3D9 always returns a mocked IDirect3DDevice9 by default, without the testing code being required to always set that action up.

  Anyway, I was thinking that normally a class that implements such a large interface wouldn't put all of the actual function definitions into a header file(which the MOCK_METHOD* macros do), thereby forcing the compiler to go through those definitions every time the file is included.  Normally, one would simply put declarations for the functions in the header and put the definitions into an implementation file, causing the compiler to generate the functions only once.

  Is there any way to do this with gmock?  If I could split the definitions for the mocked functions up into a few source files, it should ease the amount of generation the compiler has to do all at once.  It seems like that might help.  Otherwise, I can barely mock any of this interface before I run into memory issues.

Thanks again for all the help,

Randy

2009/4/30 Zhanyong Wan (λx.x x) <w...@google.com>

[Vlad Losev]

Hi Randy,

2009/4/30 Randy Schott <randy...@gmail.com>

Thank you both for the helpful suggestions.  I'm positive it is not a typo.  The code will eventually compile.  I tried Vlad's version, minus the gtest code, since we are using a different framework.  Eventually, it compiles just fine, it just uses way too much memory.  I have tried faking all methods I don't really need, which brought the count from 115+ down to just over 50 methods.  The compiler still eats up 800MB while compiling and takes quite a while.

  On top of that, if I include this header inside of any source file that has other includes (especially includes for other gmocked classes) the compiler eventually fails due to error C1128 (number of sections exceeded object file format limit).  

  For the sake of reference, we are testing a native c++ library that is part of a mixed-mode application.  We are using MS Test, so the unit tests are technically C++/CLI (managed), but they test native code, using native gmock objects.

Since you haven't provided the details of your command line options here I am assuming you are compiling with /clr. This does contribute to increased memory consumption in the compiler: the sample I sent to you compiled with the peak memory consumption of under 360MB with the options I specified. When I tried to add /clr to those option, the memory consumption quickly reached 2GB and the compiler run out of heap space.

So one thing you can do is to separate the testing code that uses gmock into natively compiled modules. You may also try to add /bigobj compiler option as suggested in http://msdn.microsoft.com/en-us/library/8578y171(VS.80).aspx to combat C1128.

My favorite suggestion, though, is to compile all the tests for your library in the native mode while using Google Test. ;-)

The original cause of this problem was that I mocked the IDirect3D9 interface (as gmock_IDirect3D9) and then had the constructor of gmock_IDirect3D9 initializing an instance of gmock_IDirect3DDevice9 and setting the default action for gmock_IDirect3D9::CreateDevice() to return the instance of gmock_IDirect3DDevice9.  The end behavior is that a mocked IDirect3D9 always returns a mocked IDirect3DDevice9 by default, without the testing code being required to always set that action up.

Another thing you may try to do is to create a separate module which would include headers for gmock_IDirect3DDevice9 and export function(s) that create its instance, set appropriate expectations on it and return it as a pointer to IDirect3DDevice9. That module would be the only one having to include have to include the gmock_IDirect3DDevice9 headers. Of course, you lose flexibility of setting expectations on your mock outside that module: you'll have to implement a separate function for each set of expectation you want to set on your IDirect3DDevice9 mocks.

  Anyway, I was thinking that normally a class that implements such a large interface wouldn't put all of the actual function definitions into a header file(which the MOCK_METHOD* macros do), thereby forcing the compiler to go through those definitions every time the file is included.  Normally, one would simply put declarations for the functions in the header and put the definitions into an implementation file, causing the compiler to generate the functions only once.

  Is there any way to do this with gmock?  If I could split the definitions for the mocked functions up into a few source files, it should ease the amount of generation the compiler has to do all at once.  It seems like that might help.  Otherwise, I can barely mock any of this interface before I run into memory issues.

I don't believe this is feasible. Each MOCK_METHOD_ macro expands into a couple of template definitions which have to be put into a header file. And creating separate macros for declarations and definitions would double the number of macros the authors have to maintain and the users have to learn and would introduce a headache of keeping declaration and definition macros in sync.

Regards,
Vlad.

[Zhanyong Wan (λx.x x)]

I took Vlad's program and played with it by commenting out some of the mock methods and measuring the compiler memory usage.  There turns out a HUGE difference with vs without /clr.

# of mock methods | peak memory usage without /clr | peak memory usage with /clr

------------------+--------------------------------+----------------------------

                0 |                          30 MB |                      167 MB

               50 |                         199 MB |                     1010 MB

              100 |                         361 MB |                     1850 MB

So adding /clr causes the compiler to use 5~6 times of memory.  Therefore I second Vlad's suggestion for avoiding /clr when compiling native test code.

I'll take a look at changing the definition of MOCK_METHOD* to make the compiler consume less memory, but it may not be feasible.  Thanks,

2009/4/30 Vlad Losev <vlad...@gmail.com>:

--
Zhanyong

[Randy Schott]

Wow, I'm actually very glad to hear that gmock isn't the culprit here.  I have enjoyed using it so far, and I didn't want to be forced to add a bunch of extra complexity to make it work.  I'll see what I can do about moving the gmock code into native modules.  Thanks again for all your help and keep up the good work!

Randy

2009/5/1 Zhanyong Wan (λx.x x) <w...@google.com>

[Zhanyong Wan (λx.x x)]

Randy,

Glad you enjoyed it.

FYI: I played with Google Mock's implementation a bit to see if I can
reduce VC's memory consumption. I didn't find anything easy that
would lead to a meaningful result. Therefore I'm shelving this for
now. Thanks,

2009/5/1 Randy Schott <randy...@gmail.com>:

--
Zhanyong