Testing with dynamically linked DLLs (Visual Studio 2008)

[Greg Malcolm]

Hi!

I'm working on a big Visual Studio 2008 based project, mostly made of
dynamic DLL libraries. The code base is fairly massive so rather than
keep the tests in separate project modules, I would like to include
them in a separate folder in each of the existing dlls with an
executable test runner invoking them. The problem is that the Test
Runner doesn't seem to be able to see test installed outside of the
executable.

I will be the first to admit this solution is a bit ugly, but I don't
want to expose every dllexport every class.

Is there a way to make gtest run tests in all these dlls? Or can
anyone offer a viable alternative strategy for testing the code in the
dlls?

The best I can think of write now is to create an executable for every
single dll and give each dll its own Main method. I'd prefer to test
all the dlls at once though.

I should probably mention upfront that I've changed the "Use of MFC"
setting to "MFC Dynamically Linked dll" to get rid of linker errors. I
don't think that's an issue, but just in case...

- Greg

[Vlad Losev]

Hi Greg,

Please see if this answer helps you: http://code.google.com/p/googletest/wiki/GoogleTestFAQ#I_define_my_tests_in_a_library_(.lib)_and_Google_Test_doesn%27.

Best regards,
Vlad.

[Zhanyong Wan (λx.x x)]

Hi Vlad,

On Thu, Mar 19, 2009 at 4:37 PM, Vlad Losev <vl...@google.com> wrote:
> Hi Greg,
>
> Please see if this answer helps you:
> http://code.google.com/p/googletest/wiki/GoogleTestFAQ#I_define_my_tests_in_a_library_(.lib)_and_Google_Test_doesn%27.

The FAQ says "static library". Should we change it to "either .lib or .dll"?

I also think it helps to make this info more visible. Perhaps move it
to the primer? Thanks,

>
> Best regards,
> Vlad.
>
> On Thu, Mar 19, 2009 at 3:53 PM, Greg Malcolm <gregm...@gmail.com> wrote:
>>
>>
>> Hi!
>>
>> I'm working on a big Visual Studio 2008 based project, mostly made of
>> dynamic DLL libraries. The code base is fairly massive so rather than
>> keep the tests in separate project modules, I would like to include
>> them in a separate folder in each of the existing dlls with an
>> executable test runner invoking them. The problem is that the Test
>> Runner doesn't seem to be able to see test installed outside of the
>> executable.
>>
>> I will be the first to admit this solution is a bit ugly, but I don't
>> want to expose every dllexport every class.
>>
>> Is there a way to make gtest run tests in all these dlls? Or can
>> anyone offer a viable alternative strategy for testing the code in the
>> dlls?
>>
>> The best I can think of write now is to create an executable for every
>> single dll and give each dll its own Main method. I'd prefer to test
>> all the dlls at once though.
>>
>> I should probably mention upfront that I've changed the "Use of MFC"
>> setting to "MFC Dynamically Linked dll" to get rid of linker errors. I
>> don't think that's an issue, but just in case...
>>
>> - Greg
>
>

--
Zhanyong

[Vlad Losev]

2009/3/19 Zhanyong Wan (λx.x x) <w...@google.com>

Hi Vlad,

On Thu, Mar 19, 2009 at 4:37 PM, Vlad Losev <vl...@google.com> wrote:
> Hi Greg,
>
> Please see if this answer helps you:
> http://code.google.com/p/googletest/wiki/GoogleTestFAQ#I_define_my_tests_in_a_library_(.lib)_and_Google_Test_doesn%27.

The FAQ says "static library".  Should we change it to "either .lib or .dll"?

I also think it helps to make this info more visible.  Perhaps move it
to the primer?  Thanks,

I have added the warning to the GoogleTestPrimer page and modified the answer to point there. Hopefully, the warning is better worded then the original answer. Please note that I have also updated the question text (to just  use word "library") and that invalidated the anchor in the link I gave to Greg. But the question is still there, the last but one on the page.
 
Regards,
Vlad.

[Greg Malcolm]

Thank for getting back to me so quickly!

I tried modifying the /OPT linker setting as suggested, but that did
not solve the problem. Also I am working in a Debug configuration, so
I think most optimizations are turned off anyway.

I will have a try at creating a new test project consisting of gtest,
an test runner exe, a dll with tests to see if the problem is easily
reproducible.

[Greg Malcolm]

I created a brand new project and can confirm it did reproduce the
problem.

A zip file containing the full project is available on request. Here
are the exact steps I followed:

A) Created Main App
1. Created win32 MFC console application Solution in Visual Studio
2008
2. Explicitly set /OPT:NOREF in project linking settings
3. Added InitGoogleTest() and RUN_ALL_TESTS() into the wizard
generated main() function. I confirmed that it creates a test report.

B) Added gtest 1.2.1 library into the solution
4. Added the main gtest.vsproj project to the solution. It
automatically converted to VS2008 flavor.
5. Changed gtest's "Use of MFC" project setting to "Use MFC in a
Shared DLL"
6. Added gtest to main app as a build dependency
7. Added gtest include folder to “Additional Include Directories” in
main app's project settings.

C) Added a DLL which will house the test cases.
8. Added an MFC dll, with "dynamically linked to MFC" specified.
9. Added the new dll as a dependency of the main app
10. Explicitly set /OPT:NOREF in project settings in the new dll
11. Added gtest as a dependency of the new dll
12. Added gtest include folder to "Additional Include Directories" in
main app's project settings.
13. Created a dll-test.cc file with a basic test:

#include "stdafx.h"

#include <gtest/gtest.h>

TEST(ExtTest, Test_It_Works) {
EXPECT_TRUE(false);
}

On building the solution, although I get a small test report, the
dll's test case does not appear on it.

I'll try some more experiments to see if its MFC, unicode or dynamic
linking causing problems.

[Vlad Losev]

Hi Greg,

I was able to reproduce the problem. It appears to be caused by a bug in VC++ linker which MS is not going to fix. A workaround they suggest is rather simple: define an exportable dummy function in your library with tests:

__declspec(dllexport) int PullInMyTestLibrary() { return 0; }

Then in your main program call that function (from main or at the global scope):

int PullInMyTestLibrary();
static int dummy = PullInMyTestLibrary();

This will make your main program reference PullInMyTestLibrary symbol in your test library and force the linker to include it in the build.

You may have another problem, though. If you use the original gtest project, it is build as a static library. If you put your tests in a DLL, both your main program and your dll link in their own copy of that static library. Each copy of the static library has its own copy of the global data which, as you understand, is really bad. At best, the tests that your test library registers with gtest will not be visible from the main program. To avoid such problems, you will either need to change Google Test into a dll or make your test library a static one. Note, if you choose to make your library static, you don't have to use __declspec(dllexport).

Hope this helps. I will update the the wiki soon.

Thanks for helping to figure this out!
- Vlad.

[Greg Malcolm]

I was thinking along the same lines with making a __declspec
(dllexport) call. Sadly it didn't seem to solve the problem, but there
was a side effect.

Beforehand breakpoints placed on tests in the exe project would
trigger at the "dynamic initializer" stage. They wouldn't for tests in
dll.

With the "pull" method getting invoked. The breakpoints are getting
triggered. So maybe the remaining problem is that gtest is a static
libary?

Turning my project dll into a static library is not an option so I
will try and convert gtest into one. Probably going to need to add a
lot __declspec(dllexport) macros to make it fly, but I'll see how it
goes...

Thanks again for all the help!

- Greg

[Zhanyong Wan (λx.x x)]

Another idea is to create a static lib for your project in addition to
a DLL. You'll continue to use the DLL in production, and will use the
static lib for testing. You'll end up linking twice, but each file is
compiled only once. This isn't ideal either, but may be less work for
you.

>
> Thanks again for all the help!
>
> - Greg
>

--
Zhanyong

[Vlad Losev]

I have updated the note at http://code.google.com/p/googletest/wiki/GoogleTestPrimer?ts=1237600674&updated=GoogleTestPrimer#Important_note_for_Visual_C++_users. Hopefully it is good enough now.

On Fri, Mar 20, 2009 at 4:20 PM, Greg Malcolm <gregm...@gmail.com> wrote:

I was thinking along the same lines with making a __declspec
(dllexport) call. Sadly it didn't seem to solve the problem, but there
was a side effect.

Beforehand breakpoints placed on tests in the exe project would
trigger at the "dynamic initializer" stage. They wouldn't for tests in
dll.

With the "pull" method getting invoked. The breakpoints are getting
triggered. So maybe the remaining problem is that gtest is a static
libary?

The problem is that gtest the static library ends up linked both into your DLL with tests and into your main program. Thus you end up with two copies of gtest's global data. When the tests self-register at startup, they modify one copy of gtest data. When the main function looks up registered tests it check another, untouched copy of the data. Naturally, it sees no tests registered and runs nothing.

Turning my project dll into a static library is not an option so I
will try and convert gtest into one. Probably going to need to add a
lot __declspec(dllexport) macros to make it fly, but I'll see how it
goes...

We generally prefer to have tests in the project that is separate from the one that contains code under test. This way we are able to put our tests in separate .exe files and avoid all the problems discussed in this thread. :-)

Thanks again for all the help!

- Greg

 

Thanks for helping get to the bottom of the problem!
- Vlad