CATCH_CONFIG_RUNNER usage

[Maxim]

Hi!

I am trying to use Catch to build a unit test library that should be invoked by an external application that is Catch-agnostic. I intend to keep unit tests in multiple files and export an entry function RunTests() that creates and runs Catch session.

This is what the library code looks like:

// TestRunner.cpp

#define CATCH_CONFIG_RUNNER

#include "catch.hpp"

void RunTests()

{

   Catch::Session session;

   session.run();

}

// Project1Test.cpp

#include "catch.hpp"

TEST_CASE( "Unit test 1", "[Project1]")
{
   // Test case body.
}

// More TEST_CASE definitions...

The problem is when I run the application, Catch reports that no tests were executed. TEST_CASE defined in "Project1Test.cpp" does not register itself to the session defined in "TestRunner.cpp". However, if I copy TEST_CASE from "Project1Test.cpp" and paste it into "TestRunner.cpp", the test is registered and executed properly.

Please let me know what I am missing here. Is it even possible to keep tests in separate files in a setup like this?

Thank you,

Maxim

[Martin Moene]

Your test does not contains any assertion and is considered not-run as a consequence (see also option -w NoAssertions)

Add

REQUIRE( false );

to your test case and it shows up ;)

cheers,
Martin

[Martin Moene]

... If there are no assertions at all

main -w NoAssertions

yields

No tests ran

[JBRWilkinson]

In additional to Martin's suggestions, is Project1Test.cpp on your link line? This has bitten us before.

[Maxim]

Hi Martin,

Thanks for your response. There is a "REQUIRE" assertion in the test case body, but I omitted it from the code example. Sorry I should have been more clear about it.

The problem is that TEST_CASE is executed only if it is defined in the same file as the entry function that runs Catch session (TestRunner.cpp, in my example). If TEST_CASE is defined in another file (Project1Test.cpp, in my example), it does not get registered with the session object and executed.

I just found sort of a workaround for this problem: if I define a dummy no-op function in Project1Test.cpp and call this function from RunTests() in TestRunner.cpp, then TEST_CASE defined in Project1Test.cpp is registered and executed properly. I wonder if the same result can be achieved somehow without calling a dummy function.

Thanks,

Maxim

[Maxim]

Thanks for your suggestion. The output from my linker says that Project1Test.obj is included (I am using MS VC++ 2010).

[Martin Moene]

I created your example with `REQUIRE(false)` added (attached).

Tested it at work (VS2010) and at home (g++ 4.8.1) and both work:

prompt>g++ -Wall -I../../Catch/single_include -o main.exe main.cpp TestRunner.cpp Project1Test.cpp && main

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 is a Catch v1.0 b11 host application.
Run with -? for options

-------------------------------------------------------------------------------
Unit test 1
-------------------------------------------------------------------------------
Project1Test.cpp:5
...............................................................................

Project1Test.cpp:8: FAILED:
  REQUIRE( false )

===============================================================================
1 test case - failed (1 assertion - failed)

[Martin Moene]

Which version of Catch areyou using?

[Maxim]

Hi Martin,

In my example, TestRunner.cpp and Project1Test.cpp are built into a static library, which is then linked to an executable that calls RunTests(). The complete VS2010 solution is attached for your convenience.

This section of GoogleTest doc mentions a bug in VS which is likely causing the problem. It can be worked around by calling a dummy Touch() function, as in the attached example, but I wonder if there is more appropriate solution.

Of course, the other way is to build Catch unit tests directly into the executable.

Thanks,

Maxim

[Maxim]

CATCH v1.0 build 11 (master branch)

Generated: 2013-10-23 15:34:32.120349

[Martin Moene]

A good thing your problem with solutions turn up here in the end!

From the Important note for Visual C++ users (googletest) :

The general conclusion here is: make your life easier - do not write your tests in libraries!

cheers,
Martin

[Phil Nash]

Hi Maxim,

It looks almost certain that you're seeing the "optimised out constructors" issue that the gtest doc references - it's one of the caveats of the auto-registering technique that Catch (and many other test frameworks) employs. I don't believe it's actually a bug, as that doc suggests, but rather just an optimisation that is outside the scope of the standard (as the standard says nothing about libraries). I could be wrong there, though,

As far as I know the touch() method is the only reliable workaround (I've experimented with the dllexport technique in the passed - with varying results - doesn't seem reliable enough to use). I've only ever had to do that once - with an in-house test framework on a project I worked on once where the tests had to be written in the same dll as the code they were testing! - presumably a misguided attempt to give the tests access to internal interfaces).

Obviously the best solution is simple to not put the tests in a separate lib (or dll) in the first place (as Martin suggests). Is there a compelling reason for doing so?

Regards,

[)o

IhIL..

[Maxim]

On Thursday, November 7, 2013 2:25:24 AM UTC-8, Phil Nash wrote:

Hi Maxim,

Hi Phil, thanks for your input.

It looks almost certain that you're seeing the "optimised out constructors" issue that the gtest doc references - it's one of the caveats of the auto-registering technique that Catch (and many other test frameworks) employs. I don't believe it's actually a bug, as that doc suggests, but rather just an optimisation that is outside the scope of the standard (as the standard says nothing about libraries). I could be wrong there, though,

Agree with you here - this is rather an optimization that compiler users should be aware of.

 

As far as I know the touch() method is the only reliable workaround (I've experimented with the dllexport technique in the passed - with varying results - doesn't seem reliable enough to use). I've only ever had to do that once - with an in-house test framework on a project I worked on once where the tests had to be written in the same dll as the code they were testing! - presumably a misguided attempt to give the tests access to internal interfaces).

Obviously the best solution is simple to not put the tests in a separate lib (or dll) in the first place (as Martin suggests).

Yes, this is what I ended up doing.

 

Is there a compelling reason for doing so?

Not really. I tried to do it that way because I came from .NET world, where keeping tests in assemblies separate from a runner executable is a norm.

Regards,

[)o

IhIL..

[arvind natarajan]

Hi Guys 

I know its been a while since this thread but I am running into something very similar.

I have a TestRunner.cxx file which has the main declaration and I have a Test1.cxx file which has a simple test case as suggested in this thread. 

When I build and link the libraries, the test in Test1.cxx does not get executed. 

However when I do a g++ TestRunner.cxx Test1.cxx and then run the exe file, the test gets executed. 

Now building this way means that I am not linking the other libraries in my project. Can someone suggest? 

Thanks in Advance. 

TestRunner.cxx 

#define CATCH_CONFIG_MAIN 

#include "catch.hpp"