Using CATCH with C++ template based tests -- see Issue #46

[Brian Plummer]

Witness the following :

------------------------------------------------------------

#include <CATCH/Latest/catch.hpp>

#include <iostream>

static int iter = 1;

template <typename IntType> void dotest(const std::string & testname) {

   std::cout << ("Verify/" + testname + "--ENTRY") << " iter=" << iter++;

   SECTION("Verify/" + testname,"") {

      std::cout << " :: IN SECTION" << std::endl;

      REQUIRE(1 == 1);

      return;

   }

   std::cout << std::endl;

}

TEST_CASE("Basic Test","[alltypes]") {

   dotest<int>("int");

   dotest<long>("long");

   dotest<char>("char");

}

------------------------------------------------------------

Output:

------------------------------------------------------------

Verify/int--ENTRY iter=1 :: IN SECTION

Verify/long--ENTRY iter=2

Verify/char--ENTRY iter=3

Verify/int--ENTRY iter=4

Verify/long--ENTRY iter=5 :: IN SECTION

Verify/char--ENTRY iter=6

Verify/int--ENTRY iter=7

Verify/long--ENTRY iter=8

Verify/char--ENTRY iter=9 :: IN SECTION

===============================================================================

All tests passed (3 assertions in 1 test case)

------------------------------------------------------------

Everything "works" fine.  All three of my tests were executed as desired.  BUT, as you can see, dotest<...> was executed 9 times.  The actual test within the SECTION was only executed 3 times.  

I stepped through the code in the debugger and found that there was a test :

    // This indicates whether the section should be executed or not

    Section::operator bool() const {

        return m_sectionIncluded;

    }

that returns false on 6 of the 9 calls causing the SECTION contents to be skipped.  I am a first class newbie to CATCH and this is my first attempt to use it in real unit tests in real code.  That said...  This is a problem because I have an elaborate Entry/Exit and Tracing framework that I use in troubleshooting code.  When I turned this framework on, there was an explosion of trace output that I couldn't understand.  In the real code where I attempted to use this idiom, the equivalent of dotest<...> was called 26 times which means it was actually called 676 (26 x 26) times.

At this point I would call this a very minor nuisance.  But I would like to deal with this if possible.  Because of the kind of development I do (template based libraries) I use this idiom in almost ever test I conduct.  Because of the combination explosion of template arguments, I have some tests that effectively call a single template based test over 1000 times.  For those playing the home game, that would mean, using my example, dotest<...> would actually be called more than 1000000 times!!  Even at that, it is still a nuisance, but I'd like to get rid of it if I can.  This example is perhaps too trivial.  Imagine if I needed to do significant test setup prior to entering the SECTION.  So, I guess, I would just do that setup inside of the SECTION.  OK, I'm learning as I go...

Any direction you could provide would be greatly appreciated!!

FYI, as I suggest in the title, I found the idiom that I described in CATCH issue #46 from July 29, 2011.

Thanks for your time!!