Hierarchical test suites in googletest

[eliben]

Hello,
Many unit testing frameworks allow nesting of test suites into other
test suites, with tests themselves being the leaves. For example:

TestMath:
TestTrig:
sine_test
cosine_test
... etc ...
TestGeom:
test_triangle
... etc ...

In this example, using googletest terminology, sine_test,
triangle_test etc. are "tests" (functions with some assertions).
TestGeom & TestTrig are test cases, so in googletest I'd write:

TEST(TestGeom, test_triangle)
{
}

However, I don't see a way to express a higher-level grouping in
googletest - in this particular example a way to say that TestMath is
a "suite" that has some "test cases" contained in it.

Did I miss something or is googletest limiting the hierarchical
organization of tests to 2 levels by design?

Thanks in advance, and happy holidays to you all
Eli

[Joey Oravec]

What's your goal with the hierarchy? Remember when you're defining a test like:

 

TEST_F(TestTrig, sine_test)

 

this defines a test (sine_test) that contains some assertions. It's defined as an object (TestTrig) that can contain member functions, member variables, etc. You can use inheritance and define your own structure to reuse code like:

 

class TestMath : public ::testing::Test {
public:

    // Some common methods

    long calcSum(long A[], size_t len);

};

 

class TestTrig : public TestMath {
public:

    // Trig-specific methods

    float calcHypotenuse(float A, float B);
};

 

But remember from the gtest primer: "Note that different tests in the same test case have different test fixture objects, and Google Test always deletes a test fixture before it creates the next one. Google Test does not reuse the same test fixture for multiple tests. Any changes one test makes to the fixture do not affect other tests." For example if you had:

 

TEST_F(TestTrig, sine_test);

TEST_F(TestTrig, cosine_test);

 

then those two tests are defined as separate objects. So if you assign a member variable in sine_test, the value won't be there there during cosine_test because it's a separate object. From this perspective gtest has no hierarchy at all and each test stands completely alone. Only code and global state is shared between tests.

 

If your goal is simply to run a group of tests, you can achieve that by clever naming. For example if you defined some tests:

 

TEST_F(TestMath_TestTrig, sine_test);

TEST_F(TestMath_TestTrig, cosine_test);

TEST_F(TestMath_TestGeom, test_triangle);

 

then you could use filtering to run any subset of tests like:

 

gtest --gtest_filter=TestMath*

or

gtest --gtest_filter=TestMath_TestTrig*:TestMath_TestGeom*

 

there are a lot of options, so you'll have to pick one that works for you.

 

PS - Your example had underscores in the name, but that's prohibited. See this thread for details: https://groups.google.com/group/googletestframework/browse_thread/thread/379034edb804be9e

 

-joey

[eliben]

> If your goal is simply to run a group of tests, you can achieve that by
> clever naming. For example if you defined some tests:
>
>  TEST_F(TestMath_TestTrig, sine_test);
> TEST_F(TestMath_TestTrig, cosine_test);
>  TEST_F(TestMath_TestGeom, test_triangle);
>
> then you could use filtering to run any subset of tests like:
>
> gtest --gtest_filter=TestMath*
> or
> gtest --gtest_filter=TestMath_TestTrig*:TestMath_TestGeom*
>

Yes, my goal is logical hierarchical grouping of suites. When a set of
tests for some code is very large, such hierarchy is helpful. As with
anything in programming, real hierarchical structures can be replaced
by name "mangling" (i.e. how namespaces are usually represented in C),
but for many reasons true hierarchical constructs are superior.

Eli

[eliben]

> PS - Your example had underscores in the name, but that's prohibited. See

> this thread for details:https://groups.google.com/group/googletestframework/browse_thread/thr...
>

This thread says underscores are OK in test names, but not recommended
in test case/fixture names, which doesn't contradict my examples. Am I
wrong?
Eli

[Joey Oravec]

On Wed, Dec 29, 2010 at 3:15 AM, eliben <eli...@gmail.com> wrote:

Yes, my goal is logical hierarchical grouping of suites. When a set of
tests for some code is very large, such hierarchy is helpful. As with
anything in programming, real hierarchical structures can be replaced
by name "mangling" (i.e. how namespaces are usually represented in C),
but for many reasons true hierarchical constructs are superior.

 

Ok, now I'm on the same page. The test_case_name and test_name strings are recorded separately within the objects. It would be one thing to extend the names for filtering, but also consider:

 

Global Environment setup / teardown

TestCase setup / teardown

... more hierarchy?

Test setup / teardown

 

It would be really useful to implement hierarchy here too, so gtest could follow a chain of SetUp() routines and get you into the X/Y/Z state. Unfortunately it doesn't work that way today.

 

-joey