running a single test case / fixture programmatically (via gtest's API)

eliben

unread,

Dec 5, 2010, 4:02:04 AM12/5/10

to Google C++ Testing Framework

Hello,

What is the best way to run a single test fixture / test from my gtest
tests via the API (not command-line)?

One solution is call RUN_ALL_TESTS, setting a filter prior to that
with ::testing::GTEST_FLAG ... This works, but feels a bit cumbersome.
Is there something that allows me, i.e:

RUN_TEST_FIXTURE(MyFixtureClase)

or

RUN_TEST(MyTestClass)

?

Thanks in advance

Vlad Losev

unread,

Dec 6, 2010, 6:47:36 PM12/6/10

to eliben, Google C++ Testing Framework

There is no supported way to run partial set of tests in Google Test. Your only option is to run a the test binary in a subprocess with the appropriate --gtest_filter flag. Please note, running RUN_ALL_TESTS() multiple times is not supported. It may work now, but it may very well get broken in some subtle ways with in future update of the library.

Zhanyong Wan (λx.x x)

unread,

Dec 6, 2010, 7:21:08 PM12/6/10

to Vlad Losev, eliben, Google C++ Testing Framework

On Mon, Dec 6, 2010 at 3:47 PM, Vlad Losev <vl...@losev.com> wrote:
> There is no supported way to run partial set of tests in Google Test.

Actually, we intend it to work to change the value of
testing::GTEST_FLAG(filter) before calling InitGoogleTest(). If it's
not clear from the docs, we should clarify it.

--
Zhanyong

eliben

unread,

Dec 6, 2010, 11:56:52 PM12/6/10

to Google C++ Testing Framework

On Dec 7, 1:47 am, Vlad Losev <v...@losev.com> wrote:
> There is no supported way to run partial set of tests in Google Test. Your
> only option is to run a the test binary in a subprocess with the appropriate
> --gtest_filter flag. Please note, running RUN_ALL_TESTS() multiple times is
> not supported. It may work now, but it may very well get broken in some
> subtle ways with in future update of the library.
>

I see.

Such an option would facilitate:

1. Integrating gtest tests into larger frameworks that "drive" the
testing process. I don't think the current "let other frameworks
drive" solution is flexible enough.
2. More seamless integration of gtest into GUI/web-based runners. I.e.
it would be great to be able to see the list of all fixtures/tests in
a GUI and choose which to run in a simple way. Consider for example a
large system with a vast amount of unit tests and system tests - some
are short "smoke tests" and other are comprehensive, running for
hours. Choosing which to run when would be nice.

Eli

Vlad Losev

unread,

Dec 7, 2010, 1:30:28 AM12/7/10

to Zhanyong Wan (λx.x x), eliben, Google C++ Testing Framework

2010/12/6 Zhanyong Wan (λx.x x) <w...@google.com>

On Mon, Dec 6, 2010 at 3:47 PM, Vlad Losev <vl...@losev.com> wrote:
> There is no supported way to run partial set of tests in Google Test.

Actually, we intend it to work to change the value of
testing::GTEST_FLAG(filter) before calling InitGoogleTest(). If it's
not clear from the docs, we should clarify it.

Yes, that will work, but only for a single invocation of RUN_ALL_TESTS(). Although, as far as I understand, this is not exactly what Eli was looking for. And there is a catch: if the process command line contains the --gtest_filter flag, it will override whatever is assigned to GTEST_FLAG(filter) prior to calling InitGoogleTest(). The caller will have to strip that flag from the command line manually before calling InitGoogleTest().

> Your
> only option is to run a the test binary in a subprocess with the appropriate
> --gtest_filter flag. Please note, running RUN_ALL_TESTS() multiple times is
> not supported. It may work now, but it may very well get broken in some
> subtle ways with in future update of the library.
>
> On Sun, Dec 5, 2010 at 1:02 AM, eliben <eli...@gmail.com> wrote:
>>
>> Hello,
>>
>> What is the best way to run a single test fixture / test from my gtest
>> tests via the API (not command-line)?
>>
>> One solution is call RUN_ALL_TESTS, setting a filter prior to that
>> with ::testing::GTEST_FLAG ... This works, but feels a bit cumbersome.
>> Is there something that allows me, i.e:
>>
>> RUN_TEST_FIXTURE(MyFixtureClase)
>>
>> or
>>
>> RUN_TEST(MyTestClass)
>>
>> ?
>>
>> Thanks in advance
>

--
Zhanyong

Vlad

Vlad Losev

unread,

Dec 7, 2010, 1:34:12 AM12/7/10

to eliben, Google C++ Testing Framework

A while ago we had a discussion on this list about an convenient API that would allow users to control from inside the process the set of tests to run. It's a non-trivial change, given that it has to be compatible with the number of existing features, such as sharding and shuffling. Given the staffing situation with gtest, neither I not Zhanyong are likely to implement it any time soon. Maybe we'll have some volunteers for it?