announce: test shuffling

[Zhanyong Wan (λx.x x)]

Hi,

Josh Kelley has implemented test shuffling for gtest.  You can now randomize your test order with a simple command line flag (or an environment variable).  In addition to making running tests more fun, it helps to uncover hidden dependencies between tests. (In case you don't realize, dependencies between tests are BAD, as they make it harder to add/remove tests and to debug a test failure.)

Taken from the advanced guide wiki (http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Shuffling_the_Tests):

You can specify the --gunit_shuffle flag (or set the GUNIT_SHUFFLE environment variable to 1) to run the tests in a program in a random order. This helps to reveal bad dependencies between tests.

By default, gUnit uses a random seed calculated from the current time. Therefore you'll get a different order every time. The console output includes the random seed value, such that you can reproduce an order-related test failure later. To specify the random seed explicitly, use the --gunit_random_seed=SEED flag (or set the GUNIT_RANDOM_SEED environment variable), where SEED is an integer in the range [0, 99999]. The seed value 0 is special: it tells gUnit to do the default behavior of calculating the seed from the current time.

If you combine this with --gunit_repeat=N, gUnit will pick a different random seed and re-shuffle the tests in each iteration

This feature is available in the SVN trunk, starting with revision 325.  It will be part of our upcoming (in several days) release of gtest 1.4.0.

Thanks to Josh for contributing the idea and the original patch.  Thanks to others who have participated in the design discussion.  Also thanks to Vlad for a careful code review.  Cheers,
--
Zhanyong

[Zhanyong Wan (λx.x x)]

Sorry, I copied from the wrong place.  The wiki snippet should be:

You can specify the --gtest_shuffle flag (or set the GTEST_SHUFFLE environment variable to 1) to run the tests in a program in a random order. This helps to reveal bad dependencies between tests.

By default, Google Test uses a random seed calculated from the current time. Therefore you'll get a different order every time. The console output includes the random seed value, such that you can reproduce an order-related test failure later. To specify the random seed explicitly, use the --gtest_random_seed=SEED flag (or set the GTEST_RANDOM_SEED environment variable), where SEED is an integer between 0 and 99999. The seed value 0 is special: it tells Google Test to do the default behavior of calculating the seed from the current time.

If you combine this with --gtest_repeat=N, Google Test will pick a different random seed and re-shuffle the tests in each iteration.

2009/10/1 Zhanyong Wan (λx.x x) <w...@google.com>

Hi,

Josh Kelley has implemented test shuffling for gtest.  You can now randomize your test order with a simple command line flag (or an environment variable).  In addition to making running tests more fun, it helps to uncover hidden dependencies between tests. (In case you don't realize, dependencies between tests are BAD, as they make it harder to add/remove tests and to debug a test failure.)

Taken from the advanced guide wiki (http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Shuffling_the_Tests):

 

This feature is available in the SVN trunk, starting with revision 325.  It will be part of our upcoming (in several days) release of gtest 1.4.0.

Thanks to Josh for contributing the idea and the original patch.  Thanks to others who have participated in the design discussion.  Also thanks to Vlad for a careful code review.  Cheers,
--
Zhanyong

--
Zhanyong