messages for success as well as failure?

[stevej]

I see this has been mentioned before, but I also would like the
ability to turn on printing of a message when a test succeeds as well
as when it fails.

I've seen various reasons for this request. My reason is that it gives
the viewer a sense of the test coverage that has been achieved. This
could suggest the need for new tests. It could also familiarize a new
developer with the existing tests.

I haven't studied the googletest code so I don't know how hard this
would be to implement but I'd guess it wouldn't be too hard. (In which
case, I might make the changes myself... but if there has been any
work on this I wouldn't want to duplicate efforts)

[Keith Ray]

What I like about google test is the it does print success messages

[ Run    ] testFixture.testName
[     OK ] testFixture.testName (0ms)
===========
[ PASSED ] 99 tests

--
C. Keith Ray
 Web: http://industriallogic.com
 Twitter: @CKeithRay, @IndustrialLogic

Amplify Your Agility
Coaching | Training | Assessment | eLearning

--

[Steve Jaffe]

Perhaps I wasn't clear. I'm asking for messages on success with the same amount of detail as those for failure. That means an individual message for each test rather than simply a summary. I wouldn't want this all the time by any means, but I think it would be a useful option to exercise occasionally.

---

From: Keith Ray <keit...@gmail.com>
To: stevej <steve...@yahoo.com>
Cc: Google C++ Testing Framework <googletes...@googlegroups.com>
Sent: Mon, July 19, 2010 7:51:54 PM
Subject: Re: [googletest] messages for success as well as failure?

[Vlad Losev]

Do you mean printing details of execution each successful test assertion, such EXPECT_EQ()?

[Steve Jaffe]

Yes, basically the same information you would get if the test failed.

---

From: Vlad Losev <vl...@google.com>
To: Google C++ Testing Framework <googletes...@googlegroups.com>
Sent: Tue, July 20, 2010 12:03:51 PM
Subject: Re: [opensource-gtest] Re: [googletest] messages for success as well as failure?

[Vlad Losev]

I see. It should be noted that the purpose of a test is to make sure the code performing to the specifications set by the design, and if it does deviate from those specifications, tell the user where and how precisely that happens. Google Test is geared towards that in a couple of ways. First, the messages generated by test assertions are all designed to inform the user how the test assumptions are being broken.Secon, for performance reasons, the library only reports as much information as necessary.

What would be the purpose of telling the user explicitly that the code performs as designed - even when the absence of the output conveys the same message? Can't that purpose be achieved by means other than re-engineering Google Test?

Thanks,
Vlad

[Tim Baverstock]

Sorry to jump in, but...

Sometimes a successful but silent test feels anticlimactic, or I can't believe a test unexpectedly passed.

Mostly that's when I'm not doing TDD.

I normally just add a print statement somewhere, briefly, but I should probably use coverage... ;)

On 20 Jul 2010 22:12, "Vlad Losev" <vl...@google.com> wrote:
> I see. It should be noted that the purpose of a test is to make sure the
> code performing to the specifications set by the design, and if it does
> deviate from those specifications, tell the user where and how precisely
> that happens. Google Test is geared towards that in a couple of ways. First,
> the messages generated by test assertions are all designed to inform the
> user how the test assumptions are being broken.Secon, for performance
> reasons, the library only reports as much information as necessary.
>
> What would be the purpose of telling the user explicitly that the code
> performs as designed - even when the absence of the output conveys the same
> message? Can't that purpose be achieved by means other than re-engineering
> Google Test?
>
> On Tue, Jul 20, 2010 at 11:45 AM, Steve Jaffe <steve...@yahoo.com> wrote:
>
>>
>> Yes, basically the same information you would get if the test failed.

>> ------------------------------
>> *From:* Vlad Losev <vl...@google.com>
>> *To:* Google C++ Testing Framework <googletes...@googlegroups.com>
>> *Sent:* Tue, July 20, 2010 12:03:51 PM
>> *Subject:* Re: [opensource-gtest] Re: [googletest] messages for success as

>> well as failure?
>>
>> Do you mean printing details of execution each successful test assertion,
>> such EXPECT_EQ()?
>>
>> On Mon, Jul 19, 2010 at 6:44 PM, Steve Jaffe <steve...@yahoo.com>wrote:
>>
>>>
>>> Perhaps I wasn't clear. I'm asking for messages on success with the same
>>> amount of detail as those for failure. That means an individual message for
>>> each test rather than simply a summary. I wouldn't want this all the time by
>>> any means, but I think it would be a useful option to exercise occasionally.

>>> ------------------------------
>>> *From:* Keith Ray <keit...@gmail.com>
>>> *To:* stevej <steve...@yahoo.com>
>>> *Cc:* Google C++ Testing Framework <googletes...@googlegroups.com>
>>> *Sent:* Mon, July 19, 2010 7:51:54 PM
>>> *Subject:* Re: [googletest] messages for success as well as failure?
>>>
>>> What I like about google test is the it *does *print success messages

[Vlad Losev]

On Thu, Jul 22, 2010 at 1:29 AM, Tim Baverstock <wea...@google.com> wrote:

Sorry to jump in, but...

Sometimes a successful but silent test feels anticlimactic, or I can't believe a test unexpectedly passed.

Yes, you need to trust the testing framework implicitly to accept silence as a sign of success. You need some degree of familiarity with it to gain confidence; some people find trusting it harder than others, and I am, of course, not at all impartial here. :)

The reason we don't have a provision for success reporting is that some projects that are using Google Test have tests that contain huge numbers of test assertions. Printing out success messages for them all -- or even simply constructing them -- was slowing those tests significantly. That goes against our stated goal of speed, so it was removed.

Now what are the workarounds to let one track the successful execution of a test? Apart from patching the library, one can print a progress message, like this:

if (!testing::Test::HasFailure()) {
  const ::testing::TestInfo* const test_info =
      ::testing::UnitTest::GetInstance()->current_test_info();
  printf(
      "Executions of %s.%s successful so far (%s:%d)\n",
      test_info->test_case_name(),
      test_info->test_name(),
      __FILE__,
     __LINE__);
}

If you want this information to go to the listeners, stream the message into the SUCCEED() macro instead of printing it:

if (!testing::Test::HasFailure()) {
  const ::testing::TestInfo* const test_info =
      ::testing::UnitTest::GetInstance()->current_test_info();
  SUCCEED()
      << "Execution of " << test_info->test_case_name() << "."
      << test_info->test_name() << " successful so far.";
}

For simplicity, the above can be wrapped into a macro. And in my experience, you can gain more confidence in your tests by making them smaller, usually by splitting them. I take a hard look at any test that is more than 50 lines long and contains more than 20-25 assertions.

Mostly that's when I'm not doing TDD.

I normally just add a print statement somewhere, briefly, but I should probably use coverage... ;)

You should. ;) Of course, the print statements only let you track a progress of your test, they do not tell you how much of your code is covered by tests. You need to run your tests under a coverage tool such as gcov/lcov or DevPartner Studio for that.

[Steve Jaffe]

Thanks, using the SUCCEED() macro to do some customized success reporting sounds like a reasonable approach.

---

From: Vlad Losev <vl...@google.com>

To: Tim Baverstock <wea...@google.com>
Cc: Google C++ Testing Framework <googletes...@googlegroups.com>
Sent: Thu, July 22, 2010 4:51:00 PM
Subject: Re: [gUnit-users] Re: [opensource-gtest] Re: [googletest] messages for success as well as failure?

[stevej]

I'm not getting any output from SUCCEED() << "whatever". I'm using
the version of gtest that comes with the most recent version of gmock
-- is there something newer I need to be using?

Thanks

On Jul 22, 10:37 pm, Steve Jaffe <steve_ja...@yahoo.com> wrote:
> Thanks, using the SUCCEED() macro to do some customized success reporting sounds
> like a reasonable approach.
>
> ________________________________
> From: Vlad Losev <vl...@google.com>
> To: Tim Baverstock <wea...@google.com>
> Cc: Google C++ Testing Framework <googletes...@googlegroups.com>
> Sent: Thu, July 22, 2010 4:51:00 PM
> Subject: Re: [gUnit-users] Re: [opensource-gtest] Re: [googletest] messages for
> success as well as failure?

[stevej]

The reason I don't see SUCCEED() messages is the following in
gtest.cc:

// Called after an assertion failure.
void PrettyUnitTestResultPrinter::OnTestPartResult(
const TestPartResult& result) {
// If the test part succeeded, we don't need to do anything.
if (result.type() == TestPartResult::kSuccess)
return;

[Vlad Losev]

On Mon, Aug 2, 2010 at 10:53 AM, stevej <steve...@yahoo.com> wrote:

The reason I don't see SUCCEED() messages is the following in
gtest.cc:

// Called after an assertion failure.
void PrettyUnitTestResultPrinter::OnTestPartResult(
   const TestPartResult& result) {
 // If the test part succeeded, we don't need to do anything.
 if (result.type() == TestPartResult::kSuccess)
   return;

Oh, yes, the standard console printer ignores the output by SUCCEED(). You'd need to write and hook up your own listener that prints it.

class SuccessPrinter : public EmptyTestEventListener {

 public:

  void OnTestPartResult(const TestPartResult& result) {

    if (result.type() == TestPartResult::kSuccess) {

      PrintTestPartResult(result);

      fflush(stdout);

    }

  }

};