fatal failure in SetUpTestCase() function should skip running tests.

[Prashant]

I think googletest should skip running tests (or automatically mark as
failed) in a particular fixture if we call SetUpTestCase() function
and there is fatal failure in it. Suppose we create some shared object
in SetUpTestCase() function and try to access that in TEST_F. But the
failure to create that object in SetUpTestCase() cause to access NULL
inside TEST_F.

So if we make fatal assertion in SetUpTestCase() itself, then above
problem can be avoided.

I read that if we use Global Set-Up, then it runs the tests if there
was no fatal failures in SetUp(). So is it possible to add similar
functionality in fixture level SetUpTestCase().

Thanks,
Prashant

[Vlad Losev]

Hmm, Our design used not to support per test case set-up failure detection. But now it is rather easy to implement using the event listener API.

One obvious downside of such a change that it changes the semantics of already written tests that have failing assertions inside their test case set up routines.

Does anyone have an opinion on this?

- Vlad

[Lenny Primak]

I think it's logical that if the setup fails,
the tests should fail as well.
In Java, you should get a NPE,
but in C++ the failure will probably coredump
or something sinister, thus justifying this.

On Dec 17, 6:45 am, Vlad Losev <vl...@google.com> wrote:
> Hmm, Our design used not to support per test case set-up failure detection.
> But now it is rather easy to implement using the event listener API.
>
> One obvious downside of such a change that it changes the semantics of
> already written tests that have failing assertions inside their test case
> set up routines.
>
> Does anyone have an opinion on this?
>
> - Vlad
>

[Zhanyong Wan (λx.x x)]

On Thu, Dec 17, 2009 at 3:45 AM, Vlad Losev <vl...@google.com> wrote:
> Hmm, Our design used not to support per test case set-up failure detection.
> But now it is rather easy to implement using the event listener API.
>
> One obvious downside of such a change that it changes the semantics of
> already written tests that have failing assertions inside their test case
> set up routines.

If a program has a failing SetUpTestCase(), it will return non-zero
and be considered failed. The proposed change won't change the test
result in such a case.

>
> Does anyone have an opinion on this?
>
> - Vlad
>
> On Thu, Dec 17, 2009 at 9:15 AM, Prashant <pvsh...@gmail.com> wrote:
>>
>> I think googletest should skip running tests (or automatically mark as
>> failed) in a particular fixture if we call SetUpTestCase() function
>> and there is fatal failure in it. Suppose we create some shared object
>> in SetUpTestCase() function and try to access that in TEST_F. But the
>> failure to create that object in SetUpTestCase() cause to access NULL
>> inside TEST_F.
>>
>> So if we make fatal assertion in SetUpTestCase() itself, then above
>> problem can be avoided.
>>
>> I read that if we use Global Set-Up, then it runs the tests if there
>> was no fatal failures in SetUp(). So is it possible to add similar
>> functionality in fixture level SetUpTestCase().
>>
>> Thanks,
>> Prashant
>
>

--
Zhanyong

[Prashant]

Hi Vlad / Zhanyong,

Is there any decision made on this?

Thanks,
Prashant

On Dec 17 2009, 9:24 pm, Zhanyong Wan (λx.x x) <w...@google.com>
wrote:

> On Thu, Dec 17, 2009 at 3:45 AM, Vlad Losev <vl...@google.com> wrote:
> > Hmm, Our design used not to support per test case set-up failure detection.
> > But now it is rather easy to implement using the event listener API.
>
> > One obvious downside of such a change that it changes the semantics of
> > already written tests that have failing assertions inside their test case
> > set up routines.
>
> If a program has a failing SetUpTestCase(), it will return non-zero
> and be considered failed.  The proposed change won't change the test
> result in such a case.
>
>
>
>
>
> > Does anyone have an opinion on this?
>
> > - Vlad
>

[Zhanyong Wan (λx.x x)]

Hi, Prashant,

I think we should do it. Could you create an issue to track it? Thanks!

--
Zhanyong

[Prashant]

Thanks Zhanyong. I have created new issue #247.
http://code.google.com/p/googletest/issues/detail?id=247