Problem with integrating google mock with other testing framework
Lior Friedman
Hi,
I’m trying to use GoogleMock with a different testing framework and I’m following the guide and using the:
testing::GTEST_FLAG(throw_on_failure) = true;
however, when a failure is caused by mismatching arguments, the entire run crashes.
When looking into the code, what seems to be happening is that during the argument matching phase an exception is being thrown,
However this terminate the test and causes the mock verification code to execute which results in another exception being thrown.
Which at the end leads to a complete crash,
Does someone knows of a good way to fix this?
Blog - http://imistaken.blogspot.com
Zhanyong Wan (λx.x x)
On Wed, Feb 3, 2010 at 7:03 AM, Lior Friedman <lfri...@gmail.com> wrote:
> Hi,
>
> I’m trying to use GoogleMock with a different testing framework and I’m
> following the guide and using the:
>
> testing::GTEST_FLAG(throw_on_failure) = true;
>
> however, when a failure is caused by mismatching arguments, the entire run
> crashes.
Which testing framework are you using? Can you configure it to catch
exceptions thrown by tests and report them as test failures?
> When looking into the code, what seems to be happening is that during the
> argument matching phase an exception is being thrown,
>
> However this terminate the test and causes the mock verification code to
> execute which results in another exception being thrown.
>
> Which at the end leads to a complete crash,
>
> Does someone knows of a good way to fix this?
You have several choices:
1) use Google Test,
2) configure your testing framework to catch exceptions, or
3) write an Google Test event listener to report the test failure
properly to your testing framework:
http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Extending_Google_Test_by_Handling_Test_Events
1) would be the easiest and 3) the hardest. What's the reason for not
using Google Test? Thanks,
> Lior Friedman
>
> Blog - http://imistaken.blogspot.com
>
>
>
>
--
Zhanyong
Zhanyong Wan (λx.x x)
the discussion)
2010/2/3 Lior Friedman <lfri...@gmail.com>:
> Hi Zhanyong,
>
>>Which testing framework are you using? Can you configure it to catch
>>exceptions thrown by tests and report them as test failures?
>
> Were using a small framework called Yaffut. Which does exactly that.
> The problem I'm seeing is that that the failure happens before the actual exception is caught by that framework.
>
>> You have several choices:
>
>>1) use Google Test,
>>2) configure your testing framework to catch exceptions, or
>>3) write an Google Test event listener to report the test failure
>>properly to your testing framework:
>>http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Extending_Google_Test_by_Handling_Test_Events
>
> 2 is good for me in fact that's what should happen. However something goes wrong.
> From what I saw the failure goes along these lines:
> we set up an expectation to fail if called with wrong argument.
> When that happens:
> 1) an exception is thrown
> 2) This leads to the destructor of the mock to be executed,
> 3) which leads to another exception to be thrown,
> 4) which completely crashes the process.
>
> I think I can demonstrate it without any relation to the testing framework in use if that can help.
I see. So the problem is that gmock throws an exception in a
destructor. That makes 2) a bad option. You are left with 1) and 3).
>>1) would be the easiest and 3) the hardest. What's the reason for not
>>using Google Test? Thanks,
>
> The tests will eventually need to run under VxWork. Trying to port Google Test is a substantial effort I hope to avoid.
gtest is actually a part of gmock. If you have the latter working,
the former already works. ;)
--
Zhanyong
Lior Friedman
> the discussion)
> I see. So the problem is that gmock throws an exception in a
> destructor.
Wouldn’t you describe this as a bug with the behavior of :
testing::GTEST_FLAG(throw_on_failure)?
I didn’t actually try but I'm under the impression that this will repeat itself even if I use google test.
And there is no actual logic of mine involved in that.
> That makes 2) a bad option. You are left with 1) and 3).
So what you are saying is that in fact google mock can't be used with any other testing framework other then google test??? (unless I go the hard 3rd way)
If that is the case I would recommend updating the documentation to reflect that.
> gtest is actually a part of gmock. If you have the latter working,
> the former already works. ;)
Actually I don’t have it working.
I was hoping that google mock can be largely decoupled from google test.
I see that is not the case.
I'm guessing we will need to reflect on where to go from here.
Thank you for your help and feedback.
If you happen to think of anything else let me know
Zhanyong Wan (λx.x x)
>> I see. So the problem is that gmock throws an exception in a
>> destructor.
> Wouldn’t you describe this as a bug with the behavior of :
> testing::GTEST_FLAG(throw_on_failure)?
>
> I didn’t actually try but I'm under the impression that this will repeat itself even if I use google test.
> And there is no actual logic of mine involved in that.
This is the expected behavior (i.e. not a bug per se).
This flag tells gtest to throw an exception on failure, which it
faithfully does.
The disconnect is in the expectation that the exception will be caught
by the testing framework. It is not always technically possible. The
test program may crash as a result of an exception thrown in a
destructor - this is how C++ is designed.
We should do a better job at describing what we mean by "gmock can
work with any testing framework if you set this flag". The statement
is true in the sense that the test will fail (either gracefully or
crashing) when there's an error. I'll see how I can clarify it.
>> That makes 2) a bad option. You are left with 1) and 3).
>
> So what you are saying is that in fact google mock can't be used with any other testing framework other then google test??? (unless I go the hard 3rd way)
You can, although you may not get the full benefit.
And the 3rd way isn't much work really.
> If that is the case I would recommend updating the documentation to reflect that.
I will. Thanks for the suggestion.
>> gtest is actually a part of gmock. If you have the latter working,
>> the former already works. ;)
> Actually I don’t have it working.
But you were able to compile it and run the test and observe the
crash? Could you clarify what you mean by "don't have it working"?
Thanks,
> I was hoping that google mock can be largely decoupled from google test.
> I see that is not the case.
>
> I'm guessing we will need to reflect on where to go from here.
>
> Thank you for your help and feedback.
> If you happen to think of anything else let me know
>
> Lior Friedman
> Blog - http://imistaken.blogspot.com
--
Zhanyong