Making EXPECT_DEATH work for optimized code

[Derek]

I'd like to run unit tests both optimized (-O3) and non-optimized (-
O0). EXPECT_DEATH often passes debug code but fails in optimized
code.

Below is some sample code showing the problem. The code also shows
some macros that attempt to wrap EXPECT_DEATH and work around the
problem with 'volatile.' Volatile doesn't seem to be a general
solution. For example, it doesn't work with pointers. There are also
problems with iterators for which I don't yet have a workaround.

Is there a possible improvement to EXPECT_DEATH to generalize the
behavior for optimized code?

dtaylor% cat try.cpp

#include <stdio.h>

#include <vector>

#include <gtest/gtest.h>

#define OPTIMIZED_EXPECT_DEATH(type_, stmt_, str_) {volatile type_
var; EXPECT_DEATH(var = stmt_, str_);}
#define PTR_EXPECT_DEATH(stmt_, str_) {void * var; EXPECT_DEATH(var =
stmt_, str_); printf("%p\n",var);}

template <class Header>
class TestObject {
private:
Header *header_;
int data_;
public:
TestObject(Header *header, int data) :
data_(data), header_(header)
{}

int data() const { return data_; }
void setData(int data) { data_ = data; }

size_t size() const { return header_->size(); }
void * ptr() { return header_->ptr(); }

std::vector<int>::iterator begin() { header_->begin(); }
std::vector<int>::iterator end() { header_->end(); }

Header *header() const { return header_; }
void setHeader(Header *header) { header_ = header; }
};

class MyHeader {
private:
size_t size_;
void *ptr_;
std::vector<int> vec_;
public:
MyHeader() : size_(0), ptr_(NULL) { }
MyHeader(size_t size, void *ptr) : size_(size), ptr_(ptr) { }

size_t size() { return size_; }
void setSize(size_t size) { size_ = size; }

void * ptr() { return ptr_; }
void setPtr(void *ptr) { ptr_ = ptr; }

std::vector<int>::iterator begin() { return vec_.begin(); }
std::vector<int>::iterator end() { return vec_.end(); }
};

typedef TestObject<MyHeader> Object;

TEST(ThisTest, Test1) {
Object obj(NULL, 1);
EXPECT_EQ(1, obj.data());

//EXPECT_DEATH(obj.size(), "");
OPTIMIZED_EXPECT_DEATH(size_t, obj.size(), "");
}

TEST(ThisTest, Test2) {
Object obj(NULL, 1);

//EXPECT_DEATH(obj.ptr(), "");
//OPTIMIZED_EXPECT_DEATH(void *, obj.ptr(), "");
PTR_EXPECT_DEATH(obj.ptr(), "");
}

TEST(ThisTest, Test3) {
Object obj(NULL, 1);

// Workaround?
//volatile std::vector<int>::iterator it;
std::vector<int>::iterator it;
EXPECT_DEATH(it = obj.begin(), "");
}
dtaylor% g++ try.cpp -O3 -lgtest -lgtest_main -o try
dtaylor% ./try
Running main() from gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from ThisTest
[ RUN ] ThisTest.Test1

[WARNING] src/gtest-death-test.cc:882: Death tests use fork(), which
is unsafe particularly in a threaded context. For this test, Google
Test couldn't detect the number of threads.
[ OK ] ThisTest.Test1
[ RUN ] ThisTest.Test2

[WARNING] src/gtest-death-test.cc:882: Death tests use fork(), which
is unsafe particularly in a threaded context. For this test, Google
Test couldn't detect the number of threads.
0x18b79388
[ OK ] ThisTest.Test2
[ RUN ] ThisTest.Test3

[WARNING] src/gtest-death-test.cc:882: Death tests use fork(), which
is unsafe particularly in a threaded context. For this test, Google
Test couldn't detect the number of threads.
try.cpp:77: Failure
Death test: it = obj.begin()
Result: failed to die.
Error msg:
[ FAILED ] ThisTest.Test3
[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran.
[ PASSED ] 2 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ThisTest.Test3

1 FAILED TEST
dtaylor%

[Zhanyong Wan (λx.x x)]

Derek,

This problem is not specific to death tests. Basically you are
testing some expected side effects of your code (in this case, the
side effect is a segfault), but the compiler is over zealous and
optimizes away the side effects, causing the test to fail.

I suggest you rewrite the test such that the compiler cannot optimize
away the side effect even in the optimized mode. One thing you can
try is to make sure the return value of the function is actually used,
e.g.

size_t size;
EXPECT_DEATH(size = obj.size(), "");

(Note that you can write an arbitrary statement inside EXPECT_DEATH,
including a compound statement.)

--
Zhanyong