Gtest crashes on test exit

[Mike Aizatsky]

I've got a test which crashes on exit. Blow is the valgrind dump. Do
you have any idea what might be causing this? As far as I understand,
it tries to delete a global instance of some String. I don't create
such instances though.

[==========] 10 tests from 2 test cases ran. (717 ms total)
[ PASSED ] 10 tests.
==21485== Invalid free() / delete / delete[]
==21485== at 0x4C29362: operator delete[](void*) (valgrind/
coregrind/m_replacemalloc/vg_replace_malloc.c:409)
==21485== by 0x41371A: testing::internal::String::~String() (/usr/
local/include/gtest/internal/gtest-string.h:218)
==21485== by 0x5E9362F: __cxa_finalize (/build/buildd/eglibc-2.11.1/
stdlib/cxa_finalize.c:56)
==21485== by 0x4E395E5: ??? (in /usr/local/lib/libgmock.so.0.0.0)
==21485== by 0x4E42200: ??? (in /usr/local/lib/libgmock.so.0.0.0)
==21485== by 0x5E93261: exit (/build/buildd/eglibc-2.11.1/stdlib/
exit.c:78)
==21485== by 0x5E78C53: (below main) (/build/buildd/eglibc-2.11.1/
csu/libc-start.c:258)
==21485== Address 0x61e53d0 is 0 bytes inside a block of size 8
free'd
==21485== at 0x4C29362: operator delete[](void*) (valgrind/
coregrind/m_replacemalloc/vg_replace_malloc.c:409)
==21485== by 0x41371A: testing::internal::String::~String() (/usr/
local/include/gtest/internal/gtest-string.h:218)
==21485== by 0x5E93261: exit (/build/buildd/eglibc-2.11.1/stdlib/
exit.c:78)
==21485== by 0x5E78C53: (below main) (/build/buildd/eglibc-2.11.1/
csu/libc-start.c:258)
==21485==

[ FATAL ] lib/gtest-1.5.0/include/gtest/internal/gtest-port.h:1114::
pthread_key_delete(key_)failed with error 22

[Vlad Losev]

Mike,

It looks like you are using an installed version of gtest/gmock. Using installed gtest leads to ODR violations so often that we dropped support for installing it. Try building gtest/gmock together with your test code and using the those binaries.

HTH,

Vlad

[Brent Weatherall]

Hi Vlad,

     I am seeing a similar problem, except that I am building GTest/GMock as part of my project.  For me, GTEST_HAS_PTHREAD is getting defined somewhere that I am having trouble tracking down.  I don't want it defined, but it seems to slip through.  My CMakeLists.txt file has the following (before I add the gtest project):

CMakeLists.txt:

...

add_definitions("-DGTEST_HAS_POSIX_RE=0")

set(gtest_disable_pthreads ON CACHE BOOL "" FORCE)

add_definitions("-DGTEST_HAS_PTHREAD=0")

add_definitions("-DGTEST_HAS_RTTI=0")

add_definitions("-D_HAS_EXCEPTIONS=0")

...

include_directories(${GMOCK_ROOT}/include)

include_directories(${GMOCK_ROOT}/gtest/include)

include_directories(${OPENMAX_ROOT})

include_directories(${CMAKE_CURRENT_SOURCE_DIR})

add_subdirectory(${GMOCK_ROOT})

When I look into the cmake generated flags files I see (this is a grep from the CMake generated directory):

gmock-1.6.0]$ find -name flags.make | xargs egrep DGTEST_HAS_PTHREAD

./gtest/CMakeFiles/gtest.dir/flags.make:CXX_DEFINES = -DGTEST_HAS_POSIX_RE=0 -DGTEST_HAS_PTHREAD=0 -DGTEST_HAS_RTTI=0 -DMAJOR_VERSION=0 -DMINOR_VERSION=85 -D_GLIBCXX_USE_NANOSLEEP -D_HAS_EXCEPTIONS=0

./gtest/CMakeFiles/gtest_main.dir/flags.make:CXX_DEFINES = -DGTEST_HAS_POSIX_RE=0 -DGTEST_HAS_PTHREAD=0 -DGTEST_HAS_RTTI=0 -DMAJOR_VERSION=0 -DMINOR_VERSION=85 -D_GLIBCXX_USE_NANOSLEEP -D_HAS_EXCEPTIONS=0

./CMakeFiles/gmock.dir/flags.make:CXX_DEFINES = -DGTEST_HAS_POSIX_RE=0 -DGTEST_HAS_PTHREAD=0 -DGTEST_HAS_RTTI=0 -DMAJOR_VERSION=0 -DMINOR_VERSION=85 -D_GLIBCXX_USE_NANOSLEEP -D_HAS_EXCEPTIONS=0

./CMakeFiles/gmock_main.dir/flags.make:CXX_DEFINES = -DGTEST_HAS_POSIX_RE=0 -DGTEST_HAS_PTHREAD=0 -DGTEST_HAS_RTTI=0 -DMAJOR_VERSION=0 -DMINOR_VERSION=85 -D_GLIBCXX_USE_NANOSLEEP -D_HAS_EXCEPTIONS=0

It is defined as zero in all cases; however it seems that somewhere else the code is still defining GTEST_HAS_PTHREAD = 1.  I am still looking for where this happens, but as far as I can tell the CMake setup is correct.  I am 100% sure of this getting defined.  I added this code:

gtest-port.h:

...

#if GTEST_HAS_PTHREAD

static const bool GTEST_HAS_PTHREAD_GOT_DEFINED = true;

...

Then searching the binary after build:

$ readelf -Ws <BINARY> | c++filt | egrep -i GTEST_HAS_PTHREAD_GOT_DEFINED

    56: 080bc3c0     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

   221: 080bcc7c     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

   326: 080bf670     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

Any thoughts?

For reference about the environment I am working in:

$ uname -r

3.17.7-200.fc20.i686+PAE

gmock-1-6.0

$ gcc --version

gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7)

Let me know what you think.  Thank you.

Best Regards,

Brent

[Corey Kosak]

I'm not sure what's going on here, but one suggestion I have to narrow it down is to induce a compile error rather than a linker symbol at that point in gtest-port.h

In other words, what happens if you try

#if GTEST_HAS_PTHREAD

+++syntax error+++

in gtest-port.h ?

Wouldn't this help you discover the (first) C++ file where GTEST_HAS_PTHREAD had (somehow) gotten set to 1?

--

---
You received this message because you are subscribed to the Google Groups "Google C++ Testing Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to googletestframe...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brent Weatherall]

Corey,

     I think I tracked down where it is going wrong (but still not sure why).  I added one more symbol to gtest-port.h:

#ifndef GTEST_HAS_PTHREAD

// The user didn't tell us explicitly, so we assume pthreads support is

// available on Linux and Mac.

//

// To disable threading support in Google Test, add -DGTEST_HAS_PTHREAD=0

// to your compiler flags.

#if GTEST_OS_LINUX || GTEST_OS_MAC

# define GTEST_HAS_PTHREAD 1

static const bool GTEST_HAS_PTHREAD_DEF_IN_PORT_H = true;

#endif // GTEST_OS_LINUX || GETST_OS_MAC

#endif

This symbol shows up, which seems to indicate that even though the flags.make contains this definition, it somehow isn't detected:

$  readelf -Ws <BINARY> | c++filt | egrep -i "GTEST_HAS_PTHREAD"

    54: 080bc3b7     1 OBJECT  LOCAL  DEFAULT   15 GTEST_HAS_PTHREAD_DEF_IN_PORT_H

    57: 080bc3c0     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

   220: 080bcc73     1 OBJECT  LOCAL  DEFAULT   15 GTEST_HAS_PTHREAD_DEF_IN_PORT_H

   223: 080bcc7c     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

   326: 080bf5e1     1 OBJECT  LOCAL  DEFAULT   15 GTEST_HAS_PTHREAD_DEF_IN_PORT_H

   329: 080bf670     1 OBJECT  LOCAL  DEFAULT   15 testing::internal::GTEST_HAS_PTHREAD_GOT_DEFINED

Per your suggestion, I added a "#error" to the same location (and removed the static symbol):

// Determines whether Google Test can use the pthreads library.

#ifndef GTEST_HAS_PTHREAD

// The user didn't tell us explicitly, so we assume pthreads support is

// available on Linux and Mac.

//

// To disable threading support in Google Test, add -DGTEST_HAS_PTHREAD=0

// to your compiler flags.

#if GTEST_OS_LINUX || GTEST_OS_MAC

# define GTEST_HAS_PTHREAD 1

#error "GTEST_HAS_PTHREAD should already be defined!!!"

#endif // GTEST_OS_LINUX || GETST_OS_MAC

#endif  // GTEST_HAS_PTHREAD

And it does indeed stop the compilation:

gtest-port.h:421:2: error: #error "GTEST_HAS_PTHREAD should already be defined!!!"

 #error "GTEST_HAS_PTHREAD should already be defined!!!"

Maybe this reaches into CMake/make subtleties I am not familiar with?  I would expect if the define is set in the flags.make compiler flags, then it should be defined in time for this file to get compiled.  Regardless of my expectation, it seems to not be defined at the point when gtest-port.h is compiled.

Maybe I am just missing something simple here?  Thank you.

Regards,

Brent

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframework+unsub...@googlegroups.com.

[Brent Weatherall]

Corey,

     I sent my previous post a bit too soon.  I see that it is explicitly undefined:

gtest.h:65:2: error: #error "GTEST_HAS_PTHREAD is getting undefined"

 #error "GTEST_HAS_PTHREAD is getting undefined"

// GTEST_HAS_PTHREAD must be used with the default value given below

#ifdef GTEST_HAS_PTHREAD

#error "GTEST_HAS_PTHREAD is getting undefined"

# undef GTEST_HAS_PTHREAD

#endif

I did not see this #undef when looking at the search results initially. I will look into why it is undefined, but maybe we have an invalid configuration in our build system or something of that nature.  I wouldn't mind building with pthreads if I didn't get a crash on exit (similar to the person that started this thread).

Thank you.

Regards,

Brent

On Monday, January 12, 2015 at 3:19:23 PM UTC-8, Corey Kosak wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframework+unsub...@googlegroups.com.

[Brent Weatherall]

Hi Corey,

     After noticing the hard coded #undef, I changed my include paths as such:

#include_directories(${GMOCK_ROOT}/include)

#include_directories(${GMOCK_ROOT}/gtest/include)

include_directories(${GMOCK_ROOT}/gtest/fused-src)

include_directories(${GMOCK_ROOT}/fused-src)

This allows my program to run without pthreads and without the strange crash when the framework exits:

gtest-port.h:1334:: pthread_key_delete(key_)failed with error 22

Which happens during:

GTestLog::~GTestLog

Forgive my naivety with GTest, but Is the intent that if you want to build without pthreads to use the fused-src includes instead?  Or have I simply worked around my problem in an unintended way?  Thank you again.

Regards,

Brent

On Monday, January 12, 2015 at 3:19:23 PM UTC-8, Corey Kosak wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframework+unsub...@googlegroups.com.

[Corey Kosak]

You should be able to see exactly what compiler invocation failed. (i.e. which .cc file is being compiled and what flags are being passed). If you are stuck, can you post exact repro instructions?

1. Current gtest version (you say you are using 1.6.0. But if this is a bug in 1.6.0 that is not in 1.7.0 we are unlikely to fix it). Can you try it with 1.7.0?

2. What platform you are on

3. Exactly what test file you are trying to compile (as tiny an example as possible please)

4. Exactly what commands you type to configure and build gtest and the example so that your problem comes up

If we can easily reproduce your issue here by following your instructions we should be in a better position to help.

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframe...@googlegroups.com.

[Brent Weatherall]

Corey,

     I ended up changing the testing approach slightly and the pthread error has gone away, but something different has popped up.  If the new problem seems legit I can open a fresh thread.  The original test dynamically loaded a DLL at run time, linked an init function from that DLL that setup function pointers and returned an object with those function pointers in place.  Those functions are the focus of the test.  The normal code runs in a DLL, but after looking closer it turns out I do NOT need to test that way.  The new test is able to simply compile in the functions calls we are testing without the use of a DLL.  This eliminated the pthread crash.

Another strange issue has come up that is not making sense at the moment.  Some of the stuff I am working on is proprietary, so I cannot put a direct copy/paste of files and the build setup and so on.  I will try to provide as much information as I can.

After changing the test to NOT load any DLLs all seemed well.  But now I am finding that using MOCK_CONST_METHOD appears to introduce random failures into the test when used with one specific function.  I can literally run the test binary back-to-back and every run is a different result.  I have a few other const methods that behave correctly.  Sometimes the test passes completely and other times I get a random error message regarding the mocked constant method at different points in the test.  Simply defining this one mock method will cause the error.  The other const methods are not causing problems:

Problematic mock definition:
The method being mocked is in the public section of the class definition (ERROR_TYPE and STATE_TYPE are typedef enums):

    virtual ERROR_TYPE GetState(

        STATE_TYPE* pState) const;

Mock definition:

  MOCK_CONST_METHOD1(GetState,

      ERROR_TYPE(STATE_TYPE* pState));

Example of mock method that does not create problems:

Class definition:

    virtual ERROR_TYPE GetParameter(

        INDEX_TYPE nIndex,

        PTR_TYPE pParameterStructure) const;

Mock definition:

  MOCK_CONST_METHOD2(GetParameter,

      ERROR_TYPE(INDEX_TYPE nIndex, PTR_TYPE pParameterStructure));

There are total of 24 methods mocked.  Some const and some not.

Test Failure Example:

When it fails, it looks like this:

$ ./bin/Test_Entrypoint

[==========] Running 18 tests from 1 test case.

[----------] Global test environment set-up.
...

[ RUN      ] Entrypoint.SendCommand

: Failure

Uninteresting mock function call - returning default value.

    Function call: GetState(0xb77262b0)

    The mock function has no default action set, and its return type has no default value set.

Aborted

I checked thoroughly and SendCommand NEVER calls GetState.

Or it can look like this:
$ ./bin/Test_Entrypoint

[==========] Running 18 tests from 1 test case.

[----------] Global test environment set-up.

...

[ RUN      ]Entrypoint.SendCommand

[       OK ] Entrypoint.SendCommand (0 ms)

[ RUN      ] Entrypoint.GetParameter

: Failure

Uninteresting mock function call - returning default value.

    Function call: GetState(0xb76de2b0)

    The mock function has no default action set, and its return type has no default value set.

[  FAILED  ] Entrypoint.GetParameter (1 ms)

The build setup is nothing special.  CMakeLists.txt adds gmock-1.6.0 to the build and in the test directory of the area being tested my test executable has:

Entrypoint.cpp contains the functions getting unit tested.

add_executable(Test_Entrypoint

    Test_Entrypoint.cpp

    ${ENTRYPOINT_DIR}/Entrypoint.cpp

    <SOURCE FILES OF CLASS BEING MOCKED WITH DEPS TO MAKE LINKER HAPPY>

    )

target_link_libraries(Test_Entrypoint

    ${GMOCK_LIBRARY}

    ${GTEST_LIBRARY}

    ${OS_LIBRARIES}

    ${CMAKE_THREAD_LIBS_INIT}

    )

I don't see anything different being done with the particular const method that keeps killing the test, but obviously something is causing it to get called internally in GMock.  I put in a print at the ONLY place the method gets called in the entrypoint code and the print does NOT show up when this error occurs.

Entrypoint.cpp when GetState is called:
static ERROR_TYPE GetState(

    HANDLE_TYPE hHandle,

    STATE_TYPE* pState){

    if ( hHandle == NULL) return ERROR_NULL;

    printf("**********************static ERROR_TYPE GetState( called.\n");

    return GetInstance(hHandle)->GetState( pState);

}

GetInstance(hHandle)->GetState <-- this is the method call being mocked and is ONLY called from this one Entrypoint function.  

I am running gtest-1.6.0, Fedora 20, CMake 2.8.12.2, gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7).  If you need more specific details let me know and I will sanitize and provide them to the extent possible.  Thank you for taking a look at this.  I am bit stumped on what I have done wrong to cause this problem. If needed, I can try to re-create the issue in something that I am able to share without modification.  Attached is a copy of the flags.make and the link.txt generated from CMake (with IP sanitized) if those help at all.

Thank you.

Regards,

Brent

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframework+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Brent Weatherall]

Corey,

     I was finally able to get the test to fail in gdb and get a backtrace. As it turns out, there is an unexpected base class thread running that is calling this function and failing the test.  I think it is safe to disregard the previous message.

[Corey Kosak]

Well that's great news! We're glad you were able to solve the problem!

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframe...@googlegroups.com.

[Brent Weatherall]

Yes, I was happy to figure out the problem.  Just to clarify, is it a known limitation of GTest that dynamically loading a DLL at run-time will cause problems with the threaded version of GTest?  And is it the case that if one is building GTest on Linux and wants to use the non-threaded version of GTest, that the includes need to be changed to fused-src instead of the normal includes?  It seems that the normal includes purposely undef the pthread define and then on Linux and Mac platforms auto-define pthreads to true.  This is just for my understanding.  Thanks!

Regards,

Brent

[Corey Kosak]

It is not a known problem so far as I know. Without doing any investigation into the issue, I would guess that you accidentally built the library with different compiler flags than your main executable, which is always a potential problem with C++

I'm not the most knowledgeable on these issues however, nor am I familiar with your fused-src question (sorry).

If you can help out and confirm that we have a problem at the repository's head (not at release 1.6.0 or even 1.7.0) and can provide a hermetic little testcase that proves it, then I think we'd be able to find some time to fix it.

To unsubscribe from this group and stop receiving emails from it, send an email to googletestframe...@googlegroups.com.

[Brent Weatherall]

OK, that sounds good.  I will try to get some time and cobble together a test case that emulates the problem.  Since some of the compiler flags are undone by the main include it could be that the options did not match up well.  Will get back to you when I have some test code up.