GTest Death Test spawning multiple processes.

[Anurag Jain]

Hello there,

I'm trying to setup a unit testing infrastructure for my company's codebase and have chosen GTest for this purpose.
I'm not able to get proper understanding of how the death tests are run. From the documentation I could find that Death tests are run in a new process.

I've the following code just to test how various death test macros work.
  
       TEST(DataObfuscatorDeathTest, ObfuscatorAllNullParamDeathTest)
       {
             EXPECT_EXIT(exit(-1), ::testing::ExitedWithCode(0), "");
       }

I'm running this on windows 7 with Visual studio 2013. My test project is a MFC project. Below are my questions.

1: How many processes are created when a death test is run ? In my case I see around 30 processes being created with following output.

 [==========] Running 2 tests from 1 test case.
1>  [----------] Global test environment set-up.
1>  [----------] 2 tests from DataObfuscatorDeathTest
1>  [ RUN      ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1>  [==========] Running 2 tests from 1 test case.
1>  [----------] Global test environment set-up.
1>  [----------] 2 tests from DataObfuscatorDeathTest
1>  [ RUN      ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1>  [==========] Running 2 tests from 1 test case.
1>  [----------] Global test environment set-up.
1>  [----------] 2 tests from DataObfuscatorDeathTest
1>  [ RUN      ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1>  [==========] Running 2 tests from 1 test case.
1>  [----------] Global test environment set-up.
1>  [----------] 2 tests from DataObfuscatorDeathTest
1>  [ RUN      ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1>  [==========] Running 2 tests from 1 test case.
....
.....
......

[==========] Running 1 test from 1 test case.
1>  [----------] Global test environment set-up.
1>  [----------] 1 test from DataObfuscatorDeathTest
1>  [ RUN      ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1>d:\workspace\anujain_kti_rpo_elise9850_mui_str\daytona\tests\obfuscatorutilitytest\DataObfuscatorTest.h(101): error : Death test: exit(-1)
1>      Result: died but not with expected exit code:
1>              Exited with exit status 3
1>  Actual msg:
1>  [  DEATH   ]
1>  [  FAILED  ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest (106 ms)
1>  [----------] 1 test from DataObfuscatorDeathTest (106 ms total)
1> 
1>  [----------] Global test environment tear-down
1>  [==========] 1 test from 1 test case ran. (106 ms total)
1>  [  PASSED  ] 0 tests.
1>  [  FAILED  ] 1 test, listed below:
1>  [  FAILED  ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest
1> 
1>   1 FAILED TEST
1>    YOU HAVE 9 DISABLED TESTS
1> 
1>  [       OK ] DataObfuscatorDeathTest.ObfuscatorAllNullParamDeathTest (209 ms)
1>  [----------] 1 test from DataObfuscatorDeathTest (209 ms total)
1> 
1>  [----------] Global test environment tear-down
1>  [==========] 1 test from 1 test case ran. (210 ms total)
1>  [  PASSED  ] 1 test.
1> 
1>    YOU HAVE 9 DISABLED TESTS

2: Why the test previously being reported as fail at last is being reported as pass.

3: Is there a way to limit the number of processes death test creates?

Kindly provide your inputs and help me understand the working of Gtest.


Thanks and regards,
Anurag Jain

[Anurag Jain]

Gtest version 1.7
Windows 7 64bit
VS2013

[Josh Kelley]

I've never done any serious work death tests myself, but from what I understand, each death test should spawn a single copy of the executable that executes the test case then exits. It passes some internal flags in order to tell the spawned copy to execute an individual death test case instead of running the normal test suite.

What does your main() look like? The fact that your code (seemingly) keeps running your regular test suite makes me wonder if the internal flags used by the death test implementation are getting lost.

--

Josh Kelley

On Mon, Apr 6, 2015 at 8:43 AM, Anurag Jain <anurag...@gmail.com> wrote:

1: How many processes are created when a death test is run ? In my case I see around 30 processes being created with following output.

[Anurag Jain]

Hey Josh,

Thanks for the prompt reply. My production code base has a few portions in MFC and thus I was forced to make my test project also an MFC one. So the app's InitiInstance() method looks as below.

BOOL CObfuscatorUtilityTestApp::InitInstance()
{
    CWinApp::InitInstance();


    EnableTaskbarInteraction(FALSE);

    // AfxInitRichEdit2() is required to use RichEdit control   
    // AfxInitRichEdit2();

    // Standard initialization
    // If you are not using these features and wish to reduce the size
    // of your final executable, you should remove from the following
    // the specific initialization routines you do not need
    // Change the registry key under which our settings are stored
    // TODO: You should modify this string to be something appropriate
    // such as the name of your company or organization
    SetRegistryKey(_T("Local AppWizard-Generated Applications"));


    // To create the main window, this code creates a new frame window
    // object and then sets it as the application's main window object
    CMainFrame* pFrame = new CMainFrame;
    if (!pFrame)
        return FALSE;
    m_pMainWnd = pFrame;
    // create and load the frame with its resources
    pFrame->LoadFrame(IDR_MAINFRAME,
        WS_OVERLAPPEDWINDOW | FWS_ADDTOTITLE, NULL,
        NULL);

    //Initializing Google Test Framework
    int argc(0);

    testing::InitGoogleMock(&argc, (wchar_t **)0);
    int run_status = RUN_ALL_TESTS();

    // The one and only window has been initialized, so show and update it
    pFrame->ShowWindow(SW_SHOW);
    pFrame->UpdateWindow();
    return false;
}

[Josh Kelley]

This is your problem:

    //Initializing Google Test Framework
    int argc(0);

    testing::InitGoogleMock(&argc, (wchar_t **)0);
    int run_status = RUN_ALL_TESTS();

Without seeing the actual command line arguments (including the internal argument that it adds for the death test), Google Test doesn't know to execute the death test instead of restarting the whole suite.

You'll need to find a way to get the actual argc and argv here and pass those instead.

--

Josh Kelley

On Tue, Apr 7, 2015 at 1:43 AM, Anurag Jain <anurag...@gmail.com> wrote:

Hey Josh,

Thanks for the prompt reply. My production code base has a few portions in MFC and thus I was forced to make my test project also an MFC one. So the app's InitiInstance() method looks as below.

On Tuesday, 7 April 2015 06:26:50 UTC+5:30, Josh Kelley wrote:

[Anurag Jain]

Thanks Josh, That suggestion worked. Now this is how I'm initiating Gtest in my InitInstance() of MFC application.

//Parsing commandline arguemnts. Necessary for death tests.
    LPWSTR *szArgList;
    int nArgs;

    szArgList = CommandLineToArgvW(GetCommandLineW(), &nArgs);
    if (NULL == szArgList)
    {
        nArgs = 0;
        szArgList = NULL;
    }

    //Initializing Google Test Framework
    testing::InitGoogleMock(&nArgs, szArgList);
    int run_status = RUN_ALL_TESTS();

Someone should add this to the 10 step guide for using GTest with MFC posted somewhere in this forum.

-Anurag Jain