Bug in MFC 7.x: Runtime Class Information Corrupted

[Winfried Peter]

Starting with MFC 7.0 Microsoft has introduced a severe bug for extension
DLLs: In the header file AFXDLLX.H that is needed for all extension DLLs
somebody replaced the external variable _pRawDllMain by a wrong one called
_pAtlChainRawDllMain (see line 44 of AFXDLLX.H). Below is the relevant part
of this header file in its original version, the faulty line is marked with
***>:

////////////////////////////////////////////////////////////////////////////
/
// RawDllMain that saves current app class list and factory list

extern "C" BOOL WINAPI ExtRawDllMain(HINSTANCE, DWORD dwReason, LPVOID);
***> extern "C" BOOL (WINAPI* _pAtlChainRawDllMain)(HINSTANCE, DWORD,
LPVOID) = &ExtRawDllMain;

extern "C"
BOOL WINAPI ExtRawDllMain(HINSTANCE, DWORD dwReason, LPVOID)
{
if (dwReason == DLL_PROCESS_ATTACH)
{
// save critical data pointers before running the constructors
AFX_MODULE_STATE* pModuleState = AfxGetModuleState();
pModuleState->m_pClassInit = pModuleState->m_classList;
pModuleState->m_pFactoryInit = pModuleState->m_factoryList;
pModuleState->m_classList.m_pHead = NULL;
pModuleState->m_factoryList.m_pHead = NULL;
}
return TRUE; // ok
}

This faulty variable is not used anywhere in MFC, ATL and the C runtime
library, it is clearly a left-over of some intermediate internal version of
MFC /ATL where some changes in the DLL initialization mechanism were
planned, but later on have been changed again. MFC 6.0 was using
_pRawDllMain which ensures that the function ExtRawDllMain() is called at
the very beginning of the DLL loading process. The correct behavior (as of
MFC 6.0) is as follows:
1. The code first executed when a DLL is loaded is the DLL entry point
called _DllMainCrtStartup() (to be found in CRTDLL.C). In case of process
attach (which is obviously the case then first loading a DLL) this function
first calls the function that _pRawDllMain points to (if not NULL), then
calls _CRT_INIT() which itself causes all global variables to be initialized
and finally calls the real DllMain() function to finish the intialization.
2. _pRawDllMain points to ExtRawDllMain() which saves the current state of
the runtime class and OLE factory information (See listing above).
3. During execution of _CRT_INIT() all global variables are initialized. One
part of this is that any class being declared as serializable (by using the
DECLARE_SERIAL and IMPLEMENT_SERIAL macros) has created a global variable of
type AFX_CLASSINIT, whose constructor calls a function named AfxClassInit().
This function adds the runtime class information for the associated class to
the runtime class list. So when _CRT_INIT() processing is finished, all
runtime class information of the extension DLL just being loaded has been
entered into the runtime class list of the module state of the hosting
process.
4. The implementation of DllMain() of an MFC extension DLL calls a function
named AfxInitExtensionModule(). This function then takes the runtime class
information form the module state of the hosting process and restores the
runtime class information of the latter by setting the head pointer back to
the value saved in ExtRawDllMain(). The code below shows the corresponding
part of AfxInitExtensionModule():

// save the start of the runtime class list
AFX_MODULE_STATE* pModuleState = AfxGetModuleState();
state.pFirstSharedClass = pModuleState->m_classList.GetHead();
pModuleState->m_classList.m_pHead = pModuleState->m_pClassInit;

The new buggy behavior is caused because _pRawDllMain is no longer
initialized to point to ExtRawDllMain(), so the latter is never called and
therefore the necessary preparation of the runtime class and OLE factory
lists cannot take place. So when trying to restore the original state of the
runtime class (and OLE factory) information as outlined in item 4 of above,
that information is corrupted because the initial values have not been
stored before.
This in turn causes a problem for any regular DLL using MFC as shared DLL
and loading additional extension DLLs (not already loaded by the hosting
process) itself, e.g. any ActiveX control. This loading of additional
extension DLLs corrupts the runtime class information of the module state of
the hosting process, i.e. sets the head pointer of the runtime class
information list (pModuleState->m_classList.m_pHead) to NULL, because
pModuleState->m_pClassInit has not been properly initialized. As soon as the
runtime class information of the hosting process is corrupted, it is no
longer possible load or store any document using the MFC serialization
mechanism.

How to Fix the Bug
==============
The only thing that needs to be done is replace the faulty variable name
_pAtlChainRawDllMain by the correct one (_pRawDllMain). Fortunately this
change can be done on any system without the need to recompile MFC itself,
because AFXDLLX.H is only used for user-built extension DLLs, not for MFC
itself.
The new version of the affected line of code reads like this:

extern "C" BOOL (WINAPI* _pRawDllMain)(HINSTANCE, DWORD, LPVOID) =
&ExtRawDllMain;

Conclusion
========
This is a severe bug, because anybody not aware of this problem can create
components (e.g. an ActiveX control) that cause any hosting application to
be broken. If this is a component vendor, he might ship versions of his
component(s) that break any hosting application (as long as it is written in
MFC) without even knowing it and also without any counter measure a
developer of the hosting application could take. So it would be just
impossible to use these components. And I am quite sure that even finding
the cause for this problem would be very difficult for the developer of the
hosting application - I myself was spending several hours of debugging MFC
code where all components involved were written by me or my colleagues. So I
had full knowledge and source code access for them. If this is not case,
this can be more than a nightmare.

-------------------------------------
Please note that my e-mail address is faked to prevent spam. If you want to
reply just remove "we.hate.spam." from the domain name.

[David Lowndes]

FWIW, I'll try to raise this with MS.

Dave
--
MVP VC++ FAQ: http://www.mvps.org/vcfaq

[Ole Wiig]

First, many thanks to Winfried for his excellent description and how to
correct the bug. A company I am working for has raised the issue with
Microsoft Support and they confirm that it is a bug since MFC7.0, but that
no hotfix yet exists. They also point out another way to work around the
problem;

Add the following lines to the extension dll's main .cpp file after the
#include for afxdllx.h:

extern "C" BOOL WINAPI ExtRawDllMain(HINSTANCE, DWORD dwReason, LPVOID);

extern "C" BOOL (WINAPI* _pRawDllMain)(HINSTANCE, DWORD, LPVOID) =
&ExtRawDllMain;

Since we have a lot of projects and also compiles some third party extension
DLLs, we stick with patching the afxdllx.h on our build machine.

Ole Wiig

"Winfried Peter" <winfrie...@omicron.we.hate.spam.at> wrote in message
news:eaqB3H1I...@TK2MSFTNGP11.phx.gbl...

[Martyn Lovell [MSFT]]

Winfried,

>Starting with MFC 7.0 Microsoft has introduced a severe bug for extension
>DLLs: In the header file AFXDLLX.H that is needed for all extension DLLs
>somebody replaced the external variable _pRawDllMain by a wrong one called
>_pAtlChainRawDllMain (see line 44 of AFXDLLX.H). Below is the relevant part
>of this header file in its original version, the faulty line is marked with
>***>:

Thanks for the report. This does appear to be a bug in MFC 7.0 and 7.1.
We'll be looking to fix it in a future relase.

Martyn Lovell
Development Lead - Libraries
Visual C++ Team
This posting is provided AS IS with no warranties, and confers no rights.