dbghelp 6.12.2.633 not backwards compatible

[rogero]

Warning: I have experienced problems using dbghelp 6.12.2.633 with
code compiled using previous versions of the dbghelp header.

One such problematic piece of code is:

IMAGEHLP_MODULE64 ModuleInfo;
ModuleInfo.SizeOfStruct = sizeof ModuleInfo;
result = SymGetModuleInfo64( hProcess, baseAddress,
&ModuleInfo );

result is failure when compiled using older dbghelp.h, success with
newer.
It appears that the size of IMAGEHLP_MODULE64 has changed in this
release but that, unlike previous changes, the old sized structures
are not correctly processed.

Regards,
Roger.

[Jochen Kalmbach [MVP]]

Hi rogero!

> It appears that the size of IMAGEHLP_MODULE64 has changed in this
> release but that, unlike previous changes, the old sized structures
> are not correctly processed.

It seems that this is a "normal" bug in dbghelp.dll...
Several years ago, it had the same bug...

--
Greetings
Jochen

My blog about Win32 and .NET
http://blog.kalmbachnet.de/

[pat styles [microosft]]

Hello Roger.

The current dbghelp.dll is designed to work with previous code built using
previous headers and libs. Many people get in trouble with mismatched
headers and libs that may leave the appearance of a bug in dbghelp.dll. In
my experience, there has not been a bug in that code in a long time.
However there was one once. That said, I believe there has been a change to
the IMAGEHLP_MODULE64 structure in the latest version. If you have a
specific scenario in which does not appear to work, could you please provide
more detail so that it can be reproduced?

What version of the debugger is the exact source of the dbghelp.h that you
used?
What is the source of dbghelp.lib and what is its file size and date?
What is the code you are executing and where is the failure?

I will make sure this is looked at promptly.

thanks,
.pat styles [microsoft]

"rogero" <roge...@gmail.com> wrote in message
news:d5bf337f-bf0b-4fc3...@g28g2000yqh.googlegroups.com...

[rogero]

Hello Pat,

> The current dbghelp.dll is designed to work with previous code built using
> previous headers and libs.

Yes - and usually it does that extremely well.

...

> That said, I believe there has been a change to
> the IMAGEHLP_MODULE64 structure in the latest version.

Correct - two fields have been added since the last version I've
download,
which was 6.11.1.404

>  If you have a
> specific scenario in which does not appear to work, could you please provide
> more detail so that it can be reproduced?

The one I looked at in most detail was the one I showed in my first
posting:

> IMAGEHLP_MODULE64 ModuleInfo;
> ModuleInfo.SizeOfStruct = sizeof ModuleInfo;
> result = SymGetModuleInfo64( hProcess, baseAddress,
> &ModuleInfo );

This was the third call to dbghelp after SymInitialize() and
SymGetModuleBase64().
Step 1: re-link the code: SymGetModuleInfo64 fails (GetLastError() ==
87)
Step 2: re-compile the code: SymGetModuleInfo64 works

I can't provide more information from here as I haven't got any way to
get access to the latest dbghelp (as per the "New WinDbg requires
entire WDK?" thread...).
I hope to provide a complete example tonight, time permitting.

Regards,
Roger.

[pat styles [microosft]]

Hello Roger.

I look forward to viewing and running your scenario when you get a chance to
send the specifics to me. If a bug was added to the dbghelp code, we want
to get it cleaned up right away.

.p

"rogero" <roge...@gmail.com> wrote in message

news:20e43377-e6b2-47d9...@y17g2000yqd.googlegroups.com...

[Jochen Kalmbach [MVP]]

Hi pat!

> I look forward to viewing and running your scenario when you get a chance to
> send the specifics to me. If a bug was added to the dbghelp code, we want
> to get it cleaned up right away.

From my test, it seems that it does not overwrite memory, but it always
returns "FALSE" if you pass the previous struct.

This is defintiv a show-stopper, because most apps are currently using
the version with size 0x688 (V3).

The new (V4) has a size of 0x690.

Here is a small repro-Code. The ourput is:
V1 is working...
V2 is working...
V3 is NOT working...
V4 is working...

Tested on:
Windows 7 x64
Application:
Running as x64

// dbghelp_bug_6.12.2.633.cpp

#undef UNICODE
#undef _UNICODE

#include <windows.h>
#include <stdio.h>
#include <Tlhelp32.h>
#include <assert.h>
#include <dbghelp.h>
#include <tchar.h>

#pragma pack(push,8) // be sure it has the right packing! THIS IS
MISSING IN DBGHELP.H!!!!

typedef struct IMAGEHLP_MODULE64_V1 {
DWORD SizeOfStruct; // set to sizeof(IMAGEHLP_MODULE64)
DWORD64 BaseOfImage; // base load address of module
DWORD ImageSize; // virtual size of the loaded module
DWORD TimeDateStamp; // date/time stamp from pe header
DWORD CheckSum; // checksum from the pe header
DWORD NumSyms; // number of symbols in the symbol table
SYM_TYPE SymType; // type of symbols loaded
CHAR ModuleName[32]; // module name
CHAR ImageName[256]; // image name
CHAR LoadedImageName[256]; // symbol file name
} IMAGEHLP_MODULE64_V1;

typedef struct IMAGEHLP_MODULE64_V2 {
DWORD SizeOfStruct; // set to sizeof(IMAGEHLP_MODULE64)
DWORD64 BaseOfImage; // base load address of module
DWORD ImageSize; // virtual size of the loaded module
DWORD TimeDateStamp; // date/time stamp from pe header
DWORD CheckSum; // checksum from the pe header
DWORD NumSyms; // number of symbols in the symbol
table
SYM_TYPE SymType; // type of symbols loaded
CHAR ModuleName[32]; // module name
CHAR ImageName[256]; // image name
CHAR LoadedImageName[256]; // symbol file name
// new elements: 07-Jun-2002
CHAR LoadedPdbName[256]; // pdb file name
DWORD CVSig; // Signature of the CV record in
the debug directories
CHAR CVData[MAX_PATH * 3]; // Contents of the CV record
DWORD PdbSig; // Signature of PDB
GUID PdbSig70; // Signature of PDB (VC 7 and up)
DWORD PdbAge; // DBI age of pdb
BOOL PdbUnmatched; // loaded an unmatched pdb
BOOL DbgUnmatched; // loaded an unmatched dbg
BOOL LineNumbers; // we have line number information
BOOL GlobalSymbols; // we have internal symbol information
BOOL TypeInfo; // we have type information
} IMAGEHLP_MODULE64_V2;

typedef struct IMAGEHLP_MODULE64_V3 {
DWORD SizeOfStruct; // set to sizeof(IMAGEHLP_MODULE64)
DWORD64 BaseOfImage; // base load address of module
DWORD ImageSize; // virtual size of the loaded module
DWORD TimeDateStamp; // date/time stamp from pe header
DWORD CheckSum; // checksum from the pe header
DWORD NumSyms; // number of symbols in the symbol
table
SYM_TYPE SymType; // type of symbols loaded
CHAR ModuleName[32]; // module name
CHAR ImageName[256]; // image name
CHAR LoadedImageName[256]; // symbol file name
// new elements: 07-Jun-2002
CHAR LoadedPdbName[256]; // pdb file name
DWORD CVSig; // Signature of the CV record in
the debug directories
CHAR CVData[MAX_PATH * 3]; // Contents of the CV record
DWORD PdbSig; // Signature of PDB
GUID PdbSig70; // Signature of PDB (VC 7 and up)
DWORD PdbAge; // DBI age of pdb
BOOL PdbUnmatched; // loaded an unmatched pdb
BOOL DbgUnmatched; // loaded an unmatched dbg
BOOL LineNumbers; // we have line number information
BOOL GlobalSymbols; // we have internal symbol information
BOOL TypeInfo; // we have type information
// new elements: 17-Dec-2003
BOOL SourceIndexed; // pdb supports source server
BOOL Publics; // contains public symbols
} IMAGEHLP_MODULE64_V3;

typedef struct IMAGEHLP_MODULE64_V4 {
DWORD SizeOfStruct; // set to sizeof(IMAGEHLP_MODULE64)
DWORD64 BaseOfImage; // base load address of module
DWORD ImageSize; // virtual size of the loaded module
DWORD TimeDateStamp; // date/time stamp from pe header
DWORD CheckSum; // checksum from the pe header
DWORD NumSyms; // number of symbols in the symbol
table
SYM_TYPE SymType; // type of symbols loaded
CHAR ModuleName[32]; // module name
CHAR ImageName[256]; // image name
// new elements: 07-Jun-2002
CHAR LoadedImageName[256]; // symbol file name
CHAR LoadedPdbName[256]; // pdb file name
DWORD CVSig; // Signature of the CV record in
the debug directories
CHAR CVData[MAX_PATH * 3]; // Contents of the CV record
DWORD PdbSig; // Signature of PDB
GUID PdbSig70; // Signature of PDB (VC 7 and up)
DWORD PdbAge; // DBI age of pdb
BOOL PdbUnmatched; // loaded an unmatched pdb
BOOL DbgUnmatched; // loaded an unmatched dbg
BOOL LineNumbers; // we have line number information
BOOL GlobalSymbols; // we have internal symbol information
BOOL TypeInfo; // we have type information
// new elements: 17-Dec-2003
BOOL SourceIndexed; // pdb supports source server
BOOL Publics; // contains public symbols
// new element: 15-Jul-2009
DWORD MachineType; // IMAGE_FILE_MACHINE_XXX from
ntimage.h and winnt.h
DWORD Reserved; // Padding - don't remove.
} IMAGEHLP_MODULE64_V4;
#pragma pack(pop)

// SymInitialize()
typedef BOOL (__stdcall *tSI)( IN HANDLE hProcess, IN PSTR
UserSearchPath, IN BOOL fInvadeProcess );
tSI pSI;

// SymLoadModule64()
typedef DWORD64 (__stdcall *tSLM)( IN HANDLE hProcess, IN HANDLE hFile,
IN PSTR ImageName, IN PSTR ModuleName, IN DWORD64 BaseOfDll, IN DWORD
SizeOfDll );
tSLM pSLM;

// SymGetModuleInfo64()
typedef BOOL (__stdcall *tSGMI_V1)( IN HANDLE hProcess, IN DWORD64
dwAddr, OUT IMAGEHLP_MODULE64_V1 *ModuleInfo );
tSGMI_V1 pSGMI_V1;
typedef BOOL (__stdcall *tSGMI_V2)( IN HANDLE hProcess, IN DWORD64
dwAddr, OUT IMAGEHLP_MODULE64_V2 *ModuleInfo );
tSGMI_V2 pSGMI_V2;
typedef BOOL (__stdcall *tSGMI_V3)( IN HANDLE hProcess, IN DWORD64
dwAddr, OUT IMAGEHLP_MODULE64_V3 *ModuleInfo );
tSGMI_V3 pSGMI_V3;
typedef BOOL (__stdcall *tSGMI_V4)( IN HANDLE hProcess, IN DWORD64
dwAddr, OUT IMAGEHLP_MODULE64_V4 *ModuleInfo );
tSGMI_V4 pSGMI_V4;

int main()
{
// Load dbghelp.dll:
HMODULE m_hDbhHelp = NULL;
TCHAR szTemp[4096];
#ifdef _M_IX86
if ( (m_hDbhHelp == NULL) &&
(GetEnvironmentVariable(_T("ProgramFiles"), szTemp, 4096) > 0) )
{
_tcscat_s(szTemp, _T("\\Debugging Tools for Windows
(x86)\\dbghelp.dll"));
// now check if the file exists:
if (GetFileAttributes(szTemp) != INVALID_FILE_ATTRIBUTES)
{
m_hDbhHelp = LoadLibrary(szTemp);
}
}
#elif _M_X64
if ( (m_hDbhHelp == NULL) &&
(GetEnvironmentVariable(_T("ProgramFiles"), szTemp, 4096) > 0) )
{
_tcscat_s(szTemp, _T("\\Debugging Tools for Windows
(x64)\\dbghelp.dll"));
// now check if the file exists:
if (GetFileAttributes(szTemp) != INVALID_FILE_ATTRIBUTES)
{
m_hDbhHelp = LoadLibrary(szTemp);
}
}
#elif _M_IA64
if ( (m_hDbhHelp == NULL) &&
(GetEnvironmentVariable(_T("ProgramFiles"), szTemp, 4096) > 0) )
{
_tcscat_s(szTemp, _T("\\Debugging Tools for Windows
(ia64)\\dbghelp.dll"));
// now check if the file exists:
if (GetFileAttributes(szTemp) != INVALID_FILE_ATTRIBUTES)
{
m_hDbhHelp = LoadLibrary(szTemp);
}
}
#endif

pSI = (tSI) GetProcAddress(m_hDbhHelp, "SymInitialize" );
pSLM = (tSLM) GetProcAddress(m_hDbhHelp, "SymLoadModule64" );
pSGMI_V1 = (tSGMI_V1) GetProcAddress(m_hDbhHelp,
"SymGetModuleInfo64" );
pSGMI_V2 = (tSGMI_V2) GetProcAddress(m_hDbhHelp,
"SymGetModuleInfo64" );
pSGMI_V3 = (tSGMI_V3) GetProcAddress(m_hDbhHelp,
"SymGetModuleInfo64" );
pSGMI_V4 = (tSGMI_V4) GetProcAddress(m_hDbhHelp,
"SymGetModuleInfo64" );

HANDLE hProcess = GetCurrentProcess();
DWORD64 baseAddr = (DWORD64) GetModuleHandle(NULL);

// SymInitialize
BOOL bRet = pSI(hProcess, NULL, FALSE);
assert(bRet != FALSE);

// Load the symbols:
HANDLE hSnap = CreateToolhelp32Snapshot(TH32CS_SNAPMODULE,
GetCurrentProcessId());
assert(hSnap != (HANDLE) -1);
MODULEENTRY32 me;
me.dwSize = sizeof(me);
Module32First(hSnap, &me);
DWORD64 dw64 = pSLM(hProcess, 0, me.szExePath, me.szModule,
(DWORD64) me.modBaseAddr, me.modBaseSize);

// V1
IMAGEHLP_MODULE64_V1 *pDataV1;
pDataV1 = new IMAGEHLP_MODULE64_V1;
memset(pDataV1, 0, sizeof(IMAGEHLP_MODULE64_V1));
pDataV1->SizeOfStruct = sizeof(IMAGEHLP_MODULE64_V1);
if (pSGMI_V1(hProcess, baseAddr, pDataV1) != FALSE)
printf("V1 is working...\n");
else
printf("V1 is NOT working...\n");

// V2
IMAGEHLP_MODULE64_V2 *pDataV2;
pDataV2 = new IMAGEHLP_MODULE64_V2;
memset(pDataV2, 0, sizeof(IMAGEHLP_MODULE64_V2));
pDataV2->SizeOfStruct = sizeof(IMAGEHLP_MODULE64_V2);
if (pSGMI_V2(hProcess, baseAddr, pDataV2) != FALSE)
printf("V2 is working...\n");
else
printf("V2 is NOT working...\n");

// V3
IMAGEHLP_MODULE64_V3 *pDataV3;
pDataV3 = new IMAGEHLP_MODULE64_V3;
memset(pDataV3, 0, sizeof(IMAGEHLP_MODULE64_V3));
pDataV3->SizeOfStruct = sizeof(IMAGEHLP_MODULE64_V3);
if (pSGMI_V3(hProcess, baseAddr, pDataV3) != FALSE)
printf("V3 is working...\n");
else
printf("V3 is NOT working...\n");

// V4
IMAGEHLP_MODULE64_V4 *pDataV4;
pDataV4 = new IMAGEHLP_MODULE64_V4;
memset(pDataV4, 0, sizeof(IMAGEHLP_MODULE64_V4));
pDataV4->SizeOfStruct = sizeof(IMAGEHLP_MODULE64_V4);
if (pSGMI_V4(hProcess, baseAddr, pDataV4) != FALSE)
printf("V4 is working...\n");
else
printf("V4 is NOT working...\n");

delete pDataV1;
delete pDataV2;
delete pDataV3;
delete pDataV4;

[rogero]

Here is a simple test program:

#include <windows.h>
#include <dbghelp.h>

#include <stdio.h>

int main()
{
HANDLE hProcess = ::GetCurrentProcess();
SymInitialize( hProcess, 0, FALSE );
CHAR szFileName[ MAX_PATH ] = "";
if ( GetModuleFileName( 0, szFileName, sizeof szFileName ) )
{
printf("About to load %s\n", szFileName);
DWORD64 baseAddress = SymLoadModule64(hProcess, 0, szFileName,
NULL, 0, 0);
printf("SymLoadModule: %lx\n", baseAddress);

IMAGEHLP_MODULE64 ModuleInfo;
for ( int i = 0; i <= sizeof ModuleInfo; i++)
{
ModuleInfo.SizeOfStruct = i;
if ( SymGetModuleInfo64(hProcess, baseAddress, &ModuleInfo))
{
printf("Success: length = %i\n", i);
}
}
}
return 0;
}

I compiled on Windows 7 using Visual Studio 2010:

cl /W4 testit.cpp dbghelp.lib

pointing to the latest 6.12.2.633 WindDbg download.

Executing with the default dbghelp:
c:\Windows\System32\dbghelp.dll: 6.1.7600.16385

I get:

About to load d:\Projects\cpp\dbghelp\testit.exe
SymLoadModule: 400000
Success: length = 584
Success: length = 1664
Success: length = 1672
Success: length = 1673
Success: length = 1674
Success: length = 1675
Success: length = 1676
Success: length = 1677
Success: length = 1678
Success: length = 1679
Success: length = 1680

Executing with the previous dbghelp:
dbghelp.dll: 6.11.1.404
I get the same result

About to load d:\Projects\cpp\dbghelp\old\testit.exe
SymLoadModule: 400000
Success: length = 584
Success: length = 1664
Success: length = 1672
Success: length = 1674
Success: length = 1675
Success: length = 1676
Success: length = 1677
Success: length = 1678
Success: length = 1679
Success: length = 1680

Executing with the new dbghelp:
dbghelp.dll: 6.12.2.633

About to load d:\Projects\cpp\dbghelp\new\testit.exe
SymLoadModule: 400000
Success: length = 584
Success: length = 1664
Success: length = 1680

So the case missing in the latest build is that of 1672, ie the size
you get when you compile using the 6.11 dbghelp header.
Sadly all my programs are compiled with that version, so for now
dbghelp.dll 6.12.2.633 has been uninstalled...

Regards,
Roger.

[rogero]

On Mar 2, 8:47 pm, "Jochen Kalmbach [MVP]" <nospam-n...@kalmbach-
software.de> wrote:
> Hi pat!

>  From my test, it seems that it does not overwrite memory, but it always
> returns "FALSE" if you pass the previous struct.

I'm reassured someone else has the same behaviour!

> This is defintiv a show-stopper, because most apps are currently using
> the version with size 0x688 (V3).

Agreed

> #pragma pack(push,8)  // be sure it has the right packing! THIS IS
> MISSING IN DBGHELP.H!!!!

It's present in the more recent versions shipping with debugging
tools.

Regards,
Roger.

[Jochen Kalmbach [MVP]]

Hi rogero!

>> #pragma pack(push,8) // be sure it has the right packing! THIS IS
>> MISSING IN DBGHELP.H!!!!
>
> It's present in the more recent versions shipping with debugging
> tools.

Thanks... I have copied some source from my stackwalker project which is
using the VS2008 dbghelp.h...

[pat styles [microosft]]

Thanks Roger.

Sorry for my delayed response and thank you for the explicit example. The
windbg team is now aware of the problem and is working on fixing it as well
as better defining the intended behavior.

.pat styles [microsoft]

"rogero" <roge...@gmail.com> wrote in message

news:eb094f14-8a8a-4c65...@g10g2000yqh.googlegroups.com...

[Jochen Kalmbach [MVP]]

Hi pat!

> Sorry for my delayed response and thank you for the explicit example. The
> windbg team is now aware of the problem and is working on fixing it

Thanks for confirming it...

> as well as better defining the intended behavior.

You cannot be serious!

The behavior should be clear: A redistributable DLL should always be up-
and downward compatible... this is the behavior with all windows DLLs,
why should dbghelp.dll have an other strategy?

[pat styles [microsoft]]

Jochen, I respectfully submit to you that you haven't the slightest concept
of what I am talking about. Nothing in your posting even brushes close to
the behavior subtleties that I am describing when I say "defining the
intended behavior". Furthermore, I am not going to air those subtleties in
this public forum because it appears obvious that they would not be
discussed rationally. I remember a previous episode in which some pretty
inaccurate conclusions were jumped to.

Microsoft is a large company with many conflicting agendas. The employees
that monitor and contribute to this newsgroup are doing so voluntarily
because they care about the community. Some people would be best served if
they knew who their friends were.

.pat styles [microsoft]

"Jochen Kalmbach [MVP]" <nospa...@kalmbach-software.de> wrote in message
news:eyZ7sK1...@TK2MSFTNGP02.phx.gbl...

[Jochen Kalmbach [MVP]]

Hallo pat!

> Jochen, I respectfully submit to you that you haven't the slightest concept
> of what I am talking about. Nothing in your posting even brushes close to
> the behavior subtleties that I am describing when I say "defining the
> intended behavior".

Then I will say sorry, for my comment.

As you know from some of my previous posts, I cannot understand many of
the desicions at MS.

I appreciate it very much that you and Ivan are responding very good in
this newgroup. A big thanks from the community for this!

[pat styles [microsoft]]

No worries, Jochen.

.pat styles [microsoft]

"Jochen Kalmbach [MVP]" <nospa...@kalmbach-software.de> wrote in message

news:Oi5PF51v...@TK2MSFTNGP02.phx.gbl...

[rogero]

On Mar 9, 3:22 am, "pat styles [microosft]" <pat.sty...@microsoft.com>
wrote:

> Thanks Roger.
>
> Sorry for my delayed response and thank you for the explicit example.  The
> windbg team is now aware of the problem and is working on fixing it as well
> as better defining the intended behavior.

Thanks.

Roger.