Visual Studio loads incorrect version of MFC100u.dll/CRT DLLs when debugging minidump file

[Kenny Liu]

Crashrpt is amazing so we plan to use in in our project.

I notice that in the bin folder of crashrpt project, there are some MS DLLs such as mfc100u.dll/msvcp100.dll. However, normally we don't distribute these DLLs to users directly, instead we do this by including the Visual C++ Redistributable in our setup program, we use VS2010 to develop our application and set the project with /MD option.

In order to test how crashrpt work when these MS DLLs are stripped, I copy these files to a new folder (e.g. E:\test, all thoese mfc100u.dll/msvcp100.dll are skipped, all these executables are 32bit):

CrashRpt1400.dll

CrashSender1400.exe

crashrpt_lang.ini

dbghelp.dll

MFCDemo.exe

MFCDemo.pdb

I run the 32bit MFCDemo.exe on 64bit Windows 7, and then I use Process Explorer (procexp.exe, formerly from sysinternals and now from Microsoft) to detect exactly what DLLs are loaded into its space, turns out these DLLs are loaded:

C:\Windows\SysWOW64\mfc100u.dll

C:\Windows\SysWOW64\msvcp100.dll

Then I follow the steps to crash MFCDemo.exe and generate minidump file, when I open it with VS2010 I notice that the DLLs listed in Visual Studio are from different path:

C:\Windows\System32\mfc100u.dll

C:\Windows\System32\msvcp100.dll

When I start debugging this minidump file, the callstack are listed as below:

> MFCDemo.exe!CMFCDemoDoc::Serialize(CArchive & ar)  Line 57 C++

  mfc100u.dll!52ba32b0()

  [Frames below may be incorrect and/or missing, no symbols loaded for mfc100u.dll]

  mfc100u.dll!52ba2824()

However, if I copy those MS DLLs (like mfc100u.dll from the Bin folder of crashrpt project) and repeat the above steps, then the callstack are properly displayed.

So I wonder, is there a solutio to handle this problem for 32bit application that runs on 64bit OS? I mean, do we have to include MS DLLs in our setup?

Thanks!

Kenny

[zexspectrum]

Hello,

When running a 32-bit app WOW64 uses something called the File System Redirector:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa384187(v=vs.85).aspx

All requests to System32 dir are redirected to the syswow64 folder (whether you want them to or not).

The 32bit and 64bit versions of the MFC and C-runtimes have the same name. They are normally deployed using the redistributable MSI files and Windows will handle the linking at run-time. However, if you absolutely MUST have a specific version linked to your app you'll need to deploy them in the same folder as your app. Don't try and mix 32/64 bit versions in the same folder. Here's what Microsoft has to say on the subject:

http://msdn.microsoft.com/en-us/library/ms235264.aspx

Note: Just to complicate the matter there are different redistributable versions of the same file, e.g. VS2010 SP1 versions.

Please let me know if this helps.

[Kenny Liu]

Hi,

Thanks for your explanations, I am aware of the File System Redirector, but it's still not clear why opening the minidump file in Visual Studio is not "redirected" to load the MFC DLLs in WOW64.

According to the MSDN, we can deploy MFC DLLs by Visual C++ Redistributable Package (VCRedist_x86.exe, VCRedist_x64.exe, or VCRedist_arm.exe), or  directly install specific Visual C++ DLLs to the application local folder (I guess that's how the crashrpt MFCDemo works)

We choose to install Visual C++ Redistributable Package in the setup program ( which I guess it's what most people choose, right?). We are also aware that the user might install the other application with newer version of MFC DLLs  included, which means we cannot assume that the user uses the exact same version as what the developers have.

As I said above, we don't want to deploy the MFC DLLs along with our application (by putting them to the same folder as the app), instead, we use CRedist_architecture.exe to do this job.

So what I am trying to do is to simulate the case that our users do: 

1. The user (in this case, it's me) run the app 

2. The app loads the MFC/CRT DLLs in WOW64 folder

3. The app crash and the user (me again) send the minidump file to me

4. I open the minidump file in Visual Studio, here comes the problem: it seems that VS somehow does not know about File System Redirector, it use the DLLs in System32 folder instead of WOW64 folder, that's why the callstack is not "correct".

All the above operations happen on the same computer (the same environment, the same setup, the same OS), if in this specific scenario, VS still cannot load the "correct" DLLs, how could we rely on it to load the correct DLLs in the case that the actual users send us the crash report (when the version/path of MFC DLLs are not clear)?

So the question is, in order for Visual Studio to correctly show the callstack for MFC modules, do we need to include the file version info (version/size/path) of MFC/CRT DLLs in the crash report such that we can try to get the exact same copies from MS (somehow, if it's possible?) to debug, or do we have to deploy the MFC DLLs in the same folder as our app such that we can assume the users are using the exact same version of MFC DLLs as we do?

Regards,

Kenny

在 2012年12月16日星期日UTC+8下午4时51分53秒，zexspectrum写道：

[zexspectrum]

Hello,

The crash minidump file contains information on each DLL module loaded into your application. Each module is identified uniquely by GUID + build Age string. So, no matter how you distribute your MFC binaries, VS is able to locate the correct DLLs and PDB files for your minidump. I don't know why it does not find those files. I only can guess. Maybe there is a bug in its search algorithm or it has no permissions to access the correct DLLs. How do you launch your VS - as admin or as a limited user? 

By the way, missing MFC binaries should not prevent you from recovering the stack trace if MFC code is not optimized. If MFC code is optimized and uses FPO (frame pointer omission), than you will need that binary file to recover the stack trace correctly. 

[Kenny Liu]

Hi,

I launch VS as admin, UAC is disabled by the way.

Normally we put our focus on our own DLL modules and we don't care about MFC DLLs. However, sometimes crash happens inside MFC DLLs, and sometimes we might need to know what MFC DLLs are doing in the context of execution by examine the call stack. So, my concern is that when this happen, how am I supposed to do if for whatever reason (e.g. incorrect version/directed location) VS cannot display the source code of MFC?

Let's put aside this special case (the user and developer are the same person, operations happen on the same computer) away, suppose we redistribute MFC DLLs with VCredist_XX.exe with our app, then the user install another app that redistribute newer version of MFC DLLs. The user run our app that loads the newer version of MFC DLLs then it crash, we receive the minidump file then try to open it in VS, but we do not have the exact same version of MFC DLLs as the user has, I guess the similar problem will happen: VS cannot show the source code for MFC modules. I did not test this by myself though, so I am wondering have you tested this case before?

How do you deploy your app by the way? If you redistribute MFC DLLs along with the same folder of your app, then you are free from the above issues but the user lose the benefit of system update. If you redistribute MFC DLls with VCRedist_XX.exe, then you will be facing the same problem, how do you handle this?

Thank you for your kind help.

Regards,

Kenny

在 2012年12月16日星期日UTC+8下午10时32分13秒，zexspectrum写道：

[zexspectrum]

I usually don't care about third-party modules and their symbols. I only look at my own modules for whom I have source code. So, I think I usually have the case you are talking about. It is a normal situation, IMHO.

For our apps, we put CRT binaries as local files to app folder, and we do not use vcredist. But, I do not see any problems with using vcredist, as we don't care about third-party modules. If we decide that vcredist is better, we will use vcredist instead.

[Kenny Liu]

I see, maybe it's too early to worry about this.

在 2012年12月16日星期日UTC+8下午11时44分52秒，zexspectrum写道：