Does Dr. Memory work with Microsoft's MSTest?

[Choiwah Chow]

Hello,

 

I just started using Dr. Memory to debug my Visual C++ dll. Since it is a dll, it can't run directly by drmemory.exe. Fortunately, we also have a test project on this dll. We run MSTest.exe offline every night, and it takes about 15 min to complete. So I run

 

drmemory.exe -- MSTest.exe <arguments for running MSTest.exe>

 

From the screen, I saw that MSTest was loaded:

 

    Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1

    Copyright (c) Microsoft Corporation. All rights reserved.

    Loading C:\Product\Development\Reval\HedgeRx\Dev\Components\RevalAnalytics\AnalyticsW\RegressionTestList.vsmdi...

Then, it seemed to run for another minute or so, and drmemory seemed to complete normally.  But that was too fast, and I didn't seen any errors on my test run. There were no indication of MSTest completed either.

Does Dr. Memory work with MSTest?

 

Thanks,

Choiwah

[Derek Bruening]

On Sun, Nov 18, 2012 at 10:58 AM, Choiwah Chow <choiwa...@gmail.com> wrote:

Then, it seemed to run for another minute or so, and drmemory seemed to complete normally.  But that was too fast, and I didn't seen any errors on my test run. There were no indication of MSTest completed either.

Does Dr. Memory work with MSTest?

Can you elaborate: drmemory completed normally means that notepad popped up for mtest?  What do you mean "no indication of MSTest completed"?  Did it exit early?  What is it supposed to do natively?

What happens when passing "-light" to Dr. Memory?  What about when running with the options "-leaks_only -no_count_leaks -no_track_allocs"?  Does "-debug -pause_at_assert" raise any errors?

- Derek

[Choiwah Chow]

Derek,

 

Thanks a lot for your prompt reply.

 

Yes, drmemory completed and exited normally with notepad popped up with the normal messages in it.

 

On a closer look of the screen, I suspect MSTest might not even have started, becuase I did not see the usual "Start Execution" message printed out by MSTest.

 

Of note:

1) Microsoft's Test projects are in fact managed C++ code (C++/CLI). The C++/CLI code would invoke the native C++ dll in a simple function call manner.  It is the memory problem in the native C++ code that we are interested. The caller (C++/CLI code) is very, very simple and straight-forward code with practically no programming logic in it.

2) I wrote a separate native C++ program to make the same calls as I did with the C++/CLI test project. Drmemory.exe worked fine with it, spilling out the memory problems in our native C++ library.  We are really interested in getting drmemroy to work with our existing MSTest test framework as a lot of test cases have already been developed.

 

Thanks a lot for your help.

--Choiwah

[Derek Bruening]

What happens when passing "-light" to Dr. Memory?  What about when running with the options "-leaks_only -no_count_leaks -no_track_allocs"?  Does "-debug -pause_at_assert" raise any errors?

When you run MSTest, does just one process get run or are there multiple?  Are they all created by the initial mstest.exe?

[Choiwah Chow]

Derek,

 

I posted a repy to this thread, but it seems to be lost in the twilight zone. I'll reply again.

 

First, thanks a lot for you help and hints in running Dr. Memory.

 

-debug -pause_at_assert:

I ran it twice, both times stopped with a popup. But the error messages were different. The second time:

ASSERT FAILURE (thread 7716): d:\derek\drmemory\withwiki\trunk\drmemory\syscall_windows.c:1267: size <= 2*(...... ) + PORT_MAXIMUM_LENGTH)(PORT_MESSAGE size much larger than expected)

 

On clicking okay:

ASSERT Failure (thread 7928): d\derek\drmemory\withwiki\trunk\drmemory\replace:720: false (failed to un-replace)    <-- This message was the first message seen in the other run.

 

On clicking OK:

Dr. Memory is paused at an assert in pid=268

 

On clicking Okay:

Dr. Memory is paused at an assert in pid=8034

 

Results.txt popped up, but it did not contain the usual ending summary.  I think Dr. Memory aborted somewhere.

 

 

-light

Didn't seem to work. I didn't see the usual MSTest diagnostics messages. MSTest printed the

Starting execution....

message. But the process ended soon afterward, with results.txt popped up. The results.txt contained only 40 error messages.

 

 

-leaks_only -no_count_leaks -no_treack_allocs

This seems to work! Execution completed and I saw the usual MSTest messages as it ran.  But the results.txt file popped up stated no errors were found, which is too good to be true.

 

 

Also, I think MSTest ran as single thread, although it is clearly capable of running multi-thread.

 

 

Please let me know if I can supply more details.

 

Thanks,

Choiwah

[Derek Bruening]

On Mon, Nov 19, 2012 at 3:15 PM, Choiwah Chow <choiwa...@gmail.com> wrote:

I posted a repy to this thread, but it seems to be lost in the twilight zone. I'll reply again.

The first reply was just to me and not the list, it looks like. 

-debug -pause_at_assert:

I ran it twice, both times stopped with a popup. But the error messages were different. The second time:

ASSERT FAILURE (thread 7716): d:\derek\drmemory\withwiki\trunk\drmemory\syscall_windows.c:1267: size <= 2*(...... ) + PORT_MAXIMUM_LENGTH)(PORT_MESSAGE size much larger than expected)

 

On clicking okay:

ASSERT Failure (thread 7928): d\derek\drmemory\withwiki\trunk\drmemory\replace:720: false (failed to un-replace)    <-- This message was the first message seen in the other run.

What version of Dr. Memory are you using?  If it's the 1.5.0 release, can you try the latest nightly build, which is a self-extracting archive at  http://build.chromium.org/p/client.drmemory/builds/drmemory-windows-latest-sfx.exe?  If you can reproduce either or both asserts there, can you file an issue in our tracker (http://code.google.com/p/drmemory/issues/entry)?

 Results.txt popped up, but it did not contain the usual ending summary.  I think Dr. Memory aborted somewhere.

It aborts on the assert (unless you pass -ignore_asserts).

 

 -leaks_only -no_count_leaks -no_treack_allocs

This seems to work! Execution completed and I saw the usual MSTest messages as it ran.  But the results.txt file popped up stated no errors were found, which is too good to be true.

 Those options disable all error checking.  But we know that the bug is somewhere on the continuum from " -leaks_only -no_count_leaks -no_track_allocs" to "-light".

Can you try passing these options to drmemory.exe (ideally the most current build of Dr. Memory) and report what happens:

* -leaks_only -no_count_leaks

* -leaks_only

* -light -dr_debug

We may need to get some log files, or else find a way to reproduce it.  Can you shrink it down to the smallest mstest workload that reproduces the problem?

- Derek

[Choiwah Chow]

Derek,

 

I got the latest nightly build and used it to replace my drmemory folders (there are more in the nightly build than the official 1.5.0 release). Then I ran the tests again:

 

-light -dr_debug

It seems to hang after printing out the loading MSTest message:

Loading ..\AnalyticsW\RegressionTestList.vsmdi...

 

I killed it with crtl-C, and at that time DynamoRIO popus up a message:

Application C:\Program Files\Microsoft Visual Sudio 10.0\Common7\IDE\MSTest.exe (11528) DynamoRIO usage error: privately-loaded library executed by app: please report this transparency violation.

 

After clicking on OK, other messages popped up about time-out etc.

 

-leaks_only

Ran very fast (too fast to have executed any test cases), but did not produce any normal MSTest messages, ended without summary:

 

Starting execution...
~~Dr.M~~ Fetching 2 symbol files...
~~Dr.M~~ [1/2] Fetching symbols for C:\WINDOWS\system32\msvcr100d.dll
WARNING: SymGetModuleInfoEx failed: 87
~~Dr.M~~ [2/2] Fetching symbols for C:\WINDOWS\system32\msvcp100d.dll
WARNING: SymGetModuleInfoEx failed: 87
~~Dr.M~~ Fetched 0 symbol files successfully

 

-leaks_only -no_count_leaks

Completed very fast (too fast to have test cases executed), with results.txt popped up. But no normal MSTest message.

All that were shown on the screen:

 

~~Dr.M~~ Dr. Memory version 1.5.1112
~~Dr.M~~ Running ""MSTest" "/testlist:RegressionTestList" "/testmetadata:..\AnalyticsW\RegressionTestList.vsmdi""

Microsoft (R) Test Execution Command Line Tool Version 10.0.30319.1
Copyright (c) Microsoft Corporation. All rights reserved.

Loading ..\AnalyticsW\RegressionTestList.vsmdi...
Starting execution...

 

 

I'll try to re-create the problem with some other MSTest project.  The one I am working on requires our big C++ dll and database to run.

 

Thanks a lot for your help.

--Choiwah

[Choiwah Chow]

Not sure whether this may help, but if I run drmemory.exe with MSTest.exe without any special flags, somewhere during the execution, I got

 

System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.AssemblyEnumerator.EnumerateAssembly(IWarningHandler warningHandler, String location, ProjectData projectData, ObjectHandle assemblyResolverWrapper)
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAttributeEnumerator.Read(ITestTypeExtensionClientSidesProvider provider, IWarningHandler warningHandler, String assemblyFileName, ProjectData projectData, TestRunConfiguration testRunConfiguration)
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestTip.Load(String assemblyFileLocation, ProjectData projectData, IWarningHandler warningHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ITip[] tips, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestsFromTipsHelper(IEnumerable`1 locations, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(IEnumerable`1 locations, ProjectData projectData, TestConflictHandler vetoingHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ProjectData projectData, TestConflictHandler vetoingHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestLinkStorageHelper.LoadTests(String fullStoragePath, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.SimpleLoadTestLinkStorageHelper.Load(String fullStoragePath, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadLinkStorage(Link link, Dictionary`2 searchPaths, Boolean tryOriginalLocation, CannotFindLinkHandler cannotFindHandler, ProjectData projectData, LoadLinkStorageHelper loadHelper)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadCategoriesAndDependencies(String categoryFile, String[] testSearchPaths, String[] runConfigSearchPaths, Boolean tryOriginalLocation, TestConflictHandler testsHandler, TestListCategoryConflictHandler categoriesIdHandler, TestListCategoryConflictHandler categoriesNameHandler, RunConfigurationConflictHandler runConfigHandler, CannotFindLinkHandler cannotFindStorageHandler, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.CommandLine.TmiAdapter.LoadMetaDataFile()
   at Microsoft.VisualStudio.TestTools.CommandLine.TestMetaDataCommand.Execute(TmiAdapter tmiAdapter)
   at Microsoft.VisualStudio.TestTools.CommandLine.Executor.Execute()
   at Microsoft.VisualStudio.TestTools.RunnerCommandline.Runner.Execute(String[] args)
   at Microsoft.VisualStudio.TestTools.RunnerCommandline.Runner.Main(String[] args)
~~Dr.M~~

[Choiwah Chow]

Derek,

 

It turns out this issue is easy to reproduce. It appears that Dr. Memory does not work with MSTest. I created an empty test project in Visual Studio 2010 on a 32-bit Windows XP machine, and I got the same problem when I ran "drmemory.ext -- MSTest". Sometime after seeing the MSTest message "loading ..\testProject.vsmdi....", I would see the System access violation message.

 

System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.AssemblyEnumerator.EnumerateAssembly(IWarningHandler warningHandler, String location, ProjectData projectData, ObjectHandle assemblyResolverWrapper)
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAttributeEnumerator.Read(ITestTypeExtensionClientSidesProvider provider, IWarningHandler warningHandler, String assemblyFileName, ProjectData projectData, TestRunConfiguration testRunConfiguration)
   at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestTip.Load(String assemblyFileLocation, ProjectData projectData, IWarningHandler warningHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ITip[] tips, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestsFromTipsHelper(IEnumerable`1 locations, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(IEnumerable`1 locations, ProjectData projectData, TestConflictHandler vetoingHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTests(String location, ProjectData projectData, TestConflictHandler vetoingHandler)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadTestLinkStorageHelper.LoadTests(String fullStoragePath, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.SimpleLoadTestLinkStorageHelper.Load(String fullStoragePath, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadLinkStorage(Link link, Dictionary`2 searchPaths, Boolean tryOriginalLocation, CannotFindLinkHandler cannotFindHandler, ProjectData projectData, LoadLinkStorageHelper loadHelper)
   at Microsoft.VisualStudio.TestTools.TestManagement.Tmi.LoadCategoriesAndDependencies(String categoryFile, String[] testSearchPaths, String[] runConfigSearchPaths, Boolean tryOriginalLocation, TestConflictHandler testsHandler, TestListCategoryConflictHandler categoriesIdHandler, TestListCategoryConflictHandler categoriesNameHandler, RunConfigurationConflictHandler runConfigHandler, CannotFindLinkHandler cannotFindStorageHandler, ProjectData projectData)
   at Microsoft.VisualStudio.TestTools.CommandLine.TmiAdapter.LoadMetaDataFile()
   at Microsoft.VisualStudio.TestTools.CommandLine.TestMetaDataCommand.Execute(TmiAdapter tmiAdapter)
   at Microsoft.VisualStudio.TestTools.CommandLine.Executor.Execute()
   at Microsoft.VisualStudio.TestTools.RunnerCommandline.Runner.Execute(String[] args)
   at Microsoft.VisualStudio.TestTools.RunnerCommandline.Runner.Main(String[] args)
~~Dr.M~~

 

I'll log an issue with the tracker.

 

Thanks,

Choiwah