MSTest with codecoverage - attempts to instruments DLLs it can't

[Tristan MSDN Keen]

I have
"Error VSP1039 : VSInstr does not support instrumenting binaries built with
prevous version of the compiler."

When trying to use MSTEST on a set of tests.

It is trying to instrument a 3rd party DLL, which is indeed compiled in VS6.0.

I don't want it to instrument that DLL, only "my" code, but I do not see any
way of ignoring the error, or getting it to not attempt to instrument it.

I presume it is aggressively following all dependancies it detects my code
uses (dll import / pinvoke on that DLL)

/IGNOREWARN:1039 appears to not be allowed (presumably because it is an
error not a warning)

If I edit the test configuration, it only allows me a granularity of "per
project" on and off, and it is a large project that happens to include at one
point the use of that 3rd party dll.

(repost, because email reply settings have changed)

[Steven Cheng [MSFT]]

Hi Tristan,

From your description, you have some .NET managed assemblies which use
MSTest to perform tests(profiling , code coverage...), however, since some
of the assembly use some unmanaged DLL, you encountered the following error
when performing the test, correct?

"Error VSP1039 : VSInstr does not support instrumenting binaries built with
prevous version of the compiler."

Are you using Visual studio 2005 or 2008 series? Since you've mentioned
there does have unmanaged assembly(built with vs 6.0) in the project, I
think it is likely to be the cause. Currently, my suggestion is that you
first try simplify the solution for test(or just generate a simplified copy
of the original solution for our troubleshooting), make sure it can result
to the same error and include minimized number of components.

Generally for the Instrumentation and Code coverage, it is a feature
provided by visual studio team system (tester edition). In Visual studio
IDE, we can run this kind of test via a test project. And based on my
research, for Code coverage, it require those assemblies(you want to
perform code coverage test) to be instrumented first. And you can configure
which assemblies you want to perform code coverage via "Test
Configuration". In visual studio, you can access the setting via the
following menu (in test project):

"Test ---> Edit Test Run Configuration---> Local Test
Run(localtestrun.testrunconfig)"

in the popup dialog, there is "code coverage" section let you to control
the assemblies to perform instrument and code coverage. You can choose to
save a copy of the configuration file.

#How to: Specify a Test Run Configuration
http://msdn.microsoft.com/en-us/library/ms182480.aspx

#Editing a Test Run Configuration XML File
http://msdn.microsoft.com/en-us/library/ms404663.aspx

For tests performed by "MSTest" command utility, I find that it support the
"/runconfig:[ file name ] " command option which is used to specify a test
run configuration. The run configuration file is just the same format of
the "Local test Run" setting I mentioned above(for IDE GUI based test):

#MSTest.exe Command-Line Options
http://msdn.microsoft.com/en-us/library/ms182489(VS.80).aspx

You can have a look to see whether it helps.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msd...@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------
>From: =?Utf-8?B?VHJpc3RhbiBNU0ROIEtlZW4=?= <QAS...@nospam.nospam>
>Subject: MSTest with codecoverage - attempts to instruments DLLs it can't
>Date: Tue, 29 Jul 2008 02:24:01 -0700

[Tristan MSDN Keen]

Yes, that's correct.

We are using Visual Studio 2008 Team System.
We have a test project in the solution, and are configuring code coverage
through the localtestrun.testconfig file, and specifying it on the MSTEST
command.

We wish to run codecoverage of all the projects (assemblies) in our
solution, so intentionally have most of them ticked for codecoverage in the
config editor.

In the editor, if I uncheck the project which includes the DllInvoke
reference to the 3rd party DLL (which is C) and test, it will successfully
instrument and perform codecoverage. If I tick that project for coverage, I
get the error, and it does not proceed.

But I do wish to unit test and codecoverage our code within that project.
Using the external DLL is not the only thing that project does.

But I don't want to attempt to instrument or code cover the DLL itself, as
it's 3rd party and not under my control.
I want to test access to it - not test _it_.

My command line looks like this:

MSTEST.EXE /testmetadata:Match\Match42.vsmdi
/runconfig:Match\localtestrun.testconfig /resultsfile:Match080730_results.trx
/testlist:MaxSizeList /testlist:StringUtils /searchpathroot:s:\MatchOutputDir

which is being executed through a build system.

[Steven Cheng [MSFT]]

Hi Tristan,

Regarding on the further description

================

In the editor, if I uncheck the project which includes the DllInvoke
reference to the 3rd party DLL (which is C) and test, it will successfully
instrument and perform codecoverage. If I tick that project for coverage, I
get the error, and it does not proceed.

======================

so the problem is mostly specific to this library project(which use pinvoke
with some unamanged dlls). Would you give me a detailed description of the
project structure( the managed and umanaged pinvokes dll used). I will
forward this to some other VS engineers for discussion.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msd...@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.

==================================================

[Tristan MSDN Keen]

The solution contains

12 class library projects
a Windows Forms App
Unit Test Project
2 Console apps
asp.net web app
web setup project

it also has a few assemblies added that are generated externally (but using
the same build system, but in separate solutions) which are in "solution
items" / "3rd party assemblies"

the unmanaged DLL we are having trouble with "QALICED.DLL" is also in the
solution items / 3rd party assemblies

now, the problem QALICED dll is referenced by one of the other 3rd part
assemblies which was built as a standalone solution, and the assembly copied
in.

That (externally built) solution was made up of two class library projects.
One of those class libraries had one source file module to "wrap" the calls
to the unmanaged DLL, and has the [DllImport] statements for the seven or so
functions in the form:

const string QADLLNAME = "qaliceb.dll";

[DllImport(QADLLNAME)]
static extern void QALCDInit();
static extern Int32 QALCDSearch([MarshalAs(UnmanagedType.LPStr)]
string Param);

etc.

which has one source module that has the

"Steven Cheng [MSFT]" wrote:

> Hi Tristan,
>

[Steven Cheng [MSFT]]

Thanks for your reply Tristan,

I've forwarded this question to some other VS code coverage engineers for
more ideas. If there is any new information, I'll inform you here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

[Steven Cheng [MSFT]]

Hi Tristan,

How are you doing?

Have you got any progress on this issue? I've got some feedback from some
Code Coverage team engineers and they're willing to help on this. So far
they also mentioned the configuration in "localtestrun.testrunconfig" file
which can configure the "Code Coverage" include assemblies. However, I
think the problem here is that your solution contains some external
assembly reference pointing to some 3rd party assemblies(also unmanaged
ones) in another solution. This will make the problem more complicated.

Is it possible that you build a simplified solution that can repro the
behavior and send it to me? For example, just a soution with a managed
project(also a test project) reference another external assembly or
pinvoke unmanaged assemblies(producted by another solution). Thus, I can
perform some further tests or discuss with the other engineers on this. You
can reach me at the following email address:

"stcheng" + @ + "microsoft.com"

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we

can improve the support we provide to you. Please feel free to let my
manager know what you think of

the level of service provided. You can send feedback directly to my manager
at: msd...@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to

http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.
--------------------

>Content-Transfer-Encoding: 7bit
>From: stc...@online.microsoft.com (Steven Cheng [MSFT])
>Organization: Microsoft
>Date: Tue, 05 Aug 2008 06:47:29 GMT

>Subject: RE: MSTest with codecoverage - attempts to instruments DLLs it can

>