NUnit Adapter not finding tests on VS TFS 2015

[Erich Zimmerman]

I'm sure this is just a configuration issue, but I don't know how to get NUnit tests running on TFS Builds.

I have a project MRIntegration in a solution. This project includes NUnit3, v. 3.7.1 and NUnit3TestAdapter v. 3.8.0. My local machine uses Visual Studio 2015. The tests in this project are SpecFlow tests, but Visual Studio doesn't have a problem discovering the tests or running them through Test Explorer.

Our team uses TFS 2015 for source control.

On our Team Foundation Server, I attempted to create a build definition to execute the tests.

In the Build configuration settings, I have a Test Assemblies step, with the following:

Test Assembly: $(Build.StagingDirectory)/MRIntegration.dll

VSTest Version: Visual Studio 2015

Path to Custom Test Adapters: $(Build.SourcesDirectory)\ERA 2.0\packages

During the Test Assemblies phase, the step actually "passes," but no tests are run. Here is the output from the test step:

2017-08-18T19:37:43.0505064Z Executing the powershell script: D:\TfsBuildAgents\TFS3BUILD2-Agent1\tasks\VSTest\1.0.32\VSTest.ps1

 2017-08-18T19:37:43.1285128Z ##[debug]Entering script VSTest.ps1

 2017-08-18T19:37:43.1285128Z ##[debug]vsTestVersion = 14.0

 2017-08-18T19:37:43.1285128Z ##[debug]testAssembly = D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll

 2017-08-18T19:37:43.1285128Z ##[debug]testFiltercriteria = 

 2017-08-18T19:37:43.1285128Z ##[debug]runSettingsFile = D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\s

 2017-08-18T19:37:43.1285128Z ##[debug]codeCoverageEnabled = false

 2017-08-18T19:37:43.1285128Z ##[debug]pathtoCustomTestAdapters = D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\s\ERA 2.0\packages\

 2017-08-18T19:37:43.1285128Z ##[debug]overrideTestrunParameters = 

 2017-08-18T19:37:43.1285128Z ##[debug]otherConsoleOptions = 

 2017-08-18T19:37:43.1285128Z ##[debug]testRunTitle = SquirrelTest - Era - MRIntegration_20170818.7

 2017-08-18T19:37:43.1285128Z ##[debug]platform = any cpu

 2017-08-18T19:37:43.1285128Z ##[debug]configuration = Squirrel_Test

 2017-08-18T19:37:43.1285128Z ##[debug]publishRunAttachments = true

 2017-08-18T19:37:43.1441115Z ##[debug]Loading module from path 'D:\TfsBuildAgents\TFS3BUILD2-Agent1\agent\worker\Modules\Microsoft.TeamFoundation.DistributedTask.Task.Internal\Microsoft.TeamFoundation.DistributedTask.Task.Internal.dll'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Add-BuildAttachment'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Convert-String'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-IndexedSourceFilePaths'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-JavaDevelopmentKitPath'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-MSBuildLocation'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-ServiceEndpoint'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-TfsClientCredentials'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-ToolPath'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-TaskVariable'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-VisualStudioPath'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-VssConnection'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-Ant'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-BatchScript'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-Maven'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-MSBuild'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-PublishSymbols'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-Tool'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Invoke-VSTest'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Publish-BuildArtifact'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Copy-BuildArtifact'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Add-BuildArtifactLink'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Register-XamarinLicense'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Unregister-XamarinLicense'.

 2017-08-18T19:37:43.1441115Z ##[debug]Importing cmdlet 'Get-X509Certificate'.

 2017-08-18T19:37:43.1597113Z ##[debug]Loading module from path 'D:\TfsBuildAgents\TFS3BUILD2-Agent1\agent\worker\Modules\Microsoft.TeamFoundation.DistributedTask.Task.Common\Microsoft.TeamFoundation.DistributedTask.Task.Common.dll'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Complete-Task'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Get-LocalizedString'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Set-TaskVariable'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Set-TaskProgress'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Add-TaskIssue'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Write-TaskDetail'.

 2017-08-18T19:37:43.1753245Z ##[debug]Importing cmdlet 'Find-Files'.

 2017-08-18T19:37:43.1753245Z ##[debug]Loading module from path 'D:\TfsBuildAgents\TFS3BUILD2-Agent1\agent\worker\Modules\Microsoft.TeamFoundation.DistributedTask.Task.TestResults\Microsoft.TeamFoundation.DistributedTask.Task.TestResults.dll'.

 2017-08-18T19:37:43.2377121Z ##[debug]Importing cmdlet 'Publish-TestResults'.

 2017-08-18T19:37:43.2377121Z ##[debug]Importing cmdlet 'Invoke-ResultPublisher'.

 2017-08-18T19:37:43.2533159Z ##[debug]No Pattern found in solution parameter.

 2017-08-18T19:37:43.2533159Z ##[debug]Calling Invoke-VSTest for all test assemblies

 2017-08-18T19:37:43.3001136Z ##[debug]Test results directory: D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\TestResults

 2017-08-18T19:37:43.3157153Z Working folder: D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37

2017-08-18T19:37:43.3157153Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll"  /logger:trx /TestAdapterPath:"D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\s\ERA 2.0\packages\"

 2017-08-18T19:37:43.9085408Z Microsoft (R) Test Execution Command Line Tool Version 14.0.25420.1

 2017-08-18T19:37:43.9085408Z Copyright (c) Microsoft Corporation.  All rights reserved.

 2017-08-18T19:37:44.6261651Z Starting test execution, please wait...

 2017-08-18T19:37:44.8289743Z Warning: Using Isolation mode to run the tests as diagnostic data adapters were enabled in the runsettings. Use the /inIsolation parameter to suppress this warning.

 2017-08-18T19:37:50.3515734Z Information: NUnit Adapter 3.8.0.0: Test execution started

 2017-08-18T19:37:50.3827861Z Information: Running all tests in D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll

 2017-08-18T19:37:50.8975929Z Information: NUnit failed to load D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll

 2017-08-18T19:37:50.9131965Z Information: NUnit Adapter 3.8.0.0: Test execution complete

 2017-08-18T19:37:51.1160031Z Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true

 2017-08-18T19:37:51.2876145Z ##[warning]No results found to publish.

 2017-08-18T19:37:51.2876145Z ##[debug]Leaving script VSTest.ps1

Based on my reading the NUnit test adapter seems to be loading correctly; my only thought is that it is not finding tests in the assembly.

This is an enterprise-wide build server, so I don't control what is installed on it, but I thought the NuGet packages were supposed to take care of that.

Thanks,

Erich

[Rob Prouse]

The NUnit error is, Unit failed to load D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll

Based on that, something is causing the your test assembly to not load. 

• Is your test assembly AnyCPU or is it x86/x64? 
• Does it reference any native assemblies or x86/x64 assemblies? 
• Are there any referenced assemblies that might not be getting copied to the bin directory on the build machine? 
• Do the tests work for you locally when you build in release?
  

Rob

--
You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nunit-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to nunit-...@googlegroups.com.
Visit this group at https://groups.google.com/group/nunit-discuss.
For more options, visit https://groups.google.com/d/optout.

[Erich Zimmerman]

Rob,

1. My build is AnyCPU

2. No native assemblies loaded,

3. I don't believe so, as things are building correctly on my local machine, and the build succeeds for the assemblies on the build server (I understand that building with an assembly is different than loading an assembly at runtime).

4. Everything passes on my local machine, in Release and Debug builds.  In fact, I copied the command line:

1. Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe 
   "D:\TfsBuildAgents\TFS3BUILD2-Agent2\_work\21\a\MRIntegration.dll"  /logger:trx /TestAdapterPath:"D:\TfsBuildAgents\TFS3BUILD2-Agent2\_work\21\s\ERA 2.0\packages\"

Of course, I tweaked it for the build location on my machine.

I also validated the build steps in the logs:

2017-08-22T00:49:47.2309949Z        _CopyAppConfigFile:

2017-08-22T00:49:47.2309949Z          Copying file from "obj\Debug\MRIntegration.dll.config" to "D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll.config".

2017-08-22T00:49:47.2309949Z        CopyFilesToOutputDirectory:

2017-08-22T00:49:47.2309949Z          Copying file from "obj\Debug\MRIntegration.dll" to "D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll".

2017-08-22T00:49:47.2309949Z          MRIntegration -> D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.dll

2017-08-22T00:49:47.2309949Z          Copying file from "obj\Debug\MRIntegration.pdb" to "D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\a\MRIntegration.pdb".

2017-08-22T00:49:47.2465946Z     87>Done Building Project "D:\TfsBuildAgents\TFS3BUILD2-Agent1\_work\37\s\ERA 2.0\MRIntegration\MRIntegration.csproj" (default targets).

Is there a way to get more verbose output from the Test process or NUnit adapter? The "could not load" statement doesn't tell me very much.

Thanks!

Erich

To unsubscribe from this group and stop receiving emails from it, send an email to nunit-discus...@googlegroups.com.