System.NotSupportedException thrown within NUnit console runner when running within TeamCity

[david.r...@viavisolutions.com]

NUnit & NUnit console runner version 3.2.

TeamCity Enterprise version 9.1.3

TechTalk.SpecFlow version 2.0

Hello,

I have a suite of software that contains 'standard' NUnit test fixtures.  I refer to these as legacy fixtures below.

I also use BDD SpecFlow scenarios (each scenario generates an NUnit test fixture).

I am trying to upgrade from NUnit 2.6.4 to NUnit 3.2.

I am struggling to get the fixtures generated by SpecFlow to run within my TeamCity continuous integration system.

The fixtures (legacy and SpecFlow) both ran ok with NUnit 2.6.4

Prior to the upgrade my CI contained a step to run the legacy fixtures using the TeamCity NUnit task.  Another step in the same TeamCity project was running the SpecFlow fixtures via TeamCity.SpecFlow.Reporting, which consists of a set of PowerShell scripts.

TeamCity runs the PowerShell script, the script launches the NUnit runner causing it to output reports, then the reports are processed and the results are imported into TeamCity.

After the upgrade I am using the same step to run the legacy fixtures.  These fixtures need to run within their own app domains.  I used the 'nunit_use_project_file=false' TeamCity configuration parameter (documented at https://blogs.endjin.com/2016/02/specflow-2-0-nunit3-visual-studio-and-teamcity/) to achieve this.

The legacy fixtures continue to run without issue.

For the SpecFlow fixtures, I modified the PowerShell scripts to use the v3.2 console runner and adjusted the command-line parameters appropriately.  Having done this, every time TeamCity attempts to execute the runner, it fails and the PowerShell scripts indicate that the runner threw an exception.

This is the output from Team City when it trys to run SpecFlows with the PowerShell script:

Step 5/8: Specflow feature runner (Powershell) (7s)

[13:06:32][Step 5/8] PowerShell Executable: C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe

[13:06:32][Step 5/8] Working directory: C:\BuildAgent\work\94f18887f3572298\Geo\BuildTools\Specflow

[13:06:32][Step 5/8] PowerShell arguments: [-NoProfile, -NonInteractive, -executionpolicy, bypass, -File, C:\BuildAgent\work\94f18887f3572298\Geo\BuildTools\Specflow\producereports.ps1, -root, ..\..\Source, -bin, ..\..\Source\GeoBuildRelease\Specs]

[13:06:33][Step 5/8] [TeamCity.SpecFlow.Reporting]: property [Configuration] was updated with value []

[13:06:33][Step 5/8] [TeamCity.SpecFlow.Reporting]: property [SpecFlowConfig] was updated with value [C:\BuildAgent\work\94f18887f3572298\Geo\BuildTools\Specflow]

[13:06:33][Step 5/8] [TeamCity.SpecFlow.Reporting]: property [PathToPackagesFolder] was updated with value [C:\BuildAgent\work\94f18887f3572298\Geo\Source]

[13:06:33][Step 5/8] [TeamCity.SpecFlow.Reporting]: property [OutputFolder] was updated with value [C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs]

[13:06:33][Step 5/8] TeamCity.SpecFlow.Reports (6s)

[13:06:38][TeamCity.SpecFlow.Reports] [TeamCity.SpecFlow.Reporting]: property [ProjectFile] was updated with value [C:\BuildAgent\work\94f18887f3572298\Geo\Source\Arieso\Data\AdvancedDiagnostics\Specifications\AdvancedDiagnostics.Specs\AdvancedDiagnostics.Specs.csproj]

[13:06:38][TeamCity.SpecFlow.Reports] TeamCity.SpecFlow.Reporting

[13:06:38][TeamCity.SpecFlow.Reporting] === Invoking nunit.console.exe ===

[13:06:38][TeamCity.SpecFlow.Reporting] Using specs assembly located: C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs\Arieso.Data.AdvancedDiagnostics.Specs.dll

[13:06:38][TeamCity.SpecFlow.Reporting] Invoking nunit with: C:\BuildAgent\work\94f18887f3572298\Geo\Source\Packages\NUnit.ConsoleRunner.3.2.0\tools\nunit3-console.exe C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs\Arieso.Data.AdvancedDiagnostics.Specs.dll  --labels=On  --out=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.txt  --result=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.xml;format=nunit2

[13:06:39][TeamCity.SpecFlow.Reporting] NUnit Console Runner 3.2.0 

[13:06:39][TeamCity.SpecFlow.Reporting] Copyright (C) 2016 Charlie Poole

[13:06:39][TeamCity.SpecFlow.Reporting] 

[13:06:39][TeamCity.SpecFlow.Reporting] Runtime Environment

[13:06:39][TeamCity.SpecFlow.Reporting]    OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1

[13:06:39][TeamCity.SpecFlow.Reporting]   CLR Version: 4.0.30319.42000

[13:06:39][TeamCity.SpecFlow.Reporting] 

[13:06:39][TeamCity.SpecFlow.Reporting] Test Files

[13:06:39][TeamCity.SpecFlow.Reporting]     C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs\Arieso.Data.AdvancedDiagnostics.Specs.dll

[13:06:39][TeamCity.SpecFlow.Reporting]      --labels=On

[13:06:39][TeamCity.SpecFlow.Reporting]      --out=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.txt

[13:06:39][TeamCity.SpecFlow.Reporting]      --result=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.xml;format=nunit2

[13:06:39][TeamCity.SpecFlow.Reporting] 

[13:06:39][TeamCity.SpecFlow.Reporting] System.NotSupportedException: The given path's format is not supported.

[13:06:39][TeamCity.SpecFlow.Reporting]    at System.Security.Permissions.FileIOPermission.QuickDemand(FileIOPermissionAccess access, String fullPath, Boolean checkForDuplicates, Boolean needFullPath)

[13:06:39][TeamCity.SpecFlow.Reporting]    at NUnit.Engine.TestPackage..ctor(String filePath)

[13:06:39][TeamCity.SpecFlow.Reporting]    at NUnit.Engine.TestPackage..ctor(IList`1 testFiles)

[13:06:39][TeamCity.SpecFlow.Reporting]    at NUnit.ConsoleRunner.ConsoleRunner.MakeTestPackage(ConsoleOptions options)

[13:06:39][TeamCity.SpecFlow.Reporting]    at NUnit.ConsoleRunner.ConsoleRunner.Execute()

[13:06:39][TeamCity.SpecFlow.Reporting]    at NUnit.ConsoleRunner.Program.Main(String[] args)

[13:06:39][TeamCity.SpecFlow.Reporting]

The NUnit command line constructed by the PowerShell script (reported on the line above containing the test "Invoking nunit with") was:

C:\BuildAgent\work\94f18887f3572298\Geo\Source\Packages\NUnit.ConsoleRunner.3.2.0\tools\nunit3-console.exe

C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs\Arieso.Data.AdvancedDiagnostics.Specs.dll

--labels=On

--out=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.txt

--result=C:\BuildAgent\work\94f18887f3572298\Geo\Source\GeoBuildRelease\Specs\Arieso.Data.AdvancedDiagnostics.Specs.xml;format=nunit2

This same command line runs without issue when executed directly from within a command prompt on the build agent following the failure.

The source code for the Microsoft 'QuickDemand' API shows that the path is probably considered to be invalid because it contains at least one ':' character after index 1 within whatever path it is working with.

I got no additional information with the PowerShell script modified to cause the NUnit command line to include the --verbose and --trace=Verbose switches.  (The output generated by such switches is included in STD out, right?)

Does anyone know what is causing this or a work-around?  Has it been seen before?  Is it something to do with the tight integration between the v3.2 console runner and TeamCity (introduced recently)?

I then tried running the SpecFlow test fixtures directly via an NUnit task within TeamCity.  This worked but was unacceptably slow.  The time for a CI build increased from 14 minutes to over 90.

SpecFlow scenarios and features cause a lot of text to be written to STD out when the console runner executes them, and as things stand with the new integration between the runner and TeamCity all of the text is being inserted into the run log on the TeamCity server.

The run log is HUGE now and I suspect that it is this transfer of text from the console runner to the TeamCity agent and then to the TeamCity server that is slowing the run down.

Is there any way to disable the integration between the NUnit runner and TeamCity?

Thanks

David Razzetti

[Charlie Poole]

The NUnit internal trace log is located in your
AppData\Local\NUnit\logs directory.

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to nunit-discus...@googlegroups.com.
> To post to this group, send email to nunit-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/nunit-discuss.
> For more options, visit https://groups.google.com/d/optout.