Exception thrown running NUnit.ConsoleRunner against plain simple nunit propject.

[Mike Nowak]

Hi,

We are in the process of migrating from NUnit 2.6.2 to the latest available in nuget.org which is 3.7.1. We have also updated the target framework of our projects to .NET 4.7. Before making those updates we were already using nunit3-console.exe in version 3.2.1 with default addins.

After updating target framework and NUnit to the latest version as well as fixing some of the tests, everything seems to run fine on in the ReSharper Test Runner. When we use Nunit.Console 3.7.0 to run tests from command line like this:
./nunit3-console.exe C:\dev\Path\To\Test\Project\bin\Release\Utilities.Tests.dll
everything seems to work fine. Same for the other test projects in our solutions.

The issues start when we create a simple nunit project, like this (is this a valid nunit project file?):

<NUnitProject>
    <Settings activeconfig="Release"/>
    <Config name="Release">
        <assembly path="C:\dev\Path\To\Test\Project\bin\Release\Utilities.Tests.dll"/>       
    </Config>
</NUnitProject>

./nunit3-console.exe tests.nunit

We are getting exception as a result:

NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Microsoft Windows NT 10.0.15063.0
  CLR Version: 4.0.30319.42000

Test Files
    tests.nunit

Errors, Failures and Warnings

1) Error : C:\dev\Path\To\Test\Project\tools\NUnit.ConsoleRunner.3.7.0\tools\tests.nunit
An exception occurred in the driver while loading tests.

Server stack trace:
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at NUnit.Engine.ITestEngineRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.ProcessRunner.RunTests(ITestEventListener listener, TestFilter filter)

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2017-07-14 18:00:03Z
    End time: 2017-07-14 18:00:03Z
    Duration: 0.454 seconds

Results (nunit3) saved as TestResult.xml

I believe this is NUnitEngineException thrown in NUnit.Engine.Runners.DirectTestRunner.cs

This issue doesn't happen if we run the newest 3.7.0 Runner against our code before the update of NUnit and .NET Framework. Same nunit project file is being generated in both cases. I believe we are missing something simple and obvious, since as I mentioned before, running nunit3-console.exe with path to the same test assembly runs fine. Hope someone had similar issue or have an idea.

Best Regards,
Mike

[Charlie Poole]

It's possible that the nunit-project-loader extension is not installed. Run nunit3-console --list-extensions to be sure.

--
You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nunit-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to nunit-...@googlegroups.com.
Visit this group at https://groups.google.com/group/nunit-discuss.
For more options, visit https://groups.google.com/d/optout.

[Mike Nowak]

Hi,

Thanks for your reply Charlie and sorry for the late answer. I didn't get notified that you have replied to my issue.

I have run the "nunit3-console --list-extensions" command, I believe it is set up correctly, here is the result:

NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Microsoft Windows NT 10.0.15063.0
  CLR Version: 4.0.30319.42000

Installed Extensions
  Extension Point: /NUnit/Engine/NUnitV2Driver
    Extension: NUnit.Engine.Drivers.NUnit2FrameworkDriver
  Extension Point: /NUnit/Engine/TypeExtensions/IService
  Extension Point: /NUnit/Engine/TypeExtensions/ITestEventListener
  Extension Point: /NUnit/Engine/TypeExtensions/IDriverFactory
  Extension Point: /NUnit/Engine/TypeExtensions/IProjectLoader
    Extension: NUnit.Engine.Services.ProjectLoaders.NUnitProjectLoader
      FileExtension: .nunit
    Extension: NUnit.Engine.Services.ProjectLoaders.VisualStudioProjectLoader
      FileExtension: .sln .csproj .vbproj .vjsproj .vcproj .fsproj
  Extension Point: /NUnit/Engine/TypeExtensions/IResultWriter
    Extension: NUnit.Engine.Addins.NUnit2XmlResultWriter
      Format: nunit2

As I have mentioned earlier, same setup against our older code base seems to be working fine - same console-runner, same setup. We started getting this with .NET 4.7 and nunit 3.
Did you manage to trace the source of the exception? Is this coming from "NUnit.Engine.Runners.DirectTestRunner.cs"?

If there anything more I could help with to track the issue please let me know?

Best regards,
Mike

[Rob Prouse]

Mike,

Does your test run without the NUnit project file when you just pass the test assembly to the console runner? Also, try using paths relative to the project file within the project file. The docs state, "If you edit the XML manually, you should make sure to use a path relative to the directory containing the project file as the location of any assemblies."

Not many people use the project files these days, so they are not well tested.

Rob

--

Rob Prouse