running NUnit tests in parallel

[fschwiet]

  Hi, what is the best way to run NUnit tests in parallel these days?  People working on the project are accustomed to using the Resharper test runner, so I don't know that moving to NUnitLite is an alternative (though I haven't tried it). 

  Though people do not run the tests in parallel with Resharper, they do run in parallel in the build process with a custom test runner.  I don't think its ideal as the tests aren't as isolated as they should be (static variables seem to be shared without extra work).

[Simone Busoli]

Hi,

in my experience I rarely needed to run unit tests in parallel during development as I could limit the tests run to those interested in the piece of code I was working on, but I understand that it might be useful in other scenarios. Currently NUnit has no way to run tests in parallel, you will most likely have to wait for NUnit 3 to have this feature. I think Charlie has already made some good progress on it and it may also happen that this becomes available in v2 as well.

As for during the build process instead, when needed I split the tests into several assemblies (categories work just as well though) and ran them through independent builds by assigning a subset of the whole test suite to each build.

Let me say that R# is definitely not my favorite runner inside VS though, I always preferred TestDriven to it, less cluttering of the user interface and way faster.

Simone

On Tue, Oct 9, 2012 at 10:45 PM, fschwiet <fsch...@gmail.com> wrote:

  Hi, what is the best way to run NUnit tests in parallel these days?  People working on the project are accustomed to using the Resharper test runner, so I don't know that moving to NUnitLite is an alternative (though I haven't tried it). 

  Though people do not run the tests in parallel with Resharper, they do run in parallel in the build process with a custom test runner.  I don't think its ideal as the tests aren't as isolated as they should be (static variables seem to be shared without extra work).

--
You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/nunit-discuss/-/Zak5TLijKfcJ.
To post to this group, send email to nunit-...@googlegroups.com.
To unsubscribe from this group, send email to nunit-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nunit-discuss?hl=en.

[Charlie Poole]

As Simone says, you can't do it with current NUnit, except at a very
high level - by running multiple assemblies in parallel in batch mode.

NUnitLite currently has an Asynchronous attribute, but that's intended
for tests that have to wait for something external to happen rather than
as a general solution for parallel execution. I don't expect to extend
full parallel execution to NUnitLite anyway.

I have code for NUnit 3.0, not yet checked in, which appears to do
a fine job of running tests on parallel threads. I've used up to 8 on
a two-processor machine and it ran about five times faster than the
using a single thread of execution. Unfortunately, for this to be useful,
you have to take great care to make the tests absolutely independent.
And, of course it's not yet released.

R# won't run NUnitLite tests - no surprise, since NUnit itself won't
run them at this time either!

The future for parallel execution rests with NUnit 3.0, which will support
both thread-level and process-level parallelization. The thread-level work
part is almost ready to checkin, but the process-level has not even
been started yet.

Charlie

[John Brett]

For development, I've been very impressed with NCrunch. If you're working in a pure unit-test environment and looking to run your tests during

development it does an excellent job of simulating parallel test execution (it instantiates multiple test runners, if I've understood Charlie correctly),

and will also have a go at dependency tracing and thereby execute only affected tests.

For build machine execution of tests, I've had poor yields in the past from parallel execution, so tend to throw hardware at the problem nowadays[1].

John

IIRC, I was always ending up with I/O or memory bottlenecks, and the extra CPUs didn't really help. To be fair, there was a spectrum of test types

from unit to system, and they often don't scale well.