Running tests in parallel

[George Cover]

I'm trying to run the tests in parallel, and using the parallelizable attribute with a scope of either fixtures or children I can see in VS that multiple tests are executed at the same time.

However, when setting the scope to self, it appears that the tests are executed serially. I've tried a couple of different versions of NUnit3 (3.4.1, 3.5), along with both the Nuget and VSix adapters.

Does anyone have any ideas about what might be happening here? Why would one scope not work, whilst others seem fine. Is there perhaps something in the lifecycle of the test that NUnit picks up on and refuses to run in parallel?

For full disclosure: I'm running SpecFlow scenarios, so the nunit fixtures are generated. I've cross posted on the group over there as well to see if anyone has any input.

https://groups.google.com/forum/#!topic/specflow/SC3r4V8mvag

Thanks

George (@TheOptimist)

[Charlie Poole]

Where are you placing the Attribute?

--
You received this message because you are subscribed to the Google Groups "NUnit-Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nunit-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to nunit-...@googlegroups.com.
Visit this group at https://groups.google.com/group/nunit-discuss.
For more options, visit https://groups.google.com/d/optout.

[George Cover]

In AssemblyInfo.cs.

[Charlie Poole]

ParallelScope.Self means that the test may run in parallel. It has no effect on descendants.

Placing it on the assembly therefore has no meaning. The assembly is a test that contains all your tests so there is nothing with which it can run in parallel.

We should probably give a warning if Self is used at the assembly level.

On Nov 23, 2016 5:35 AM, "George Cover" <george...@gmail.com> wrote:

In AssemblyInfo.cs.

[George Cover]

Thanks Charlie, I was starting to think that might be the case. I'll head back to the SpecFlow guys to see how we can incorporate something in the generator to put this attribute on each test...

Or, I'll look at the Children scope (although some of my tests fail when run in parallel at Children scope)....

[George Cover]

For clarification, if I set the scope of parallel at the assembly level and use Children scope, does that mean "Fixtures" or "Tests"?

SpecFlow Feature => NUnit.TestFixture

SpecFlow Scenario => NUnit.TestCase

Specflow generates a class per feature, with each test being a TestCase attributed method. The docs (https://github.com/nunit/docs/wiki/Parallelizable-Attribute) say that parallel execution of methods within a class is not yet implemented.

Am I being too eager to run my tests in parallel (given a SpecFlow Scenario is really a method in a class)?

[Charlie Poole]

Anything you set at the assembly level can be overridden at a lower
level, on a class for example. What you place on the assembly level
should generally depend on what you want the default to be for newly
added tests. I find Fixture scope to be a good compromise in the tests
of NUnit itself.

Children means anything below this point can run in parallel. It would
be risky for any tests that use shared state in the class. Of course,
at the present time, it is a synonym for Fixture because we don't yet
implement parallelism below the fixture level.

It seems reasonable to me that SpecFlow might provide an option to run
Features in parallel, by adding the attribute to the generated class.
However, their users might also want a way to turn off parallelism for
certain features. Something similar could be done for Scenarios when
we actually implement parallel test case execution.

Charlie

> --
> You received this message because you are subscribed to the Google Groups
> "NUnit-Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to nunit-discus...@googlegroups.com.