SpecRun skipping testing rather than running them when executing more than 16

[Sean Rand]

Good Morning

I've changed my Runner to be SpecRun (1.9.0) within my solution, evaluation version of SpecRun I'm using.  I have around 200 specflow tests which I usually run with MSTest but I want to try to utilise SpecRun's power of running in parallel but every time I try to run my 200 or so tests it will execute 16 of them and skip the rest with no reason as to why within SpecRun?

I can't see anywhere on the evaluation version which states only batches of a certain number of tests can be run at anyone time and it's really looking as though SpecRun isn't up to the job possibly

http://www.specflow.org/plus/evaluation/

Really looking for some help by the community in understanding why SpecRun will skip tests however something like MSTest or NUnit will run them every single time successfully,

Many thanks

Sean

[Gáspár Nagy]

Hi,

There is no such restriction for the evaluation mode. The skipped tests might be caused for another reason.

The default SpecRun profile configured in a way, that it stops after a certain number of failures and skips the rest of the tests. Can it be that this was the reason in your case?

This behavior can be set in the "stopAfterFailures" attribute of the "Execution" element in the profile. To disable this behavior, you have to set it to 0.
  <Execution stopAfterFailures="0" />

Br,
Gaspar

[Priti Fhuse]

how to run in parallel

[Patrick Karcher]

I am also have this issue.  For me it is about 5% of tests that mysteriously are skipped when running in parallel.  When I run on just 1 thread, I never have any "Skipped" tests.  When I run on 2 or more threads, I usually have a few Skipped.  I'd love for it to stop.

-Patrick

[Stephen McCafferty]

What have you set stopAfterFailures to in your .srprofle? It's possible that tests are being skipped because the number of failures has been reached, and this may differ when using multiple threads if you have conflicts with the different threads accessing the same data.

Can you try setting stopAfterFailures to 0 (which disables this function) and see if you are still getting skipped tests?

Note that this value defaults to 10 if not specified. The default value in the .srprofile included in the SpecRun package is 3.