How to run suites sequentially (not concurrently), while its nested suite classes run in parallel

22 views
Skip to first unread message

Tom Work

unread,
Jun 4, 2020, 9:06:41 AM6/4/20
to scalatest-users
Hi all!

This is my first email to the list.
I've been working with ScalaTest for a few years and have some suites of tests running in parallel and other ones sequentially.
At the moment, I run tests on TeamCity and it's set up to run in 2 different steps, to avoid some conflicts in data that tests change
  • Step1: Changes system configurations (+/- 800 tests)
  • Step2: Tests the business logic of the application (+/- 5500 tests)

Recently we found more things to test that cannot be run at the same time as the Step1 or Step2 and we would like to remove the need of the configuration on TeamCity to have 2 different steps. We want to make it just one step.
To do that, I would need to run basically 2 big suites of tests, each one of them with many nested suites inside.
I would like to run the 2 big suites sequentially, while all its nested suites in parallel.

In terms of code, it should work like this:

// Below the FunSpec classes of tests
class Spec1 extends FunSpec {}  
class Spec2 extends FunSpec {}  
class Spec3 extends FunSpec {}  
class Spec4 extends FunSpec {}
 


// Belwo the Suites classes
class Step1Suite extends Suites(Seq(new Spec1(), new Spec2()))
class Step2Suite extends Suites(Seq(new Spec3(), new Spec4()))


class MainModuleSuite(Seq(new Step1Suite(), new Step2Suite())) extends SequentialNestedSuiteExecution


The problem above is that when mixing in SequentialNestedSuiteExecution, it makes all nested classes run sequentially (not in parallel).

The expected behaviour would be:
  1. Start running all specs inside class Step1Suite in parallel, in this case, Spec1 and Spec2.
  2. Do not start running any spec inside class Step2Suite (or any spec that is not in Step1Suite), in this case, Spec3 and Spec4.
  3. Finish running all specs in class Step1Suite.  (wait for all instances of nested suites from Step1Suite be finished)
  4. Start running all specs inside class Step2Suite, in this case, Spec3 and Spec4.
  5. And so on...

Is it possible to do it in ScalaTest, once everything turns out to be of type Suite?

I appreciate any help on this.

Thanks!

Tom
Reply all
Reply to author
Forward
0 new messages