ParallelTestExecution doesn’t seem to work well with dynamically created tests

[Daniel Neades]

Hello,

I am endeavouring to use ParallelTestExecution with dynamically created tests, along these lines:

final class MyParallelTestSuite extends FunSuite with ParallelTestExecution {
  for {
    a <- as
    b <- bs
    c <- cs
  } test(makeSomeName(a, b, c)) {
    //...do something...
  }
}

The for loop generates many thousands of tests, which takes a non-trivial amount of time (a sizeable fraction of a second) – the initial test generation thus takes much longer than a typical test takes to run.

The problem I am having is that the tests in this parallel test suite run glacially slowly – far, far slower than they do in a normal, sequential test suite. I believe this may be because each test in a ParallelTestExecution suite is given its own instance of the test suite class, and each such instance therefore presumably dynamically (and slowly) creates *all* the tests by running the for loop.

Questions:

1. Is there a another, recommended way to approach the problem of parallelizing dynamically created tests?

2. Alternatively, if I am diagnosing the problem correctly, is there a way to prevent ParallelTestExecution from instantiating a separate instance of the test suite for each test? (I am thinking there isn’t, based on other people asking questions about this, and also from having looked at the source code.)

Many thanks in advance for any assistance.

-- 

Daniel

[Bill Venners]

Hi Daniel,

ParallelTestExecution will run each test in its own instance to make it less likely that different threads to access shared mutable state (in instance vars or vals holding references to mutable object), which means people don't need to think as much about synchronizing access to that state (which often I figure they wouldn't think about anyway in the heat of battle). 

I'd suggest instead of making tests dynamically you make nested suites dynamically, and put one test in each instance. You don't need to use ParallelTestExecution because Suites run in parallel by default so long as parallel is enabled (which it is by default in sbt in case you're using sbt). You might try something like:

import org.salatest.Suites

class MyParallelTestSuite extends Suites(

  for {

    a <- as,

    b <- bs,

    c <- cs

  } yield {

    new FunSuite {

      test(makeSomeName(a, b, c)) {

        // ...do something...

      }

    }

  }: _*

)

Except I just realized you'll probably hit the JVM's limit on how many parameters you can pass to a constructor. You can instead either make a tree, such as one nested suite for each A that has one nested suite for each B, which has one nested suite for each C. Or just extend FunSuite and override nestedSuites, something like this:

class MyParallelTestSuite extends FunSuite {

  override def nestedSuites: IndexedSeq[Suite] = 

     for {

    a <- as,

    b <- bs,

    c <- cs

  } yield {

    new FunSuite {

      test(makeSomeName(a, b, c)) {

        // ...do something...

      }

    }

  }

}

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalatest-users@googlegroups.com
To unsubscribe from this group, send email to
scalatest-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Bill Venners
Artima, Inc.
http://www.artima.com

[Daniel Neades]

Hi Bill,

Thank you for the very helpful response – that does give me some things to try!

(Incidentally, I do understand why ParallelTestExecution creates a new instance for each test, and that obviously makes sense for many circumstances. In my particular case, however, the entire test state is immutable, so I do not need separate instances.)

Thanks again,

-- 

Daniel

To post to this group, send email to scalate...@googlegroups.com

To unsubscribe from this group, send email to

scalatest-use...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.