How to run asynchronous tests concurrently without ParallelTestExecution?

82 views
Skip to first unread message

sub...@gmail.com

unread,
Sep 29, 2016, 8:19:14 PM9/29/16
to scalatest-users
Hi, 

I've been working with several tests that deal with Futures. I'd like to run the suite asynchronously but I can only seem to do this by introducing ParallelTestExecution and -P(number of tests). I've tried using both AsyncFlatSpec and ScalaFutures.whenReady but in all cases the tests run sequentially, so it's unclear to me why one would use either of those since you could just Await on the future with the same result. I can understand why ParallelTestExecution would be useful to execute a suite of synchronous tests concurrently but I'm already dealing with asynchronous tests/futures, so that doesn't make much sense. I can do the following but it's not pretty. Note, this is not using ParallelTestExecution, so single-threaded.

val queue = new mutable.Queue[Future[String]]

test1:
val future1 = ...
queue += future1

test2:
val future2 = ...
queue += future2

test3:
"await all" in {
for (future <- queue)
ScalaFutures.whenReady(future, patience) { s =>
s should
??
}
}

This works in that the none of the tests block and the suite takes only as long as the longest test, but it's clearly ugly and the assertions do not occur in the context of the right test.

So, I'd be very interested in finding out if there any way in ScalaTest to execute these asynchronous tests without blocking until the futures of all tests have been collected, without using ParallelTestExecution.

Thanks,

Andrew
 


Bill Venners

unread,
Sep 29, 2016, 8:24:59 PM9/29/16
to scalate...@googlegroups.com
Hi,

ParallelTestExecution is the way to do it. The reason we run them one after another by default is to prevent too many tests running at once, which could cause timeouts. They are asynchronous. The act of starting the second test is registered as a callback to be executed when the future of the first test completes. The act of starting the third test is registered as a callback on the second test's future, etc. They are chained futures in other words.

For most test suites my expectation is that running suites in parallel is sufficient parallelism to get the maximum or at least satisfactory performance. If in a few suites you really want tests to run in parallel, whether they be sync or async tests, you can achieve that by mixing in ParallelTestExecution.

Is there a reason you don't want to mix in ParallelTestExecution?

Bill

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalatest-users@googlegroups.com
To unsubscribe from this group, send email to
scalatest-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Bill Venners
Artima, Inc.
http://www.artima.com

sub...@gmail.com

unread,
Sep 30, 2016, 9:25:11 AM9/30/16
to scalatest-users
Hi Bill,

Well I don't want to create additional threads when I already have Futures that occupy threads, efficiently in some cases with an event loop (Finagle). For context, these tests are streaming http, so if I have 10 tests that stream for 10s I want the suite to complete in roughly 10s, without having to think about the what the -P parameter should be every time I add or remove a test. I haven't looked at the ScalaTest code but it seems to be, much in the way that ParallelTestExecution executes each test in a thread and returns a future, I should be able to map my future with the same type (Assertion I'd guess) and return that. The ScalaTest code that deals with the Futures should operate the same, whether a thread was spawned for the test or not. I found the AsyncFlatSpec confusing because I could simply Await.result(Future) in the test with exact same concurrency characteristics. Thanks for your time.

Andrew

To post to this group, send email to scalate...@googlegroups.com

To unsubscribe from this group, send email to

For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages