Slow test execution

[Przemek Piotrowski]

Hi all.

I've just joined to project developed in Play Framework and MongoDB. There is about 50 tests using play.api.test.WithApplication (and few unit tests). It takes about 200 seconds to execute them
They looks like example:

"do sth" in new WithApplication {
      val result = route(someCustomRequest(GET, "/some/url")).get

      status(result) must equalTo(OK)
      contentAsString(result) must contain("""form action="/some/url" method="POST"""")
    }

Moreover there is plenty of that entries in test log.

[info] play - Starting application default Akka system.
[info] play - Shutdown application default Akka system.
[info] play - Starting application default Akka system.
[info] play - Shutdown application default Akka system.
[info] play - Starting application default Akka system.
[info] play - Shutdown application default Akka system.

I think, it comes from play.api.libs.concurrent.AkkaPlugin. We cooperate with that plugin with code similar to this:

class Actors(implicit app: Application) extends Plugin {
  lazy val mailSender = play.api.libs.concurrent.Akka.system.actorOf(Props[MailSender], name = "mailSender")
  override def onStart() = {mailSender}
}
object Actors {
  def plugin = Play.current.plugin[Actors].getOrElse(throw new RuntimeException("Actors plugin not loaded"))
  def mailSender = plugin.mailSender
}

Could somebody explain me what we are doing wrong or is it just normal execution time and steps in testing play.
There is a chance that this strange behavior comes from https://github.com/athieriot/specs2-embedmongo used in project. If it is so, I'm sorry.

[Ian Rae]

This is a recurring problem with Play. Many people combine tests so that a single fakeapplication runs many tests, with code to reset db tables in between.

Some people feel that testing against anything else than the real db is a waste of time. However if your controllers are nnontrivial, unit testing against an in-memory db can be worthwhile. I am working on a framework (still experimentalal) that moves your app code from controllers to presenters where they can be tested against a very fast mock db.

[Pathikrit Bhowmick]

I am also having same issue: https://groups.google.com/forum/#!searchin/play-framework/slow$20akka/play-framework/ecZVe5pA8eo/V3hO3NJ6QxEJ

How did you solve it?? We have hundreds of test cases each booting up a fake play app and we spend 90% of our time in Akka shutdown!!

[Ian Rae]

Solved it through avoidance. Only two of my hundred test cases use FakeApplication.  All others test app logic in layers that are outside Play.  There are tradeoffs to this approach but slow unit tests is not one of them.  An earlier version of this is available at: https://github.com/ianrae/mettle-framework

[Josh Padnick]

We are still in the process of building our app, and are just starting to ramp up our test-driven development.  We are writing tests for most of our DAO's and service methods, and you can see an example of our tests at https://groups.google.com/d/msg/jooq-user/GkBW5ZGdgwQ/kSqDH89zMSYJ.

As you can see from the example, we use fakeApplication() for every DAO test.  In my local dev environment, I'm testing against a MacBook Pro Retina running a Postgres VM, and so far, the performance seems like it will be ok:

[info] Passed: Total 31, Failed 0, Errors 0, Passed 31

[success] Total time: 8 s, completed Sep 19, 2014 6:58:26 PM

Assuming the same ratio of 31 tests in 8 seconds (~0.25 seconds per test), at 500 tests, we'll be right around 2 minutes.  We're not seeing the kind of slow response time that others in this post have reported.  My feeling is that as the number of tests rises, we can partition these or run them in parallel in our build environment.  I appreciate James chiming in on how to debug slow tests; that will surely come in handy later.

Josh

[Ian Rae]

For integration tests 2 minutes is good.  For unit tests, 8 seconds is probably on the high side, and may cause developers to avoid running them.  TDD is about rapid feedback. See this article: http://blog.ploeh.dk/2012/05/24/TDDtestsuitesshouldrunin10secondsorless/

I believe there are ways to run multiple tests per fakeApplication instance, with @Before and @After method to clean database tables between tests.  A bit more work, but probably a large speedup.