Fail Fast

[Matt]

Is there any way to configure ScalaTest to stop on the first test failure?  Often when I'm making changes to a test, either my fix is going to fix everything or nothing. 

[Bill Venners]

Hi Matt,

We've been using a trait in our Stairway to Scala course exercises that does that. It does it for just the suite you mix it into. Is that the kind of behavior you want, or do you want an entire run to abort after the first failure perhaps?

Here's the trait we are using:

import org.scalatest._
import events.Event
import events.TestSucceeded
import scala.collection.mutable.ListBuffer

trait StopOnFirstFailure extends SuiteMixin { this: Suite =>

  override def runTests(testName : Option[String], args: Args): Status = {

    import args._

    val stopRequested = stopper

    // If a testName is passed to run, just run that, else run the tests returned
    // by testNames.
    testName match {
      case Some(tn) => runTest(tn, args)
      case None =>
        val statusList = new ListBuffer[Status]()
        val tests = testNames.iterator
        var failed = false
        for (test <- tests) {
          if (failed == false) {
            val status = runTest(test, args)
            statusList += status
            failed = !status.succeeds()
          }
        }
        new CompositeStatus(Set.empty ++ statusList)
    }
  }
}

This isn't in ScalaTest itself because no one ever asked for this behavior until today, so I wasn't sure whether it was generally needed. It is useful for training people, because the students can just focus on the next failed test. Not sure still how common this need is generally.

Bill

On Fri, Feb 21, 2014 at 12:44 PM, Matt <hughe...@gmail.com> wrote:

Is there any way to configure ScalaTest to stop on the first test failure?  Often when I'm making changes to a test, either my fix is going to fix everything or nothing. 

--
You received this message because you are subscribed to the Google
Groups "scalatest-users" group.
To post to this group, send email to scalate...@googlegroups.com
To unsubscribe from this group, send email to
scalatest-use...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/scalatest-users?hl=en
ScalaTest itself, and documentation, is available here:
http://www.artima.com/scalatest
---
You received this message because you are subscribed to the Google Groups "scalatest-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalatest-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Bill Venners
Artima, Inc.
http://www.artima.com

[Bill Venners]

Hi Matt,

It occurred to me right after I sent the previous email that perhaps you'd be happy with canceling after the first failure instead of stopping. That has been asked for in the past and is in this trait:

http://doc.scalatest.org/2.0/index.html#org.scalatest.CancelAfterFailure

The reason I felt canceling was better was because you see all the tests that were canceled in the report. If this gets left in and happens in production, you won't be missing tests that silently get skipped.

Bill