Abort sequential tests if specific test fails.

[Everson Alves da Silva]

Hello,

I have a spec that should be executed sequentially but there one of the tests that, if it fails, all the following tests will always fail. So is there a way to abort all tests in that case? Adding a condition for each subsequent test is not optimal.

[etorreborre]

You can use the `stopOnFail` argument at the beginning of your specification.

Eric.

[Everson Alves da Silva]

Thank you Eric, but I mean if there's how to apply that to a specific test:

MySpec extends mutable.Specification {

args(sequential = true)

"this might or not fail. continue" >> {...}

"if this fail, abort it." >> {...}

"only reached if previous does not fail." >> {...}

[etorreborre]

Hi, 

I propose the following addition to specs2 (you can try that in the latest 1.12-SNAPSHOT):

  MySpec extends mutable.Specification {

    "this might or not fail. continue" >> {...}

    "if this fail, abort it." >> {...}

    step(stopOnFail = true)

    

    "only reached if previous does not fail." >> {...}

  }

This will actually work whether the specification is sequential or not. The next group of examples will be skipped if there is a failure in the previous group of examples.

Does that work for you?

[Everson Alves da Silva]

Hello Eric. Well, it helps but does not completely solve my issue. You see, I have to abort only if one (maybe two), example fails among dozens of them. The way I came up until now is using AroundExample test with a boolean flag whether I should or not execute the example.

Thanks for the help.

[etorreborre]

Can you please give a small self-contained example of what you're doing with the AroundExample context?

If I understand correctly, you want to abort the rest of the specification if, say, one of 2 examples fail. You can either write:

   "ex1" >> ok

   "ex2" >> ok

    

    "ex3" >> ko

     step(stopOnFail=true)

    "ex4" >> ok

Then ex4 will be skipped if ex1, ex2 or ex3 fails. 

Or you can write:

   "ex1" >> ok

   "ex2" >> ok

    

     step()

    "ex3" >> ko

     step(stopOnFail=true)

    "ex4" >> ok

which will abort the rest of the spec (i.e. ex4) only if ex3 fail. To make this idiom clearer you can code something like:

   def checkpoint(ex: =>Example) = {

     step()

     ex

     step(stopOnFail = true)

   }

   "ex1" >> ok

   "ex2" >> ok

    

     checkpoint { 

       "ex3" >> ko

     }

    "ex4" >> ok

But that might not be what you want so if you can give a short example + the expected console output that would help me understand better.

Thanks!

[Everson Alves da Silva]

Oh, great, now your sample solves my issue. In the previous response I got the impression that the would stop if the whole block of examples had any failed test. The idiom was not clear to me but the last one example rocks. Thanks again.

The AroundExample was just a workaround referring to a global var which was set manually(e.g. testWillFail = value != expectedValued). You see, the whole specification takes 50s (and counting) to run and that critical test happens around 10s. that's why I want abort it.

 

again, many thanks! And, nice I have the opportunity to say, congratulations on this amazing framework :), specs2 rocks!