Single Failure in Inherited BeforeMethod Causes All Subsequent Invocations to be Skipped

[gyrm]

Hi,

I'm seeing that if an inherited @BeforeMethod fails a single time, it
causes all subsequent invocations to be skipped, thus also skipping
their associated tests. This is not the behavior I want - I would like
the specific test to be skipped, but for any other tests to still be
run, as it may simply be the case that a sporadic error was
encountered.

Is the current behavior (skip all future invocations) intended? It
does not seem to be well documented (I couldn't find anything about
it). If so, is there any way to override it? It seems desirable to be
able to configure this behavior on a per-method basis.

I was able to find this issue ticket and associated thread describing
similar behavior, but related to Factory generated tests:
http://jira.opensymphony.com/browse/TESTNG-202

Here is the Groovy code I used to demonstrate this to myself (just to
give an idea):


<< SomeAbstractTest.groovy >>

import com.googlecode.lightest.core.LightestTestCase
import org.testng.annotations.*

class SomeAbstractTest extends LightestTestCase {
boolean forceFailure = false

@BeforeMethod (alwaysRun = true)
void prepareTest() {
if (forceFailure) {
throw new Exception('Failure was forced!')
}

println 'Test is now prepared.'
}
}


<< SomeTest1.groovy >>

import org.testng.annotations.*

class SomeTest1 extends SomeAbstractTest {

SomeTest1() {
forceFailure = true
}

@Test
void doTheTest() {
println '1'
}
}


<< SomeTest2.groovy >>

import org.testng.annotations.*

class SomeTest2 extends SomeAbstractTest {

@Test
void doTheTest() {
println '2'
}
}


The output indicates that the both test methods are skipped:


===============================================
Suite1
Total tests run: 2, Failures: 0, Skips: 2
Configuration Failures: 1, Skips: 1
===============================================


Any insight would be appreciated - thanks!


Haw-Bin

[Cédric Beust ♔]

Hi Haw-Bin,

Yes, this is the intended behavior.  The assumption here is that if @BeforeMethod fails once, it's probably going to fail for all the subsequent attempts as well, so there is no point trying to run it again.  If your method can fail sporadically, you should probably work around it inside the method itself...

--

Cédric

[gyrm]

Hi Cedric,

Thanks for the reply - it makes sense to simply handle the error in
such a way that it does not cause the method to fail. I'll give it a
shot anyhow - thanks again!


Haw-Bin

On May 13, 12:28 pm, Cédric Beust ♔ <cbe...@google.com> wrote:
> Hi Haw-Bin,
>
> Yes, this is the intended behavior.  The assumption here is that if
> @BeforeMethod fails once, it's probably going to fail for all the subsequent
> attempts as well, so there is no point trying to run it again.  If your
> method can fail sporadically, you should probably work around it inside the
> method itself...
>
> --

> ***Cédric
> *