I'm running into this problem, where it's really important that a listener's "onTestStart" does NOT execute if the configuration failed, i.e. if the @BeforeClass method failed.
However, I am seeing that onTestStart still runs, even though in the final test report the test in question is skipped.
For example, the code is very simple:
public class MyListener implements ITestListener {
@Override
public void onTestStart(ITestResult result) {
System.out.printlnt("Running onTestStart...");
}
...implement the other methods...
}
public class MyTestClass {
@BeforeClass
public void setup() {
throw new RuntimeException("Setup failed!");
}
@Test
public void doTest() {
System.out.println("Executing test...");
}
}
Now, this is annoying, because the listener actually opens a web browser in a remote Selenium Grid, and we don't want to waste these resources if the configuration failed!!