Should "next" @BeforetMethod be skipped if "previous" @AfterMethod failed?

[Tarun Bhadauria]

This is my super class set up which is extended by all test classes -

• a couple of @BeforeClass
• one @BeforeMethod - which starts Selenium 2
• one @AfterMethod - which shuts Selenium 2

Now when @AfterMethod fails, like -  browser is shut down for what so ever reason before Selenium 2 could actually kill it then I notice that all of @BeforeMethod are skipped
and this eventually skips all the tests.

So it looks to me that previous @AfterMethod causes my next @BeforeMethod skip.

I am using testng 5.12.1 because of the the issue I had mentioned here - http://groups.google.com/group/testng-users/browse_thread/thread/bfb86a7d6f056a72

~ tarun

[Cédric Beust ♔]

If a @BeforeMethod or an @AfterMethod fails, all the methods remaining in this class are automatically skipped, since the assumption is that your environment is now in an unstable state.

-- 

Cédric

~ tarun

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/R6JN58vyJqsJ.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

[Tarun Bhadauria]

but why does it have to skip all the methods. I was thinking that it should skip only one method if @Before fails.

I have intermittent browser failure with one test method but then it cause other tests to be skipped too even if they are not affected by the failure of this one test method.

Is there any way to override this behaviour?

[Cédric Beust ♔]

@Before/@AfterMethod applies to all your test methods, so it makes sense to assume that they will all be returning bogus results from now on.

If it's not a fatal error to you, just catch the exception so TestNG won't see it.

-- 

Cédric

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/p2hi1Wqo9RsJ.

[Tarun Bhadauria]

I can do that but was wondering that if a setup failed once should mean setup is going to fail forever hence setup should be skipped.

Or if setup should be allowed to take place and setup failed once should not affect next cycle of same setup. Considering it may not always fail?

[Cédric Beust ♔]

It only needs to fail once to put your entire test suite in jeopardy.

And TestNG is smart about this by failing only what is relevant to the configuration method that failed: if it's @AfterMethod/@BeforeMethod, only test methods wrapped by these will be skipped. If it's a @BeforeClass/@AfterClass, the entire class becomes skipped, etc...

-- 

Cédric

On Sun, Dec 11, 2011 at 10:23 AM, Tarun Bhadauria <tku...@gmail.com> wrote:

I can do that but was wondering that if a setup failed once should mean setup is going to fail forever hence setup should be skipped.

Or if setup should be allowed to take place and setup failed once should not affect next cycle of same setup. Considering it may not always fail?

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/xD3hKcVxSwEJ.

[Tarun Bhadauria]

Thanks for your comments Cedric.

I guess I see different behavior .

For example I have two classes and each test method in those classes are run independently. I have @Before/@After methods which in a super class extended by each of those classes.

@Before and @After are as simple as launching and closing down the browser.

Now this is what happens - during test execution of one of the test method I closed the browser. Test was doomed to fail and it indeed and so as @AfterMethod as there is no browser to be shut down.

Following this all @Before fail and all methods in first class are skipped. (This looked weird to me as I thought configuration method should not have been dependent on past failed configuration)

But not only this, All the methods in second class are also skipped as all @Before are skipped too which would otherwise had been invoked.

Now even if I do catch exception in @After, Idon't think it is going to get me bogus results. If browser is not there, test would fail any way.

But then I am not very comfortable to suppress error this way.

What I am not comfortable with is - "It only needs to fail once to put your entire test suite in jeopardy."

As there are intermittent failures at times and I don't think they should put entire test suite in jeopardy. 

But then this is how testng is designed or this is how xunit framework work, and my little mind is not able to get to this.

~tarun

[Cédric Beust ♔]

If they are intermittent failures, why not catch the exception then?

-- 

Cédric

--

You received this message because you are subscribed to the Google Groups "testng-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/testng-users/-/PIjgcpyR6BwJ.

[Jek]

You could try setting configfailurepolicy=continue.

http://groups.google.com/group/testng-users/msg/c9616bb45100b563?

Ideally you wouldn't have any intermittent failures.

-John

[Tarun Bhadauria]

I agree jek, but I encounter such intermittent failures when executing test in cloud. During one of the test method (and this is not fixed) browser gives up and all subsequent tests in the suite are skipped. Because of this I need  to execute test again so was wondering if this behavior of TestNG expected or an anomaly.
Cedric has clarified on it.