Selenium+TestNG Test fails on the TearDown method

[ALISTER ERNEST]

Hello Everyone,

I am back and this time stuck on something pretty simple. I am getting deep into automation with Selenium+TestNG with the base test code in Java, but I am a newb in Java, and not very good at debugging issues in the code itself. Eclipse Intellisense helps a lot, but I am stuck here.

Below is my test class, and though the "Assertions" in the "try-catch" block pass, the test case fails in the @AfterClass method. I have included the Stack Trace below. 

Weird thing is, with the TearDown class I have (exactly as below), the test passes if the Assertions fail. Its behaving in an awkwardly reverse manner.

I have deduced that the issue has to do something with the way I am appending the errors to the "verificationErrorsString" and in the way I have my final "IF" block setup in the TearDown class. But I am not able to put a finger on it and say "Eureka".

Please help a fellow tester out.

//TEST CLASS

package com.test.example;

//Bunch of Imports

public class ExampleTest {

public WebDriver driver;

public StringBuffer verificationErrors = new StringBuffer();

@Parameters({"browser"})

@BeforeTest

public void setup(String browser) throws MalformedURLException, InterruptedException {

DesiredCapabilities capability=null;

if(browser.equalsIgnoreCase("firefox")){

System.out.println("firefox");

capability= DesiredCapabilities.firefox();

capability.setBrowserName("firefox"); 

capability.setPlatform(org.openqa.selenium.Platform.ANY);

}

driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);

driver.navigate().to("http://myWebAppLogin");

}

@Test

public void testExampleTest() throws Exception {

//driver login

try {

Assert.assertEquals("assertion1);

Assert.assertEquals("assertion2);

} catch (Error e) {

verificationErrors.append(e.toString());

}

//driver logout

}

@AfterTest

public void tearDown() throws Exception{

        

driver.close();  

String verificationErrorString = verificationErrors.toString();

if (!"".equals(verificationErrorString)) {

fail(verificationErrorString);  //This is the line that my Stack Trace refers to.

}

}

}

//STACK TRACE

FAILED CONFIGURATION: @AfterTest tearDown

java.lang.AssertionError: java.lang.AssertionError: expected [true] but found [false] //This is where I have been pounding my head & racking my brains to no avail.

at org.testng.Assert.fail(Assert.java:89)

at com.cntntmgmt.content.ExampleTest.tearDown(ExampleTest.java:119)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)

at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:564)

at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:213)

at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:138)

at org.testng.TestRunner.afterRun(TestRunner.java:1021)

at org.testng.TestRunner.run(TestRunner.java:621)

at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)

at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)

at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)

at org.testng.SuiteRunner.run(SuiteRunner.java:240)

at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)

at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)

at org.testng.TestNG.runSuitesSequentially(TestNG.java:1197)

at org.testng.TestNG.runSuitesLocally(TestNG.java:1122)

at org.testng.TestNG.run(TestNG.java:1030)

at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)

at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)

at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)

[Peter Gale]

Alister

At first glance, this seems to be a TestNG issue, and not a Selenium one. I don't use TestNG so I for one can't help.

Someone might be able and willing to help, but this forum should be kept for specifically Selenium issues.

Even if you get a little help, it will only be of use until the next TestNG/Java issue which you need to bring back to us for help with.

So you really need to find some tutorials and Java & TestNG forums for these sort of questions. If you're developers use Java and TestNG then they would be a first point of contact for help and support on these types of issues too.

Sorry if t might not seem like much help with in the short term, but it would be a much better way to deal with this sort of issue in the longer term. And you'll find more poeple here have more time to deal with you Slenium related issues too.

Peter

---

Date: Mon, 17 Sep 2012 10:24:34 -0700
From: aliste...@gmail.com
To: seleniu...@googlegroups.com
Subject: [selenium-users] Selenium+TestNG Test fails on the TearDown method

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/2NEdp3Ea5NwJ.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[ALISTER ERNEST]

Peter - 

I do completely agree with your synopsis, and I will try to keep issues isolated going further.

-Alister

[Darrell Grainger]

Alister,

You cut out all the import statements, I assume to keep the size of the code down. However, this could be the problem. From what you have posted it seems like it is a Java/TestNG problem and nothing to do with Selenium. 

I use jUnit but the concept will be the same for all Java code. If I want to use the fail() method I can have no import statement and just use:

    org.junit.Assert.fail("your failure message goes here.");

Or I can have the following:

    import static org.junit.Assert.fail;

    fail("your failure message goes here.");

Or (a little dangerous because it might include too much):

    import static org.junit.Assert.*;

    fail("your failure message goes here.");

I use the 'static' keyword on the import statement because things like fail() and assertTrue() are static methods of org.junit.Assert. Now the import thing to note, and why this is a little dangerous, is that org.junit.Assert.* exists and org.testng.Assert; exists. One is static and the other dynamic. Because the class in jUnit has the same name as the class in TestNG, if you mix both in with your import statements you could have the compiler mildly confused.

This is why Peter is actually 99.999% sure this is a TestNG or Java problem and nothing to do with Selenium.

Have a look at http://docs.oracle.com/javase/tutorial/java/package/index.html for a little more information about packages. Remember that jUnit and TestNG will have classes with the same name but completely different behaviour. They are both following the information you'll find at the Oracle site.

In general, if you are new to Java you might want to just go through http://docs.oracle.com/javase/tutorial/java/TOC.html. The time you will spend reading this will be far less than the total amount of time you will spend trying to make Java fit the way you think. ;)

Darrell

[ALISTER ERNEST]

Darrell,

You are right, I cut out the import statements to keep the pasted code size small. And below are the import statements I use in my test class, I am using the "junit.Assert" import.

And, like you and Peter mention, this is more of a Java/TestNG issue and not so much as related to Selenium.

But, thank you guys again for taking out the time and pointing me in the right place to look for.

import java.net.MalformedURLException;

import java.net.URL;

import java.util.concurrent.TimeUnit;

import org.testng.annotations.*;

import static org.junit.Assert.fail;

import static org.junit.Assert.assertEquals;

import static org.junit.Assert.assertTrue;

import org.openqa.selenium.*;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

-Alister

[ALISTER ERNEST]

Hello again everyone -

Just wanted to provide closure to this issue (I have been on forums where issues are raised but the solution is never posted even if the issue was ever solved). And, the real reason that I am posting this here is because, I really did not do anything to change the script/code of the test. It still remains the same and now it works. Though this is a solution to my issue, it scares me more than it calms me down, because something that was broke fixed itself. Now that is scary.

I am not going to go ahead and declare that my machine is haunted, but I shall hunt a bit more to see what really happened. There should be a logical explanation. You'll know it if I do find out.

Again, thanks everyone, and I will be more aware of the issue before just posting it where it does not belong.

-Alister

[Peter Gale]

Glad it's working, Alister.

Yes, it is always better to understand why things doesn't work but, sadly, sometimes you never get to the bottom of it before everything just starts working again and the problem can no longer be replicated.

Thanks for a great attitude and a clear explanation of you problem right from the start. I look forward to being able to help more directly in the future.

Peter

---

Date: Mon, 17 Sep 2012 15:14:08 -0700
From: aliste...@gmail.com
To: seleniu...@googlegroups.com
Subject: [selenium-users] Re: Selenium+TestNG Test fails on the TearDown method

--
You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/hyBx3-HTrNcJ.

[Krishnan Mahadevan]

Alister,

I suspect that one of your assertions failed earlier which was what was causing your @AfterTest to conk. This time when you said it worked your assertions seemed to have passed. 

The first step for debug I would suggest is that you setup some System.out.println statements in your catch block and print the assertion failure reasons. That would be a good start. 

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

[Satish Kumar]

Alister,

I would suggest, you let the test fail where it is supposed to which is testExampleTest() .Teardown should be used to do clean up and not to assess pass or fail. If you look at you code the test will pass no matter. If we have to write a test that will always pass what is the point of writing the test in the first place. So here is what I think should be the correct code approach

package com.test.example;

//Bunch of Imports

public class ExampleTest {

public WebDriver driver;

@Parameters({"browser"})

@BeforeTest

public void setup(String browser) throws MalformedURLException, InterruptedException {

DesiredCapabilities capability=null;

if(browser.equalsIgnoreCase("firefox")){

System.out.println("firefox");

capability= DesiredCapabilities.firefox();

capability.setBrowserName("firefox"); 

capability.setPlatform(org.openqa.selenium.Platform.ANY);

}

driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), capability);

driver.navigate().to("http://myWebAppLogin");

}

@Test

public void testExampleTest() throws Exception {

//driver login

Assert.assertEquals("assertion1); // Let the test fail here and you will get the correct stack trace of what wet wrong

Assert.assertEquals("assertion2); // Let the test fail here and you will get the correct stack trace of what wet wrong

//driver logout

}

@AfterTest

public void tearDown() throws Exception{

        

driver.close();  

}

}

If you want to do additional book keeping after the test method is executed, I suggest you use method interceptors or listeners

http://testng.org/doc/documentation-main.html#methodinterceptors

http://testng.org/doc/documentation-main.html#testng-listeners

-Thanks

Satish Kumar Busi

 

--

You received this message because you are subscribed to the Google Groups "Selenium Users" group.
To post to this group, send email to seleniu...@googlegroups.com.
To unsubscribe from this group, send email to selenium-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msg/selenium-users/-/2NEdp3Ea5NwJ.

[ALISTER ERNEST]

Krishnan,

Now that I think about it, I believe that could have been the case too but I might have been too focused on the tear-down method to actually figure out there might be something else wrong. I am not completely sure, but that has the most probability of happening. Thank you for pointing that out. 

That's one thing I truly appreciate about this group - no matter what the question, you always learn something new.

-Alister 

On Monday, September 17, 2012 7:18:59 PM UTC-6, Krishnan wrote:

Alister,

I suspect that one of your assertions failed earlier which was what was causing your @AfterTest to conk. This time when you said it worked your assertions seemed to have passed. 

The first step for debug I would suggest is that you setup some System.out.println statements in your catch block and print the assertion failure reasons. That would be a good start. 

On Tuesday, September 18, 2012, Peter Gale wrote:

Glad it's working, Alister.

Yes, it is always better to understand why things doesn't work but, sadly, sometimes you never get to the bottom of it before everything just starts working again and the problem can no longer be replicated.

Thanks for a great attitude and a clear explanation of you problem right from the start. I look forward to being able to help more directly in the future.

Peter

---

Date: Mon, 17 Sep 2012 15:14:08 -0700

To: selenium-users@googlegroups.com

To post to this group, send email to selenium-users@googlegroups.com.
To unsubscribe from this group, send email to selenium-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[ALISTER ERNEST]

Krishnan,

After some more debugging and encountering the same issue more and more, I finally figured out the problem.

And, you were right. It always happened when one of my "assertions" failed. But again, it has a pattern.

If an "assertEquals" statement fails, there is a good clean error message that says "Expected ABC but saw XYZ"

But, when an "assertTrue(isElementPresent)" statement fails, there is no error message that says which element was not found or was missing. This is usually frustrating if I have 10 "assertTrue" statements, and I only want to check for the elements to be present and not an actual text comparison.

But, I felt compelled to post this out here for others who may encounter this issue.

I do feel now that this is not a Java or a TestNG issue, or as a matter of fact, it was not an issue at all, but a mis-understanding of what was happening.

Very grateful to you Krishnan for pointing me in the right direction for further investigation.

Also, do any of you have an better suggestion on making my error statements be a bit more detailed when they fail on non comparisons?

Again, this happens only if I put all my "assertions" in a try-catch block to make my tests more "verification based" than "assertions based".

-Alister

On Monday, September 17, 2012 7:18:59 PM UTC-6, Krishnan wrote:

Alister,

I suspect that one of your assertions failed earlier which was what was causing your @AfterTest to conk. This time when you said it worked your assertions seemed to have passed. 

The first step for debug I would suggest is that you setup some System.out.println statements in your catch block and print the assertion failure reasons. That would be a good start. 

On Tuesday, September 18, 2012, Peter Gale wrote:

Glad it's working, Alister.

Yes, it is always better to understand why things doesn't work but, sadly, sometimes you never get to the bottom of it before everything just starts working again and the problem can no longer be replicated.

Thanks for a great attitude and a clear explanation of you problem right from the start. I look forward to being able to help more directly in the future.

Peter

---

Date: Mon, 17 Sep 2012 15:14:08 -0700

To: selenium-users@googlegroups.com

To post to this group, send email to selenium-users@googlegroups.com.
To unsubscribe from this group, send email to selenium-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.