Re: TestNG & Selenium RC verifyTrue

[BG_Hatch]

Don't know if I have the exact same problem as Roy ("doesn't work" - does that mean there's a Java runtime error, compiler error, or is it not evaluating the condition properly?).

For me, verifyTrue is causing a runtime error when I try using TestNG (piggybacked on JUnit though using SeleneseTestCase), the trace is:
{font:style=font-family: courier new,courier; font-size: 8pt;} \\{font}
{font:style=font-family: courier new,courier; font-size: 8pt;}java.lang.AssertionError: java.lang.AssertionError: null \\     at com.thoughtworks.selenium.SeleneseTestBase.fail(SeleneseTestBase.java:360) \\     at com.thoughtworks.selenium.SeleneseTestBase.assertTrue(SeleneseTestBase.java:365) \\     at com.thoughtworks.selenium.SeleneseTestBase.assertTrue(SeleneseTestBase.java:369) \\     at com.thoughtworks.selenium.SeleneseTestBase.verifyTrue(SeleneseTestBase.java:129) \\     at com.thoughtworks.selenium.SeleneseTestCase.verifyTrue(SeleneseTestCase.java:119) \\     at com.core.MyTest.myTest(MyTest.java:23) \\     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) \\     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) \\     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) \\     at java.lang.reflect.Method.invoke(Unknown Source) \\     at junit.framework.TestCase.runTest(TestCase.java:164) \\     at junit.framework.TestCase.runBare(TestCase.java:130) \\     at com.thoughtworks.selenium.SeleneseTestCase.runBare(SeleneseTestCase.java:212) \\     at junit.framework.TestResult$1.protect(TestResult.java:110) \\     at junit.framework.TestResult.runProtected(TestResult.java:128) \\     at junit.framework.TestResult.run(TestResult.java:113) \\     at junit.framework.TestCase.run(TestCase.java:120) \\     at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:128) \\     at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196) \\{font}
{font:style=font-family: courier new,courier; font-size: 8pt;} \\{font}
{font:style=font-family: courier new,courier; font-size: 8pt;}    at com.thoughtworks.selenium.SeleneseTestBase.fail(SeleneseTestBase.java:360) \\     at com.thoughtworks.selenium.SeleneseTestBase.checkForVerificationErrors(SeleneseTestBase.java:398) \\     at com.thoughtworks.selenium.SeleneseTestBase.tearDown(SeleneseTestBase.java:410) \\     at com.thoughtworks.selenium.SeleneseTestCase.tearDown(SeleneseTestCase.java:84) \\     at junit.framework.TestCase.runBare(TestCase.java:136) \\     at com.thoughtworks.selenium.SeleneseTestCase.runBare(SeleneseTestCase.java:212) \\     at junit.framework.TestResult$1.protect(TestResult.java:110) \\     at junit.framework.TestResult.runProtected(TestResult.java:128) \\     at junit.framework.TestResult.run(TestResult.java:113) \\     at junit.framework.TestCase.run(TestCase.java:120) \\     at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:128) \\     at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386) \\     at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196){font}


I have the same imports as Roy. The AssertTrue test suggested by Dmitri works, but I need the test to continue running if it can't find text on the page.

[Dmitri Tikhanski]

Hi,

you might want to use try/catch block like:


// preceding test logic

try\\         {
            // make some assertion s.a. Assert.assertTrue(selenium.isTextPresent("my brave new text"));\\             Assert.assertTrue(false);\\         }\\         catch (Throwable ex)\\         {
            // do some logging or whatever\\             System.out.println(ex.getMessage());\\         }


// the test continues

Hope this helps,

Dmitri

[BG_Hatch]

Hi Dmitri,

Strictly speaking, this works, e.g.:

{font:style=font-family: courier new,courier; font-size: 8pt;}        try { \\             Assert.assertTrue({font}{font:style=font-family: courier new,courier; font-size: 8pt;}selenium.isTextPresent("some text to find"){font}{font:style=font-family: courier new,courier; font-size: 8pt;}); \\         } catch (Throwable ex) { \\             System.out.println(ex.getMessage()); \\             ex.printStackTrace(); \\         }{font}

{font:style=font-family: courier new,courier; font-size: 8pt;} \\{font}

{font:style=font-family: courier new,courier; font-size: 8pt;}        try { \\             Assert.assertTrue({font}{font:style=font-family: courier new,courier; font-size: 8pt;}selenium.isElementPresent("link=Other Media"){font}{font:style=font-family: courier new,courier; font-size: 8pt;}); \\         } catch (Throwable ex) { \\             System.out.println(ex.getMessage()); \\             ex.printStackTrace(); \\         }{font}

However, this doesn't seem like an effective solution because this doesn't get reported to JUnit or TestNG as a failure. We'd have to manually check the console/logs after each test.

To get around that problem, I overloaded the verifyTrue method to accept a String message, and tried using {font:style=font-family: courier new,courier; font-size: 10pt;}fail(){font} in the catch blocks, e.g. +{font:style=font-family: courier new,courier;}{font}+

{font:style=font-family: courier new,courier; font-size: 8pt;}    private void verifyTrue(boolean condition, String msg) { \\         try { \\             Assert.assertTrue(condition); \\         } catch (Throwable ex) { \\             System.out.println("verifyTrue(false) for: " + msg); \\             fail("verifyTrue(false) for: " + msg); \\         } \\     }{font}

The fail() method is used by the SeleneseTestBase class, and JUnit now reports this as a failure... but it throws an AssertionError and immediately ends the test, reporting only the first failure.

In Selenium IDE, verify* will note the failure but finishes the test (unless it fails a later assert* of course), and the total number of failures is shown. I expected this to work the same way under JUnit/TestNG and Selenium RC, am I completely wrong in this assumption?

Thanks for your help!

[Roy de Kleijn]

Now I use Junit instead of TestNG. Now everything works fine.

Roy

[BG_Hatch]

I think we're on the same page, i.e. try/catch blocks around statements we want to verify, is solving the wrong problem.

Recalling the first lines in the error trace, i.e.:

{font:style=font-family: courier new,courier; font-size: 8pt;}java.lang.AssertionError: java.lang.AssertionError: null \\     at com.thoughtworks.selenium.SeleneseTestBase.fail(SeleneseTestBase.java:360) \\     at com.thoughtworks.selenium.SeleneseTestBase.assertTrue(SeleneseTestBase.java:365) \\     at com.thoughtworks.selenium.SeleneseTestBase.assertTrue(SeleneseTestBase.java:369) \\     at com.thoughtworks.selenium.SeleneseTestBase.verifyTrue(SeleneseTestBase.java:129) \\     at com.thoughtworks.selenium.SeleneseTestCase.verifyTrue(SeleneseTestCase.java:119) \\     at com.core.MyTest.myTest(MyTest.java:23){font}

The fail() method throws an AssertionError with a null message string:
{font:style=font-family: courier new,courier; font-size: 8pt;}public static void fail(String message) { \\      throw new AssertionError(message); \\ }{font}

However, this never makes it back to the verifyTrue(boolean b) method (SeleneseTestBase.java:127), which assembles the Verification Errors string later used by checkForVerificationErrors():
    {font:style=font-family: courier new,courier; font-size: 8pt;}/** Like assertTrue, but fails at the end of the test (during tearDown) */ \\     public void verifyTrue(boolean b) { \\         try { \\             assertTrue(b); \\         } catch (Error e) { \\             verificationErrors.append(throwableToString(e)); \\         } \\     }{font}


It looks like an error in the Selenium classes themselves, rather than my code, in which case I should file a bug report against the SeleneseTestBase class...

Looking at it another way--what kind of error/fail messages should I be seeing, if a verify* statement fails?