Timeouts and exceptions

[testng...@opensymphony.com]

My test method is expected to throw an exception, so I'm using @ExpectedExceptions({ IllegalStateException.class }).

I'm also trying to use a timeout (@Test(timeOut = 1000)), so that if the method doesn't complete (in this case, by throwing an exception), the test method will be failed.

However, when I do this, and the method throws the expected exception, I get a long strack trace (attached below) and it's marked as a failure.
Note that, if I remove the timeout, the method passes.

Should I be expecting to be able to use @ExpectedExceptions and the timeout in this way together?

Thanks,
Calum



com.beust.testng.internal.thread.ThreadExecutionException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.IllegalStateException: This message connection has been closed.
at com.beust.testng.internal.thread.FutureResultAdapter.get(FutureResultAdapter.java:28)
at com.beust.testng.internal.Invoker.invokeMethod(Invoker.java:298)
at com.beust.testng.internal.Invoker.invokeTestMethod(Invoker.java:200)
at com.beust.testng.internal.Invoker.invokeTestMethods(Invoker.java:373)
at com.beust.testng.internal.TestMethodWorker.run(TestMethodWorker.java:45)
at com.beust.testng.TestRunner.privateRun(TestRunner.java:542)
at com.beust.testng.TestRunner.run(TestRunner.java:439)
at com.beust.testng.SuiteRunner.privateRun(SuiteRunner.java:149)
at com.beust.testng.SuiteRunner.run(SuiteRunner.java:87)
at com.beust.testng.TestNG.run(TestNG.java:182)
at com.beust.testng.TestNG.main(TestNG.java:220)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:86)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.IllegalStateException: This message connection has been closed.
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:205)
at java.util.concurrent.FutureTask.get(FutureTask.java:80)
at com.beust.testng.internal.thread.FutureResultAdapter.get(FutureResultAdapter.java:26)
... 15 more
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: This message connection has been closed.
at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.IllegalStateException: This message connection has been closed.
at com.aspectssoftware.util.Preconditions.checkState(Preconditions.java:200)
at com.aspectssoftware.smartstation3.messaging.impl.BasicMessageConnection.checkNotClosed(BasicMessageConnection.java:107)
at com.aspectssoftware.smartstation3.messaging.impl.BasicMessageConnection.sendMessage(BasicMessageConnection.java:155)
at com.aspectssoftware.smartstation3.messaging.impl.TestBasicMessageConnectionClosing.testClosingThreads(TestBasicMessageConnectionClosing.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:293)
at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:34)

---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8102#8102

[Alexandru Popescu]

#: the mind was *winged* after testng...@opensymphony.com said on 6/7/2005 3:07 PM :#

as far as i can see you should declare either
com.beust.testng.internal.thread.ThreadExecutionException or
java.util.concurrent.ExecutionException in your @ExpectedException as one of those is the real
thrown exception.

pls let me know if this is fixing the problem in the first phase. It would be very helpful if you
can send me the test you are using.

:alex |.::the_mindstorm::.|

[testng...@opensymphony.com]

I've attached a simplified version of my test code, as well as the stack trace this particular method causes.

Without a timeout, the expected exception is thrown and the test passes.
With a timeout, the outcome should be the same. As I understand it, the timeout should only have an effect if the method fails to complete within the specified time.
If the method does complete within that time, then the timeout should have had no effect. That is what I expected, but does not appear to be the case.

Thanks,
Calum


import com.beust.testng.annotations.*;

public class TestTimeout
{
@Test(timeOut = 1000)
@ExpectedExceptions({ IllegalStateException.class })
public void testTimeout()
{
throw new IllegalStateException("Expected failure");
}
}


Stack trace:

com.beust.testng.internal.thread.ThreadExecutionException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.IllegalStateException: Expected failure

at com.beust.testng.internal.thread.FutureResultAdapter.get(FutureResultAdapter.java:28)
at com.beust.testng.internal.Invoker.invokeMethod(Invoker.java:298)
at com.beust.testng.internal.Invoker.invokeTestMethod(Invoker.java:200)
at com.beust.testng.internal.Invoker.invokeTestMethods(Invoker.java:373)
at com.beust.testng.internal.TestMethodWorker.run(TestMethodWorker.java:45)
at com.beust.testng.TestRunner.privateRun(TestRunner.java:542)
at com.beust.testng.TestRunner.run(TestRunner.java:439)
at com.beust.testng.SuiteRunner.privateRun(SuiteRunner.java:149)
at com.beust.testng.SuiteRunner.run(SuiteRunner.java:87)
at com.beust.testng.TestNG.run(TestNG.java:182)
at com.beust.testng.TestNG.main(TestNG.java:220)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:86)

Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.IllegalStateException: Expected failure

at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:205)
at java.util.concurrent.FutureTask.get(FutureTask.java:80)
at com.beust.testng.internal.thread.FutureResultAdapter.get(FutureResultAdapter.java:26)
... 15 more

Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Expected failure

at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

Caused by: java.lang.IllegalStateException: Expected failure
at TestTimeout.testTimeout(TestTimeout.java:9)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:293)
at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:34)

... 6 more

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8109#8109

[testng...@opensymphony.com]

Thanks very much for the test. I can reproduce it and I will fix it asap. You can drop me a personal email if you want to send you the new jar containing the fix.

thanks again,
:alex |.::the_mindstorm::.|

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8113#8113

[testng...@opensymphony.com]

I hope the attached version is fixing the problem. Pls give it a try and let me know.

cheers,
:alex |.::the_mindstorm::.|

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8143#8143

[testng...@opensymphony.com]

Thanks Alexandru

That works fine - the test passes.

One further (more minor) comment: the stack trace given (attached below) shows a RuntimeException thrown by the TestNG hierarchy. Ideally, I would expect this to only show the particular exception thrown in my code.
So the stack trace shown when timeout is used, is slightly different from when the timeout is not used. Ideally, I would expect that, if the timeout is not exceeded, the behaviour should be exactly the same as in the case that the timeout is not used at all.

Thanks for your help so far,
Calum

java.lang.RuntimeException: java.lang.IllegalStateException: This message connection has been closed.

at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

Caused by: java.lang.IllegalStateException: This message connection has been closed.
at com.aspectssoftware.util.Preconditions.checkState(Preconditions.java:200)
at com.aspectssoftware.smartstation3.messaging.impl.BasicMessageConnection.checkNotClosed(BasicMessageConnection.java:107)
at com.aspectssoftware.smartstation3.messaging.impl.BasicMessageConnection.sendMessage(BasicMessageConnection.java:155)

at com.aspectssoftware.smartstation3.messaging.impl.TestBasicMessageConnectionClosing.testClosingThreads(TestBasicMessageConnectionClosing.java:73)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)

at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:294)

at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:34)
... 6 more

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8183#8183

[Alexandru Popescu]

#: the mind was *winged* after testng...@opensymphony.com said on 6/8/2005 10:26 AM :#

Let me see if i get it right :): do you still see the RuntimeException inside the trace? I
completely agree with you that this is not correct. Pls confirm and I will give it another try -
sorry if i missed this.

:alex |.::the_mindstorm::.|

[testng...@opensymphony.com]

> Let me see if i get it right :): do you still see the
> RuntimeException inside the trace? I
> completely agree with you that this is not correct.
> Pls confirm and I will give it another try -
> sorry if i missed this.
>
> :alex |.::the_mindstorm::.|
>

That's right.

When I run (the example I gave you) without a timeout, I get the following stack trace:

java.lang.IllegalStateException: Expected failure
at TestTimeout.testTimeout(TestTimeout.java:9)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:294)

at com.beust.testng.internal.Invoker.invokeMethod(Invoker.java:240)
at com.beust.testng.internal.Invoker.invokeTestMethods(Invoker.java:357)
at com.beust.testng.internal.TestMethodWorker.run(TestMethodWorker.java:57)
at com.beust.testng.TestRunner.privateRun(TestRunner.java:542)
at com.beust.testng.TestRunner.run(TestRunner.java:442)
at com.beust.testng.SuiteRunner.privateRun(SuiteRunner.java:185)
at com.beust.testng.SuiteRunner.run(SuiteRunner.java:116)
at com.beust.testng.TestNG.run(TestNG.java:233)
at com.beust.testng.TestNG.main(TestNG.java:298)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)

at com.intellij.rt.execution.application.AppMain.main(AppMain.java:86)


When I run the example with the timeout (and the expected exception being thrown), I get the following stack trace:

java.lang.RuntimeException: java.lang.IllegalStateException: Expected failure

at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:37)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

Caused by: java.lang.IllegalStateException: Expected failure
at TestTimeout.testTimeout(TestTimeout.java:9)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:294)
at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:34)
... 6 more

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8187#8187

[Alexandru Popescu]

#: the mind was *winged* after testng...@opensymphony.com said on 6/8/2005 10:56 AM :#

oke ... i will fix this and let you know. thanks for the help.

:alex |.::the_mindstorm::.|

[testng...@opensymphony.com]

Please try the attached version. Thanks again for your help.

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8202#8202

[testng...@opensymphony.com]

Thanks again for the prompt response - gives a very good impression of the project.

Yes - this fixes it. The stack trace I get is attached below, and it seems correct.

Many thanks,
Calum

java.lang.IllegalStateException: Expected failure
at TestTimeout.testTimeout(TestTimeout.java:9)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.beust.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:294)
at com.beust.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:34)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=3062&messageID=8206#8206