TestNG doesnot recognize exception thrown from child threads

[Fakrudeen Shahul]

Hi Cedric,

 

When a test method spawns a new thread and if the new thread raises an exception, i expected the test will fail but it passed. Is this by design or a bug?

I have included sample code which will show the problem.

 

With Regards,

Fakrudeen

 

public class Demo4Test {

 public static void main(String[] arg) {

  Demo4Test four = new Demo4Test();

  MyThread my = new MyThread();
  four.holdAndDelegate(my);
  System.out.println("should not come");
 }
 
 @Test
 public void test(){
  
  MyThread my = new MyThread();
  holdAndDelegate(my);
  System.out.println("should not come");
  
 }

 public void holdAndDelegate(Thread t) {
  t.start();
  while (t.isAlive()) {
   try {
    Thread.sleep(1000 * 20);
   } catch (InterruptedException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
   }
  }

 }
}

class MyThread extends Thread {
 String user;

 public String getUser() {
  return user;
 }

 public void run() {

  boolean b = true;
  if (b) {
   throw new Exception();
  }
  user = "my name is khan";

  try {
   Thread.sleep(1000 * 60 * 4);
  } catch (InterruptedException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
  }

  
 }
}

[Krishnan Mahadevan]

Cedric,

Can you please comment on this ?

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng...@googlegroups.com.
To unsubscribe from this group, send email to testng-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en.

--
Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

[Cédric Beust ♔]

This is standard JVM behavior. Use the ExecutorService to manage your threads and you will have more control over the exception behavior.

-- 

Cédric

[Krishnan Mahadevan]

Cedric,

Thank you for responding back.

Following is what I learnt. Thought I would share this information in the forum as well.

This is actually a limitation within Reflection. Using reflection when we spawn a method and when that method in turns spawns a child thread, it is not possible to get to know the exceptions that may be thrown in the child thread.

Since TestNG leverages reflection for method invocation, TestNG also inherits this limitation.

The way to circumvent this problem is to use ExecutorService (as suggested by Cedric).

Here's a working example, that has 4 test methods. 

The below test methods use ExecutorService for Thread management and as such always pass.

• testUsingExecutorService()
• testUsingExecutorServiceWithException()

However the below test methods spawn Threads in the normal way and as such one of the tests will always fail.

• testUsingThreads()
• testUsingThreadsWithExceptions() - This method would always fail even though it expects a RuntimeException because the main thread wherein the Test method runs never gets to know the exceptions being thrown in the child thread it spawns.

package testng.samples;

import java.util.concurrent.Callable;

import java.util.concurrent.ExecutionException;

import java.util.concurrent.Executors;

import java.util.concurrent.Future;

import org.testng.annotations.Test;

public class ThreadExecutorServiceDemo {

private static final String ERROR_MSG = "From thread you asked me throw an exception";

@Test

public void testUsingExecutorService() throws InterruptedException, ExecutionException {

MyCallableService service = new MyCallableService(false);

Future<String> returnValue = Executors.newSingleThreadScheduledExecutor().submit(service);

System.out.println(returnValue.get());

}

@Test(expectedExceptions = ExecutionException.class, expectedExceptionsMessageRegExp = ".*" + ERROR_MSG + ".*")

public void testUsingExecutorServiceWithException() throws InterruptedException, ExecutionException {

MyCallableService anotherService = new MyCallableService(true);

Future<String> anotherReturnValue = Executors.newSingleThreadScheduledExecutor().submit(anotherService);

System.out.println(anotherReturnValue.get());

}

@Test

public void testUsingThreads() throws InterruptedException {

MyThreadService mt = new MyThreadService(false);

mt.start();

while (mt.isAlive() == true) {

Thread.sleep(10000);

}

System.out.println(mt.getServiceName());

}

@Test(expectedExceptions = RuntimeException.class, expectedExceptionsMessageRegExp = ERROR_MSG)

public void testUsingThreadsWithExceptions() throws InterruptedException {

MyThreadService mt = new MyThreadService(true);

mt.start();

while (mt.isAlive() == true) {

Thread.sleep(10000);

}

System.out.println(mt.getServiceName());

}

public class MyThreadService extends Thread {

private boolean throwException;

private String serviceName;

public String getServiceName() {

return serviceName;

}

public MyThreadService(boolean throwException) {

this.throwException = throwException;

}

public void run() {

try {

sleep(25000);

this.serviceName = "TestNG threaded Service";

} catch (InterruptedException e) {

throw new RuntimeException(e);

}

if (throwException) {

throw new RuntimeException(ERROR_MSG);

}

}

}

public class MyCallableService implements Callable<String> {

private boolean throwException;

public MyCallableService(boolean throwException) {

this.throwException = throwException;

}

@Override

public String call() throws Exception {

Thread.sleep(25000);

if (throwException) {

throw new RuntimeException(ERROR_MSG);

}

return "Callable Service Invoked";

}

}

}

Output:

[TestNG] Running:

  C:\Users\krmahadevan\AppData\Local\Temp\testng-eclipse--667883055\testng-customsuite.xml

Callable Service Invoked

TestNG threaded Service

Exception in thread "Thread-2" java.lang.RuntimeException: From thread you asked me throw an exception

at testng.samples.ThreadExecutorServiceDemo$MyThreadService.run(ThreadExecutorServiceDemo.java:71)

TestNG threaded Service

PASSED: testUsingExecutorService

PASSED: testUsingExecutorServiceWithException

PASSED: testUsingThreads

FAILED: testUsingThreadsWithExceptions

org.testng.TestException: 

Expected exception java.lang.RuntimeException but got org.testng.TestException: 

Method ThreadExecutorServiceDemo.testUsingThreadsWithExceptions()[pri:0, instance:testng.samples.ThreadExecutorServiceDemo@4fbc9499] should have thrown an exception of class java.lang.RuntimeException

at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1485)

at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1233)

at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)

at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)

at org.testng.TestRunner.privateRun(TestRunner.java:768)

at org.testng.TestRunner.run(TestRunner.java:617)

at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)

at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)

at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)

at org.testng.SuiteRunner.run(SuiteRunner.java:240)

at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)

at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)

at org.testng.TestNG.runSuitesSequentially(TestNG.java:1188)

at org.testng.TestNG.runSuitesLocally(TestNG.java:1113)

at org.testng.TestNG.run(TestNG.java:1025)

at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:109)

at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:202)

at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:173)

Caused by: org.testng.TestException: 

Method ThreadExecutorServiceDemo.testUsingThreadsWithExceptions()[pri:0, instance:testng.samples.ThreadExecutorServiceDemo@4fbc9499] should have thrown an exception of class java.lang.RuntimeException

at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1500)

at org.testng.internal.Invoker.invokeMethod(Invoker.java:751)

at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:894)

at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1219)

... 16 more

===============================================

    Default test

    Tests run: 4, Failures: 1, Skips: 0

===============================================

===============================================

Default suite

Total tests run: 4, Failures: 1, Skips: 0

===============================================

Thanks & Regards

Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

[Cédric Beust ♔]

Thanks for this summary, Krishnan.

-- 

Cédric