VertxUnitRunner ignores Junit Exception

[Serafima Gurevich]

Hello,

VertxUnitRunner seems to ignore Junit Exceptions. If a test throws a junit.framework.ComparisonFailure, the test is still considered as a success and is marked green.

The test setup is using vertx-core 3.0, vertx-unit 3.0, junit 4.12, xmlunit 1.6

Here is a code snippet:

@RunWith( VertxUnitRunner.class )
public class FooTest {

   @Test
   public void testFoo {
      // ...
      XMLAssert.assertXpathEvaluatesTo( title, "//ShortText", messageXML );
      // ...
   }
}

The assertion above is throwing a junit.framework.ComparisonFailure. In the console, these lines are displayed:

Aug 12, 2015 10:53:47 AM io.vertx.core.impl.ContextImpl
SCHWERWIEGEND: Unhandled exception
junit.framework.ComparisonFailure: expected:<[XXX.]> but was:<[YYY.]>
    at junit.framework.Assert.assertEquals(Assert.java:100)
    at junit.framework.Assert.assertEquals(Assert.java:107)
    at org.custommonkey.xmlunit.XMLAssert.assertXpathEvaluatesTo(XMLAssert.java:862)
    ...

but the test is still green.

Thanks in advance for any suggestions!

Regards,
Serafima

[Alexander Lehmann]

That is not so much a problem in vertx unit, its a general problem in the async execution. If an exception happens in an asynchronous part of your code, its not really inside the method where you defined the handler, so it is not thrown in the method (you will notice this is you throw a checked exception inside a handler that is not declared).

The easiest solution for this is to either use the asserts provided by vertx-unit (using the TestContext object) or to catch your exceptions in the handler and report the failure via the TestContext object.

Something like:

@RunWith( VertxUnitRunner.class )
public class FooTest {

   @Test

   public void testFoo(TestContext testContext) {
      ...
      v -> {
        try {
          // ...

          XMLAssert.assertXpathEvaluatesTo( title, "//ShortText", messageXML );
          // ...

        }
        catch(Exeption ex) {
          testContext.fail(ex);
        }
      }
   }
}

[Serafima Gurevich]

Hello Alexander,

thanks for your reply!

Catching the exception works. (Just need to catch a throwable in this case.)

Bests,
Serafima

[Julien Viet]

Actually this should work, I tried this and its red for me in Intellij :

@RunWith(VertxUnitRunner.class)
public class SimpleJUnitTest {

  @Test
  public void test1() {
    throw new ComparisonFailure("azef", "efef", "efef");
  }

}

with this console:

/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/bin/java -ea -Didea.launcher.port=7534 "-Didea.launcher.bin.path=/Applications/More/IntelliJ IDEA 14.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Applications/More/IntelliJ IDEA 14.app/Contents/lib/idea_rt.jar:/Applications/More/IntelliJ IDEA 14.app/Contents/plugins/junit/lib/junit-rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/julien/java/vertx-examples/unit-examples/target/test-classes:/Users/julien/java/vertx-examples/unit-examples/target/classes:/Users/julien/.m2/repository/io/vertx/vertx-core/3.0.0/vertx-core-3.0.0.jar:/Users/julien/.m2/repository/io/netty/netty-common/4.0.28.Final/netty-common-4.0.28.Final.jar:/Users/julien/.m2/repository/io/netty/netty-buffer/4.0.28.Final/netty-buffer-4.0.28.Final.jar:/Users/julien/.m2/repository/io/netty/netty-transport/4.0.28.Final/netty-transport-4.0.28.Final.jar:/Users/julien/.m2/repository/io/netty/netty-handler/4.0.28.Final/netty-handler-4.0.28.Final.jar:/Users/julien/.m2/repository/io/netty/netty-codec/4.0.28.Final/netty-codec-4.0.28.Final.jar:/Users/julien/.m2/repository/io/netty/netty-codec-http/4.0.28.Final/netty-codec-http-4.0.28.Final.jar:/Users/julien/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar:/Users/julien/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar:/Users/julien/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar:/Users/julien/.m2/repository/io/vertx/vertx-unit/3.0.0/vertx-unit-3.0.0.jar:/Users/julien/.m2/repository/junit/junit/4.12/junit-4.12.jar:/Users/julien/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/Users/julien/java/vertx-examples/examples-utils/target/classes:/Users/julien/.m2/repository/io/vertx/vertx-codetrans/3.0.0/vertx-codetrans-3.0.0.jar:/Users/julien/.m2/repository/io/vertx/vertx-codegen/3.0.0/vertx-codegen-3.0.0.jar:/Users/julien/.m2/repository/org/mvel/mvel2/2.2.0.Final/mvel2-2.2.0.Final.jar:/Users/julien/.m2/repository/io/vertx/vertx-lang-js/3.0.0/vertx-lang-js-3.0.0.jar:/Users/julien/.m2/repository/io/vertx/vertx-lang-groovy/3.0.0/vertx-lang-groovy-3.0.0.jar:/Users/julien/.m2/repository/org/codehaus/groovy/groovy-all/2.3.10/groovy-all-2.3.10.jar:/Users/julien/.m2/repository/io/vertx/vertx-lang-ruby/3.0.0/vertx-lang-ruby-3.0.0.jar:/Users/julien/.m2/repository/org/jruby/jruby-complete/1.7.20/jruby-complete-1.7.20.jar:/Users/julien/.m2/repository/org/slf4j/slf4j-api/1.6.2/slf4j-api-1.6.2.jar" com.intellij.rt.execution.application.AppMain com.intellij.rt.execution.junit.JUnitStarter -ideVersion5 io.vertx.example.unit.test.SimpleJUnitTest

org.junit.ComparisonFailure: azef expected: java.lang.String<efef> but was: java.lang.String<efef>

Expected :efef
Actual   :efef
   <Click to see difference>


	at io.vertx.example.unit.test.SimpleJUnitTest.test1(SimpleJUnitTest.java:27)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at io.vertx.ext.unit.junit.VertxUnitRunner.invokeTestMethod(VertxUnitRunner.java:96)
	at io.vertx.ext.unit.junit.VertxUnitRunner.lambda$invokeExplosively$1(VertxUnitRunner.java:107)
	at io.vertx.ext.unit.junit.VertxUnitRunner$$Lambda$2/707806938.handle(Unknown Source)
	at io.vertx.ext.unit.impl.TestContextImpl.run(TestContextImpl.java:146)
	at io.vertx.ext.unit.impl.TestContextImpl.execute(TestContextImpl.java:130)
	at io.vertx.ext.unit.impl.TestContextImpl.execute(TestContextImpl.java:17)
	at io.vertx.ext.unit.impl.ExecutionContext.run(ExecutionContext.java:22)
	at io.vertx.ext.unit.impl.ExecutionContext.run(ExecutionContext.java:27)
	at io.vertx.ext.unit.junit.VertxUnitRunner.invokeExplosively(VertxUnitRunner.java:120)
	at io.vertx.ext.unit.junit.VertxUnitRunner.access$100(VertxUnitRunner.java:38)
	at io.vertx.ext.unit.junit.VertxUnitRunner$1.evaluate(VertxUnitRunner.java:87)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)


Process finished with exit code 255

what is your runner ?

-- 
Julien Viet
www.julienviet.com

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Visit this group at http://groups.google.com/group/vertx.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/10bc9098-a6cd-413a-82a4-c7176e802937%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Serafima Gurevich]

Hello Julien,

thanks for your message. I tried it and it worked. Unfortunately, we have trouble reproducing the issue from yesterday, while yesterday it happened several times. But since it works now, this issue can be considered as closed. Thanks!
However, we still couldn't find a solution for the verticle zombie issue we described in https://groups.google.com/forum/#!topic/vertx/oUJsWiPto4A - if you have any suggestions there, it would help us a lot. Thanks in advance!

Bests,
Serafima