GWTTest Case Error - Cross Talk Between TestClasses - Am I crazy?

[Joseph Lust]

I have no idea what is going on here. The gist:

• GWTTestSuite is reporting MeasureDTOCacheTest is failing at on testGetSummaryData()
• The output for the above is showing the failing assert in ZoomOutAnimationTest.zoomOutAnimationTest() 's onComplete() method

On checking, there is no link, at all between these two test cases. Nothing is shared in common and neither call anything having to do with the other. 

AND this test has passed 5000 times, but recently, on a new build agent (VM Ubunut 12.04 1GB ram, x64) it fails 50% of the time. There are 4 more identical clones of this VM and they never fail.

Here is the kicker, the failure in ZoomOutAnimationTest is because it was written wrong. It makes a scheduled event for 1s, but the dev never delayed the test finish, so the asserts in onComplete() never ran. But now, that onComplete() is getting run, from MeasureDTOCacheTest. Is this a thread safety bug in GWTTestCase and GWTTestSuite?

 // MeasureDTOCacheTest.java
    public void testGetSummaryData() {

        delayTestFinish(AJAX_TEST_FINISH_DELAY); // put test into async mode
        //run
        service.getSummaryData(new AsyncCallbackProvider<MeasureDTOContainer>() {
            @Override
            public void onSuccess(MeasureDTOContainer result) {
                Assert.assertNotNull( result.getMeasureDTOArr() ); // should be original model
                finishTest(); 
            }
        });
    }

// ZoomOutAnimation.java
    public void testZoomOutAnimation() {
        ElementWrapper containerElement = new MockElementWrapper(645, 455);
        final ElementWrapper zoomingElement = new MockElementWrapper(1122, 792, -337, -447);

        ZoomOutAnimation zoomOutAnimation = new ZoomOutAnimation(containerElement) {
            @Override
            protected void onComplete() {
                super.onComplete();               
                // animation will take size from 0->testHeight, test that it ends at testHeight
                assertEquals(645, zoomingElement.getHeight()); // this was wrong, so it will always fail
                assertEquals(455, zoomingElement.getWidth());
            }
        };
        zoomOutAnimation.run(1000); // scheduled for 1s, but then thread exits, so onComplete() never run
    }

The trace:

Tests run: 191, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 72.559 sec <<< FAILURE!

testGetSummaryData(com.foo.client.resources.cache.MeasureDTOCacheTest)  Time elapsed: 3.487 sec  <<< FAILURE!

junit.framework.AssertionFailedError: Remote test failed at 127.0.0.1 / Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.19) Gecko/2010031422 Firefox/3.0.19

expected=645 actual=1122

at junit.framework.Assert.fail(Assert.java:197)

at junit.framework.Assert.assertEquals(Assert.java:94)

at junit.framework.Assert.assertEquals(Assert.java:43)

at com.foo.client.widgets.documents.animations.ZoomOutAnimationTest$1.onComplete(ZoomOutAnimationTest.java:22)

at com.google.gwt.animation.client.Animation.update(Animation.java:232)

at com.google.gwt.animation.client.Animation.updateAnimations(Animation.java:55)

at com.google.gwt.animation.client.Animation.access$0(Animation.java:47)

at com.google.gwt.animation.client.Animation$1.run(Animation.java:148)

at com.google.gwt.user.client.Timer.fire(Timer.java:141)

at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:616)

at com.google.gwt.dev.shell.MethodAdaptor.invoke(MethodAdaptor.java:103)

at com.google.gwt.dev.shell.MethodDispatch.invoke(MethodDispatch.java:71)

at com.google.gwt.dev.shell.OophmSessionHandler.invoke(OophmSessionHandler.java:167)

at com.google.gwt.dev.shell.BrowserChannelServer.reactToMessagesWhileWaitingForReturn(BrowserChannelServer.java:326)

at com.google.gwt.dev.shell.BrowserChannelServer.invokeJavascript(BrowserChannelServer.java:207)

at com.google.gwt.dev.shell.ModuleSpaceOOPHM.doInvoke(ModuleSpaceOOPHM.java:132)

at com.google.gwt.dev.shell.ModuleSpace.invokeNative(ModuleSpace.java:561)

at com.google.gwt.dev.shell.ModuleSpace.invokeNativeObject(ModuleSpace.java:269)

at com.google.gwt.dev.shell.JavaScriptHost.invokeNativeObject(JavaScriptHost.java:91)

at com.google.gwt.core.client.impl.Impl.apply(Impl.java)

at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:214)

at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:616)

at com.google.gwt.dev.shell.MethodAdaptor.invoke(MethodAdaptor.java:103)

at com.google.gwt.dev.shell.MethodDispatch.invoke(MethodDispatch.java:71)

at com.google.gwt.dev.shell.OophmSessionHandler.invoke(OophmSessionHandler.java:167)

at com.google.gwt.dev.shell.BrowserChannelServer.reactToMessages(BrowserChannelServer.java:281)

at com.google.gwt.dev.shell.BrowserChannelServer.processConnection(BrowserChannelServer.java:531)

at com.google.gwt.dev.shell.BrowserChannelServer.run(BrowserChannelServer.java:352)

at java.lang.Thread.run(Thread.java:679)

Sincerely,

Joseph

[Colin Alworth]

The callbacks you are configuring aren't running until the later test is in the middle of working. You are using the finishTest() and delayTestFinish(int) correctly in your first test, but the second has a pair of asserts going off asynchronously, but you are not delaying that test's completion. As a result, when the test order is randomized, testGetSummaryData gets killed off when the zoom out animation callback completes (and fails!).

When run in the other order, no failure apparently occurs because that callback finishes after the test is done, so the test system doesnt have anywhere to report that failure to.

To fix this, try adding finishTest/delayTestFinish to the zoomOut test.

[Joseph Lust]

Colin,

Thanks. That fix is how I fixed it, but I did not expect that JUnit would report the failure of one test as the failure of another, unrelated async test. That is what I can't figure out.

Sincerely,

Joseph