Test timeout in JUnitShell ? (GWT 2.0.0)

226 views
Skip to first unread message

Ed

unread,
Oct 1, 2010, 9:06:59 AM10/1/10
to Google Web Toolkit Contributors
Sometimes my GWT test's fail (during nightly build) and give me the
following error.
-----
[INFO] com.google.gwt.junit.client.TimeoutException: The browser did
not contact the server within 60000ms.
[INFO] - NO RESPONSE: 192.168.1.65 / Mozilla/5.0 (Windows; U; Windows
NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
[INFO] - 1 client(s) haven't responded back to JUnitShell since the
start of the test.
[INFO] Actual time elapsed: 60.004 seconds.
------

This occurs when running my tests through the maven codehaus plugin,
starting first tomcat with the cargo plugin.
I played with the timeout settings I have in the maven and cargo
plugin, but don't seem to be able to solve it.
The maven plugin will run GWT test suites that extend from
GWTTestSuite that contain test cases that extend GWTTestCase.
The first GWT test suite always run with success and then the
following GWT test suite gives the above error

The code in JUnitShell throwing the above error:
----
} else if (testBeginTimeout < currentTimeMillis) {
double elapsed = (currentTimeMillis - testBeginTime) / 1000.0;
throw new TimeoutException(
"The browser did not contact the server within "
+ TEST_BEGIN_TIMEOUT_MILLIS + "ms.\n"
+ messageQueue.getUnretrievedClients(currentTestInfo)
+ "\n Actual time elapsed: " + elapsed + " seconds.\n");
}
----

I don't understand well how the testing with server/client concept
works.
Why is this timeout generated and how can I solve this?
Shouldn't the timeout be adjustable as I the timeout is contained in
the constant TEST_BEGIN_TIMEOUT_MILLIS ?

Freeland Abbott

unread,
Oct 1, 2010, 10:08:33 AM10/1/10
to google-web-tool...@googlegroups.com
That timeout means that GWT has launched a browser pointed at itself, to run the test, but the browser didn't actually connect to us within 60 seconds.  The constant isn't directly configurable, no, but it's not about the size of your test, just getting that first useful access from the browser (so it's supposed to encompass browser start time + network latency; 60s should be way over adequate!).

I have seen the timeout message happen due to a compile error in the GWT code, but you'd see those errors in the log also.

I don't know much about the maven codehaus plugin, but here's roughly what GWT tests do:
  1. GWT sets itself up as a web server on some random port.  JUnitShell derives off the older GWTShell, so this should mean using our internal tomcat as the server.  (I'm not sure your "tomcat with the cargo plugin" is relevant!)
  2. That server is set to serve a synthetic GWT module, which is created via a generator that can reflect on your test code.
  3. Depending on the -runStyle flag, GWT chooses a browser to launch.  By default, we'd use HtmlUnit (mostly because we know where it is!), but there are other options available, including Manual (which has no timeout, and expects you to point a browser at GWT's server) and External (you give GWT an executable that accepts a URL as an argument).
  4. If the browser we launch doesn't successfully access the web server we have, on that random port, you will get the timeout you cite.  If it does access in time, the page should pull your code and start walking through the tests, reporting status and moving to the next test (or batch, depending on your batch strategy).
I hope that helps.  HtmlUnit does have some known issues, so especially if you see this happening "sometimes" you might want to experiment with either -runStyle Manual or -runStyle External to see if those are better-behaved for you.



Ed

unread,
Oct 1, 2010, 12:32:23 PM10/1/10
to Google Web Toolkit Contributors
Thanks for the detailed explanation, that sure helps.

> I have seen the timeout message happen due to a compile error in the GWT
> code, but you'd see those errors in the log also.
Could it have something to do with issue 4700 ?:
http://code.google.com/p/google-web-toolkit/issues/detail?id=4700

Some more details:
+ I run the the tests with the following jvm arguments:
-Xms756m -Xmx756m -Dcatalina.base.create=${project.basedir} -
Dgwt.args="-logLevel INFO -Xtries 1"

+ I run the tests in noserver mode against an external tomcat server
that runs the backend that is started by the cargo maven plugin. I use
a proxy servlet in the internal tomcat server that is started by GWT
like explained in issue 4615:
http://code.google.com/p/google-web-toolkit/issues/detail?id=4615

What is the relation between HTMLUnit and the internal tomcat that GWT
uses ?..


On Oct 1, 4:08 pm, Freeland Abbott <fabb...@google.com> wrote:
> That timeout means that GWT has launched a browser pointed at itself, to run
> the test, but the browser didn't actually connect to us within 60 seconds.
>  The constant isn't directly configurable, no, but it's not about the size
> of your test, just getting that first useful access from the browser (so
> it's supposed to encompass browser start time + network latency; 60s should
> be way over adequate!).
>
> I have seen the timeout message happen due to a compile error in the GWT
> code, but you'd see those errors in the log also.
>
> I don't know much about the maven codehaus plugin, but here's roughly what
> GWT tests do:
>
>    1. GWT sets itself up as a web server on some random port.  JUnitShell
>    derives off the older GWTShell, so this should mean using our internal
>    tomcat as the server.  (I'm not sure your "tomcat with the cargo plugin" is
>    relevant!)
>    2. That server is set to serve a synthetic GWT module, which is created
>    via a generator that can reflect on your test code.
>    3. Depending on the -runStyle flag, GWT chooses a browser to launch.  By
>    default, we'd use HtmlUnit (mostly because we know where it is!), but there
>    are other options available, including Manual (which has no timeout, and
>    expects you to point a browser at GWT's server) and External (you give GWT
>    an executable that accepts a URL as an argument).
>    4. If the browser we launch doesn't successfully access the web server we
>    have, on that random port, you will get the timeout you cite.  If it does
>    access in time, the page should pull your code and start walking through the
>    tests, reporting status and moving to the next test (or batch, depending on
>    your batch strategy).
>
> I hope that helps.  HtmlUnit does have some known issues, so especially if
> you see this happening "sometimes" you might want to experiment with either
> -runStyle Manual or -runStyle External to see if those are better-behaved
> for you.
>

Freeland Abbott

unread,
Oct 1, 2010, 1:35:40 PM10/1/10
to google-web-tool...@googlegroups.com
On Fri, Oct 1, 2010 at 12:32 PM, Ed <post2...@gmail.com> wrote:
Thanks for the detailed explanation, that sure helps.

> I have seen the timeout message happen due to a compile error in the GWT
> code, but you'd see those errors in the log also.
Could it have something to do with issue 4700 ?:
http://code.google.com/p/google-web-toolkit/issues/detail?id=4700

Nominally unrelated... 4700 is about benign-but-scary error messages resulting from sloppy gwt.xml specifications (and/or sloppy code arrangement) interacting with GWT's need to sniff around for "all available" classes rather than loading things only as referenced from your entry point.

However, those benign-but-scary ones look just like less benign messages for actually referenced code, and that would be a fatal compilation error that, if I remember right, may indeed result in this timeout.  Precisely, again if I remember, the client connects to the web server, gets a synthetic module that starts by making a GWT RPC to get the first test(s) to be run, and that RPC is what GWT didn't get in the one-minute timeout.  If your code didn't compile, that RPC can't happen, and we time out.  Failure to find source for classes we touch is one way to fail to compile (but identical-seeming errors for classes we don't touch are 4700).
 
+ I run the tests in noserver mode against an external tomcat server
that runs the backend that is started by the cargo maven plugin. I use
a proxy servlet in the internal tomcat server that is started by GWT
like explained in issue 4615:
http://code.google.com/p/google-web-toolkit/issues/detail?id=4615

Ah.  I haven't attempted that pattern.  More thoughts below, though.
 
What is the relation between HTMLUnit and the internal tomcat that GWT
uses ?..

None whatsoever: tomcat is a web server and servlet container; htmlunit is a web browser and JavaScript engine that happens to be (a) implemented in Java and (b) bundled into GWT. 

It sounds as though the RPC request for the junitshell servlet (i.e. JUnitHostImpl, implementing JUnitHost/JUnitHostAsync as an RPC interface) is not getting to "our" servlet; I can't tell you why.  It might be misconfiguration in your proxy, and being sent to your tomcat server, which either wouldn't handle it or would handle it but not be the test infrastructure waiting for the call, and so the timeout would happen.  Something like Charles or wireshark, or just close-reading your logs and configurations, might help you work out what was happening there.  I suppose it might be that HtmlUnit is tripping itself up on startup, too, and sometimes not even making that RPC... same tools to diagnose.


Ed

unread,
Oct 2, 2010, 9:22:05 AM10/2/10
to Google Web Toolkit Contributors
Thanks for the details, it gives me a better understanding of what's
going on such that I know where to look.

I notices that issue 4700 (the compiler errors) isn't of concern
during the nightly build as during the build all sources are correctly
included through the GWT maven plugin (not compiler errors in the
log).

testing in noserver mode:
I did improve the logging in the Proxy servlet such that I can see
better what is happening and if a call isn't forwarded to his parent
class: the GWT servlet (GWTShellServlet).

The test seem to run "fine" now, but let's see what happens the coming
days. I now know the buttons/settings to adjust to solve it I think.

I run 3 GWT Test suites through the maven plugin (pom part:
<includes>**/GwtBackEnd*Suite.java</includes>).
The first suite always runs fine. What suprised me was that the second
suite shows the timeout errors in the logging because of a compile
error (I forced a compile error manually) in the thirds suite, whereas
this compile error show up latter in the logging :(... A bit strange
and misleading... When I solve this compile error in the third suite,
the timeout errors in the second suite also disappear and the whole
build is successful....

I was able to reproduce this error by first fixing it such that it run
with success and then running it with the same compile error again.
In the logging I see that the Proxy servlet does forward the calls
correctly to the parent servlet: GWTShellServlet.

Any idea why these timeout occur in this case?

So in the logging this looks like this (starting from suite 2):
----
[INFO] -------------------------------------------------------
[INFO] T E S T S (SUITE 2)
[INFO] -------------------------------------------------------
[INFO] Running Run all GWT Profile backend tests.
[INFO] Starting HTTP on port 0
[INFO] [WARN] catalina.base.create is deprecated. Use
catalina.base, and it will be created if necessary.
....
...
>>> PROXY SERVLET forward calls to parent class:
[INFO] 13:04:09.676 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junit.html?
gwt.codesvr=82.94.165.225:54470
[INFO] 13:04:09.676 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junit.html?
gwt.codesvr=82.94.165.225:54470

...
...
[INFO] Tests run: 8, Failures: 0, Errors: 8, Skipped: 0, Time elapsed:
383.244 sec <<< FAILURE!
[INFO]
testSubscribe(com.bv.gwt.profile.GwtBackendSubscribeMemberOkTest)
Time elapsed: 383.213 sec <<< ERROR!
[INFO] com.google.gwt.junit.client.TimeoutException: The browser did
not contact the server within 60000ms.
[INFO] - NO RESPONSE: 192.168.1.34 / Mozilla/5.0 (Windows; U; Windows
NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
[INFO] - 1 client(s) haven't responded back to JUnitShell since the
start of the test.
[INFO] Actual time elapsed: 60.002 seconds.
[INFO] at com.google.gwt.junit.JUnitShell.notDone(JUnitShell.java:
896)
....
....
[INFO] -------------------------------------------------------
[INFO] T E S T S (SUITE 3)
[INFO] -------------------------------------------------------
[INFO] Running Run all GWT Declare backend tests.
[INFO] Starting HTTP on port 0
[INFO] [WARN] catalina.base.create is deprecated. Use
catalina.base, and it will be created if necessary.
[INFO] [WARN] A new version of GWT (2.0.3) is available
[INFO] For additional info see: file:/tmp/gwt-update-2.0.3.html
....
....
[INFO] 13:10:38.401 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.declare.intern.DeclareGwtTest.JUnit/junithost
[INFO] 13:10:38.401 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.declare.intern.DeclareGwtTest.JUnit/junithost
[INFO] Rebinding com.bv.gwt.declare.GwtBackendUpdateCategoriesOkTest
[INFO] Adding '1' new generated units
[INFO] Validating newly compiled units
[INFO] [ERROR] Errors in 'generated://
F5C72292886AF6E0B1AAD76DB720CCD8/com/bv/gwt/declare/
__GwtBackendUpdateCategoriesOkTest_unitTestImpl.java'
[INFO] [ERROR] Line 3: The type
__GwtBackendUpdateCategoriesOkTest_unitTestImpl cannot subclass the
final class GwtBackendUpdateCategoriesOkTest
[INFO] [ERROR] Line 7: The method testUpdateCategories()
is undefined for the type
__GwtBackendUpdateCategoriesOkTest_unitTestImpl
[INFO] [ERROR] Unable to find recently-generated type
'com.bv.gwt.declare.__GwtBackendUpdateCategoriesOkTest_unitTestImpl
[INFO] [ERROR] Deferred binding failed for
'com.bv.gwt.declare.GwtBackendUpdateCategoriesOkTest'; expect
subsequent failures
[INFO] 13:10:38.741 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.declare.intern.DeclareGwtTest.JUnit/junithost
...
...
INFO] Results :
[INFO]
[INFO] Tests in error:
[INFO]
testUpdateCategories(com.bv.gwt.declare.GwtBackendUpdateCategoriesOkTest)
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO]
[INFO]
------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO]
------------------------------------------------------------------------
[INFO] There was test failures.
----



On Oct 1, 7:35 pm, Freeland Abbott <fabb...@google.com> wrote:
> On Fri, Oct 1, 2010 at 12:32 PM, Ed <post2edb...@gmail.com> wrote:
> > Thanks for the detailed explanation, that sure helps.
>
> > > I have seen the timeout message happen due to a compile error in the GWT
> > > code, but you'd see those errors in the log also.
> > Could it have something to do with issue 4700 ?:
> >http://code.google.com/p/google-web-toolkit/issues/detail?id=4700
>
> Nominally unrelated... 4700 is about benign-but-scary error messages
> resulting from sloppy gwt.xml specifications (and/or sloppy code
> arrangement) interacting with GWT's need to sniff around for "all available"
> classes rather than loading things only as referenced from your entry point.
>
> *However*, those benign-but-scary ones look just like less benign messages
> for actually referenced code, and *that* would be a fatal compilation error

Ed

unread,
Oct 3, 2010, 1:42:29 PM10/3/10
to Google Web Toolkit Contributors
Today the nightly build failed again and no compiler error occurred as
appears in the above log snippet :(
Below the concerning logging snippet.
In the logging you see that the Proxy servlet does forward calls to
the parent class: GWT servlet: GWTShellServlet ("No Proxy forward" in
the logging means that it's not forwarded to the external tomcat
server.
So I don't understand the timeout error indicating that the no
connection is made :(...

I will now try the Runstyle option manual as you describe above.

------
[INFO] 15:31:29.140 [main] INFO org.apache.catalina.startup.Embedded
- Starting tomcat server
[INFO] 15:31:29.243 [main] INFO o.a.catalina.core.StandardEngine -
Starting Servlet Engine: Apache Tomcat/5.0.28
[INFO] 15:31:29.264 [main] INFO o.apache.catalina.core.StandardHost -
XML validation disabled
[INFO] 15:31:29.282 [main] INFO o.apache.catalina.core.StandardHost -
Create Host deployer for direct deployment ( non-jmx )
[INFO] 15:31:29.288 [main] INFO o.a.c.core.StandardHostDeployer -
Installing web application at context path from URL file:/
root/.hudson/jobs/Belastingvriendin/workspace/Belastingvriendin/bv-web-
test/tomcat/webapps/ROOT
[INFO] 15:31:29.614 [main] INFO o.a.coyote.http11.Http11Protocol -
Initializing Coyote HTTP/1.1 on http-0
[INFO] 15:31:29.666 [main] INFO o.a.coyote.http11.Http11Protocol -
Starting Coyote HTTP/1.1 on http-0
[INFO] 15:31:44.821 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junit.html?
gwt.codesvr=82.94.165.225:58507
[INFO] 15:31:44.846 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junit.html?
gwt.codesvr=82.94.165.225:58507
[INFO] 15:31:45.625 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit.nocache.js
[INFO] 15:31:45.625 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit.nocache.js
[INFO] 15:31:45.867 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/hosted.html?
com_bv_gwt_profile_intern_ProfileGwtTest_JUnit
[INFO] 15:31:45.867 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/hosted.html?
com_bv_gwt_profile_intern_ProfileGwtTest_JUnit
[INFO] 15:31:47.897 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 15:31:47.897 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 15:31:51.137 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 15:31:51.137 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No Proxy forward, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] [WARN]
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit:com.bv.gwt.profile.GwtBackendSubscribeMemberOkTest.testSubscribe
is being retried, retry attempt = 1
[INFO] [WARN]
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit:com.bv.gwt.profile.GwtBackendSubscribeMemberOkTest.testSubscribe
is being retried, retry attempt = 2
[INFO] Tests run: 8, Failures: 0, Errors: 8, Skipped: 0, Time elapsed:
382.879 sec <<< FAILURE!
[INFO]
testSubscribe(com.bv.gwt.profile.GwtBackendSubscribeMemberOkTest)
Time elapsed: 382.845 sec <<< ERROR!
[INFO] com.google.gwt.junit.client.TimeoutException: The browser did
not contact the server within 60000ms.
[INFO] - NO RESPONSE: 82.94.165.225 / Mozilla/5.0 (Windows; U;
Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
[INFO] - 1 client(s) haven't responded back to JUnitShell since the
start of the test.
[INFO] Actual time elapsed: 60.015 seconds.
[INFO] at com.google.gwt.junit.JUnitShell.notDone(JUnitShell.java:
896)
[INFO] at com.google.gwt.junit.JUnitShell.runTestImpl(JUnitShell.java:
1176)
[INFO] at com.google.gwt.junit.JUnitShell.runTestImpl(JUnitShell.java:
1198)
[INFO] at com.google.gwt.junit.JUnitShell.runTestImpl(JUnitShell.java:
1198)

Ed

unread,
Oct 3, 2010, 2:10:55 PM10/3/10
to Google Web Toolkit Contributors
Just played with the -runStyle Manual option....
I am afraid that is not a correct option for me as I will run the
tests during the nightly build and with -runStyle option Manual will
stop when running the test and I have to manually copy/past the
displayed url, like explained in:
http://code.google.com/webtoolkit/doc/latest/DevGuideTesting.html#Manual_Mode.
Of course this isn't possible during a nightly build.

I will try the option "-prod", this might bypass the HtmlUnit code
that might give problems as you explain above.
I tried this option in Eclipse and the test do run fine with this
option.

I hope this will solve the problem, as I have no idea what else it
could be :(
In Eclipse, the GWT tests always run fine (through launch files, I
don't use the plugin), and never give me the timeout problem.

If you have any more ideas, giving my posts above, please let me know?

Ed

unread,
Oct 5, 2010, 5:50:55 AM10/5/10
to Google Web Toolkit Contributors
Yesterday my test was run with success and last night it failed again,
without anything changing in the meantime :(

I was digging a bit more in the GWT Junit test code and I think the
test fails because I don't use a servlet that is hosted by the Tomcat
container started by GWT.
Like Freeland mentions above, I have to contact the backend within 60
sec to overcome the timeout exception. I think I have to contact a
servlet run in the GWT Tomcat instance as otherwise it's not seen as a
backend contact.
Is this the true ?

In my case, because I run in noserver mode, I run my own tomcat
standalone server and use a proxy servlet that extends from
GWTShellServlet to forward any business/servlet calls to my standalone
server. So I never use any servlet in the GWT tomcat instance and as
such receive a timeout exception when my tests don't run fast enough,
which is what sometimes happens....

Please feedback on this and how I can solve this ?
The only option I see now is: disable my GWT test cases as the current
sometimes-failures is unacceptable on the build machine :(
I have no idea how to solve it otherwise...


On Oct 3, 8:10 pm, Ed <post2edb...@gmail.com> wrote:
> Just played with the -runStyle Manual option....
> I am afraid that is not a correct option for me as I will run the
> tests during the nightly build and with -runStyle option Manual will
> stop when running the test and I have to manually copy/past the
> displayed url, like explained in:http://code.google.com/webtoolkit/doc/latest/DevGuideTesting.html#Man....

John Tamplin

unread,
Oct 5, 2010, 8:07:14 AM10/5/10
to google-web-tool...@googlegroups.com
On Tue, Oct 5, 2010 at 5:50 AM, Ed <post2...@gmail.com> wrote:
Yesterday my test was run with success and last night it failed again,
without anything changing in the meantime :(

I was digging a bit more in the GWT Junit test code and I think the
test fails because I don't use a servlet that is hosted by the Tomcat
container started by GWT.
Like Freeland mentions above, I have to contact the backend within 60
sec to overcome the timeout exception. I think I have to contact a
servlet run in the GWT Tomcat instance as otherwise it's not seen as a
backend contact.
Is this the true ?

There are really two apps running in this case.  First, there is the one created by JUnitShell, which contacts an RPC servlet to manage running the test.  The timeout you see is because that servlet didn't get contacted.  Second, there is your actual test code, which may contact your own servlets that you create.  Due to SOP, these must be on the same server that the code was loaded from, but your proxy servlet can run there and redirect the request to another server since it isn't bound by that restriction.
 
In my case, because I run in noserver mode, I run my own tomcat
standalone server and use a proxy servlet that extends from
GWTShellServlet to forward any business/servlet calls to my standalone
server. So I never use any servlet in the GWT tomcat instance and as
such receive a timeout exception when my tests don't run fast enough,
which is what sometimes happens....

When you say noserver mode, what exactly do you mean?  JUnitShell doesn't support -noserver, since as mentioned above it's own servlet has to run to operate the test infrastructure.
 
Please feedback on this and how I can solve this ?
The only option I see now is: disable my GWT test cases as the current
sometimes-failures is unacceptable on the build machine :(
I have no idea how to solve it otherwise...

-runStyle Manual was the suggestion for how to debug what is going on.  There you should be able to see what is going wrong.

If you need a real browser to run the test in an automated way, the recommendation would be to use -runStyle Selenium:uri and run a Selenium-RC server with the browser you want to test with.  If you are on Linux, you can, for example, run Firefox inside an Xvfb instance with a dedicated profile that includes the Developer Plugin and run your test in DevMode or compiled to JS.

--
John A. Tamplin
Software Engineer (GWT), Google

Ed

unread,
Oct 5, 2010, 8:47:20 AM10/5/10
to Google Web Toolkit Contributors
Hi John,

Thanks for your reply.

> There are really two apps running in this case. First, there is the one
> created by JUnitShell, which contacts an RPC servlet to manage running the
> test. The timeout you see is because that servlet didn't get contacted.

Could you please give me some more info and pointers here?
I find the GWT Junit code difficult to follow, especially debugging is
a bit hard.
Where and how does this required servlet get contacted to overcome the
timeout?
If I follow the service method in GWTShellServlet, I don't understand
where the required contact is made.
And I can't find the relation with the messageQueue in the method
JUnitShell.notdone() that must contain the required info to overcome
the timeout exception :(

The GWT Tomcat instance that contains my proxy servlet and my
standalone Tomcat instance, run on the same server (the Cargo maven
plugin starts the stanalone tomcat instance on port 8090)

> When you say noserver mode, what exactly do you mean? JUnitShell doesn't

Maybe it's better not to use the word noserver in this case it might
be misleading. I use the noserver mode during development mode with
own launch files.
In this case it doesn't make sense as GWT Junit always uses his own
Tomcat instance like you explain.

> If you need a real browser to run the test in an automated way, the
> recommendation would be to use -runStyle Selenium:uri and run a Selenium-RC

I also use Selenium, have about 2500 tests that run through Selenium,
that take about 50 hours to complete ;)...
Some background info:
I don't use runstyle for this. I run the tests against production
ready code produced during the nigtly build.
I run my tests through testNG with Maven surefire plugin.

I have build a little testing framework around Selenium RC in which
case Selenium RC is just an implementation detail. I could also use
other Ajax testing framework by creating a class that implements my
Browser interface and connects to the testing framework.
I model all elements on the screen that I want to test as objects and
inject a Browser instance (link to Selenium). I then ask the element
object something like isPresent(), isPresentWait(), typeText(String),
etc... or in case of a DatePickerElement, something like:
setDate(Date). The Date picker implementation will then open the date
picker and choose the specified date and close it again.
So you end up having a class hierarchy of Elements like TextElement,
DateElement, etc....

I use it now for about two years and it works well and the tester that
uses it, which isn't a experienced developers, likes it and can make
his test scenarios well in eclipse...


On Oct 5, 2:07 pm, John Tamplin <j...@google.com> wrote:

John Tamplin

unread,
Oct 5, 2010, 8:51:51 AM10/5/10
to google-web-tool...@googlegroups.com
On Tue, Oct 5, 2010 at 8:47 AM, Ed <post2...@gmail.com> wrote:
> There are really two apps running in this case.  First, there is the one
> created by JUnitShell, which contacts an RPC servlet to manage running the
> test.  The timeout you see is because that servlet didn't get contacted.

Could you please give me some more info and pointers here?
I find the GWT Junit code difficult to follow, especially debugging is
a bit hard.
Where and how does this required servlet get contacted to overcome the
timeout?
If I follow the service method in GWTShellServlet, I don't understand
where the required contact is made.
And I can't find the relation with the messageQueue in the method
JUnitShell.notdone() that must contain the required info to overcome
the timeout exception :(

Look at the comment at the top of JUnitShell -- it explains the connections. 
 
> If you need a real browser to run the test in an automated way, the
> recommendation would be to use -runStyle Selenium:uri and run a Selenium-RC

I also use Selenium, have about 2500 tests that run through Selenium,
that take about 50 hours to complete ;)...
Some background info:
I don't use runstyle for this. I run the tests against production
ready code produced during the nigtly build.
I run my tests through testNG with Maven surefire plugin.

Right, but there your Selenium tests are exercising the running app.  I am suggesting it can also be used for running GWTTestCases in a real browser automatically.

Thomas Broyer

unread,
Oct 5, 2010, 9:05:38 AM10/5/10
to Google Web Toolkit Contributors

On Oct 5, 2:51 pm, John Tamplin <j...@google.com> wrote:
> > I also use Selenium, have about 2500 tests that run through Selenium,
> > that take about 50 hours to complete ;)...
> > Some background info:
> > I don't use runstyle for this. I run the tests against production
> > ready code produced during the nigtly build.
> > I run my tests through testNG with Maven surefire plugin.
>
> Right, but there your Selenium tests are exercising the running app.  I am
> suggesting it can also be used for running GWTTestCases in a real browser
> automatically.

Or in other words: http://code.google.com/p/google-web-toolkit/wiki/RemoteTesting#Selenium
;-)

Ed

unread,
Oct 5, 2010, 10:12:06 AM10/5/10
to Google Web Toolkit Contributors
Thanks,

I found the servlet you meant: JUnitHostImpl
I see that the url that touches this servlet is correctly forwarded by
the proxy and received by this servlet when debugging in Eclipse:
The url that touches it: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost (also appears
in the logging below)

However, when it's running during the nightly build and fails, I's
hard to find out what went wrong as the this servlet doesn't contain
debug/trace logging. It would be nice to see the path of execution in
the logging such that I can see why the servlet isn't touched.

Any idea's how to solve this?
Or any idea about what would be going wrong ?

Below the proxy part code that is very straightforward and some
logging that might be of interest.
In the logging I noticed that just after receiving the /junit url that
should touch the JUnitHostImpl, a retry is attempted. Does that give
you any ideas?

----
protected void service(HttpServletRequest request,
HttpServletResponse response) throws ServletException, IOException {
final String uri = createUriRequest(request);
LOG.debug("Receiving url request, url: {}", uri);

if (uriContainsRequestParts(uri)) {
LOG.debug("Forwarding url to external system.\nUrl: {}\nTarget
host: {}\nTarget port: {}\nTarget protocol: {}", new Object[] { uri,
this.host,
this.port, this.protocol });
proxy(request, response, uri);
}
else {
LOG.debug("No forward. Calling parent method. Url: {}", uri);
super.service(request, response);
}
}

-----
The output snippet looks like:
[INFO] 14:27:07.709 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 14:27:07.709 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No forward. Calling parent
method. Url: /com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 14:27:10.891 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /bv-
web/intern/rpc-gwt/srv-general
[INFO] 14:27:10.892 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Forwarding url to external
system.
...
...
[INFO] 14:27:12.909 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - Receiving url request, url: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] 14:27:12.909 [http-0-Processor4] DEBUG
com.ited.gwt.dev.server.ProxyServlet - No forward. Calling parent
method. Url: /com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost
[INFO] [WARN]
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit:com.bv.gwt.profile.XGwtBackendTestUpdateLoggedInMemberOk.testUpdateLoggedInTaxer
is being retried, retry attempt = 1
[INFO] [WARN]
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit:com.bv.gwt.profile.XGwtBackendTestUpdateLoggedInMemberOk.testUpdateLoggedInTaxer
is being retried, retry attempt = 2
[INFO] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
385.986 sec <<< FAILURE!
[INFO]
testUpdateLoggedInTaxer(com.bv.gwt.profile.XGwtBackendTestUpdateLoggedInMemberOk)
Time elapsed: 385.952 sec <<< ERROR!
[INFO] com.google.gwt.junit.client.TimeoutException: The browser did
not contact the server within 60000ms.
[INFO] - NO RESPONSE: 192.168.1.45 / Mozilla/5.0 (Windows; U; Windows
NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
[INFO] - 1 client(s) haven't responded back to JUnitShell since the
start of the test.
---


On Oct 5, 2:51 pm, John Tamplin <j...@google.com> wrote:

John Tamplin

unread,
Oct 5, 2010, 10:24:11 AM10/5/10
to google-web-tool...@googlegroups.com
On Tue, Oct 5, 2010 at 10:12 AM, Ed <post2...@gmail.com> wrote:
I found the servlet you meant: JUnitHostImpl
I see that the url that touches this servlet is correctly forwarded by
the proxy and received by this servlet when debugging in Eclipse:
The url that touches it: /
com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost (also appears
in the logging below)

However, when it's running during the nightly build and fails, I's
hard to find out what went wrong as the this servlet doesn't contain
debug/trace logging. It would be nice to see the path of execution in
the logging such that I can see why the servlet isn't touched.

Any idea's how to solve this?
Or any idea about what would be going wrong ?

If it works when you run it directly, yet fails in the continuous build, then something is different between the two that matters.  I would suggest running the test in your continuous build environment with remote debugging enabled and attaching to it with a debugger to see what is going on.

Ed

unread,
Oct 5, 2010, 10:56:49 AM10/5/10
to Google Web Toolkit Contributors
> running the test in your continuous build environment with remote debugging
> enabled and attaching to it with a debugger to see what is going on.

Thanks again. John
I did that already but will do it again now that I have a better
understanding of the GWT Junit code.
(I only have to use some tricks to attach the remote debugger
correctly ...)


On Oct 5, 4:24 pm, John Tamplin <j...@google.com> wrote:

Ed

unread,
Oct 13, 2010, 8:40:07 AM10/13/10
to Google Web Toolkit Contributors
I give up for now :(...
This issue is getting me against the roof... :(
Even with remote debugging it all go well
But let's not forget that the error only occurs sometimes: about 50%
of the time and mostly during the day when I run the build manually
when the server is more busy. During the night when the build server
isn't doing much, most of the time the timeout exception doesn't
occur.

Almost always the exception occurs in a test class (extending
GWTTestCase) that contains more then one test. As far as I can see,
it "never" happens in test classes with only one test method... So it
seems that more tests in one GWTTestCase class take more time and in
someway triggers the timeout exception...

Also, in case the timeout exception occurs, the logging indicates that
the GWT servlet GWTShellServlet is correctly informed, but still the
exception occurs...

For now, I disabled these GWT test case that need the backend (through
RPC) as I can't work with this unpredictable behavior.

Ed

unread,
Oct 20, 2010, 9:22:35 AM10/20/10
to Google Web Toolkit Contributors
I couldn't hold back investigating a bit more.
Like described in my last post, the timeout" always" occurred/started
in a GWTTestCase class that contained two test cases that involve RPC
calls.

I did split up these test cases such that they have their own
GWTTestCase class, and the timeout appearantly disappeared as I have
seen it now for a week. All tests run fine now.

Any idea how/why this is ?
After all the debugging/work I get the feeling that there exists some
"deep hidden bug" causing this timeout. ..

Ed


On Oct 13, 2:40 pm, Ed <post2edb...@gmail.com> wrote:
> I give up for now :(...
> This issue is getting me against the roof... :(
> Even with remote debugging it all go well
> But let's not forget that the error only occurs sometimes: about 50%
> of the time and mostly during the day when I run the build manually
> when the server is more busy. During the night when the build server
> isn't doing much, most of the time thetimeoutexception doesn't
> occur.
>
> Almost always the exception occurs in atestclass (extending
> GWTTestCase) that contains  more then onetest. As far as I can see,
> it "never" happens intestclasses with only onetestmethod... So it
> seems that more tests in one GWTTestCase class take more time and in
> someway triggers thetimeoutexception...
>
> Also, in case thetimeoutexception occurs, the logging indicates that
> the GWT servlet GWTShellServlet is correctly informed, but still the
> exception occurs...
>
> For now, I disabled these GWTtestcase that need the backend (through
> RPC) as I can't work with this unpredictable behavior.
>
> On Oct 5, 4:56 pm, Ed <post2edb...@gmail.com> wrote:
>
>
>
> > > running thetestin your continuous build environment with remote debugging
> > > enabled and attaching to it with a debugger to see what is going on.
>
> > Thanks again. John
> > I did that already but will do it again now that I have a better
> > understanding of the GWT Junit code.
> > (I only have to use some tricks to attach the remote debugger
> > correctly ...)
>
> > On Oct 5, 4:24 pm, John Tamplin <j...@google.com> wrote:
>
> > > On Tue, Oct 5, 2010 at 10:12 AM, Ed <post2edb...@gmail.com> wrote:
> > > > I found the servlet you meant: JUnitHostImpl
> > > > I see that the url that touches this servlet is correctly forwarded by
> > > > the proxy and received by this servlet when debugging in Eclipse:
> > > > The url that touches it: /
> > > > com.bv.gwt.profile.intern.ProfileGwtTest.JUnit/junithost (also appears
> > > > in the logging below)
>
> > > > However, when it's running during the nightly build and fails, I's
> > > > hard to find out what went wrong as the this servlet doesn't contain
> > > > debug/trace logging. It would be nice to see the path of execution in
> > > > the logging such that I can see why the servlet isn't touched.
>
> > > > Any idea's how to solve this?
> > > > Or any idea about what would be going wrong ?
>
> > > If it works when you run it directly, yet fails in the continuous build,
> > > then something is different between the two that matters.  I would suggest
> > > running thetestin your continuous build environment with remote debugging

John Tamplin

unread,
Oct 20, 2010, 9:43:53 AM10/20/10
to google-web-tool...@googlegroups.com
On Wed, Oct 20, 2010 at 9:22 AM, Ed <post2...@gmail.com> wrote:
I couldn't hold back investigating a bit more.
Like described in my last post, the timeout" always" occurred/started
in a GWTTestCase class that contained two test cases that involve RPC
calls.

I did split up these test cases such that they have their own
GWTTestCase class, and the timeout appearantly disappeared as I have
seen it now for a week. All tests run fine now.

Any idea how/why this is ?
After all the debugging/work I get the feeling that there exists some
"deep hidden bug" causing this timeout. ..

It's hard to say what the problem is without seeing your code, but take a look at GWT's ValueTypeTest for an example that does very many RPCs in one test class.  

The important things are:
  • call delayTestFinish before making the async call
  • in all cases of the result, make sure to fail the test in some way or call finishTest 
Reply all
Reply to author
Forward
0 new messages