Can't get 'ant test' to pass it times out

[Scott Morgan]

Hi All,

    I am trying to contribute NoSuchFieldError (and it's parent classes) to GWT.   I have not been able to get get the 'ant test' target to work, my most recent attempt is getting a timeout after 5 hours :(

BUILD FAILED

/home/scott/gwt-src/trunk/build.xml:162: The following error occurred while executing this line:/home/scott/gwt-src/trunk/build.xml:27: 

The following error occurred while executing this line:/home/scott/gwt-src/trunk/build.xml:71:

 The following error occurred while executing this line:/home/scott/gwt-src/trunk/user/build.xml:471: 

Interrupted task <parallel>. Waited 5 hour, but this task did not complete.

Total time: 331 minutes 11 seconds

Any idea what I am doing wrong, or how to fix it?

This is a copy of;

http://code.google.com/p/google-web-toolkit/issues/detail?id=8931&q=adligo.com&colspec=ID%20Type%20Status%20Owner%20Milestone%20Summary%20Stars

I was having issues posting to this group.

Cheers,

Scott

[Thomas Broyer]

On Tuesday, October 7, 2014 9:34:30 PM UTC+2, Scott Morgan wrote:

Hi All,

    I am trying to contribute NoSuchFieldError (and it's parent classes) to GWT.   I have not been able to get get the 'ant test' target to work, my most recent attempt is getting a timeout after 5 hours :(

BUILD FAILED

/home/scott/gwt-src/trunk/build.xml:162: The following error occurred while executing this line:/home/scott/gwt-src/trunk/build.xml:27: 

The following error occurred while executing this line:/home/scott/gwt-src/trunk/build.xml:71:

 The following error occurred while executing this line:/home/scott/gwt-src/trunk/user/build.xml:471: 

Interrupted task <parallel>. Waited 5 hour, but this task did not complete.

Total time: 331 minutes 11 seconds

Any idea what I am doing wrong, or how to fix it?

AFAIK, GWT cannot be built using JDK 1.8 yet (there are issues with Hibernate Validator, there's a patch in review).

But "ant test" runs all the tests in many different configurations. You might want to a) disable some of these configurations and b) possibly filter tests to run.

For example, you can disable tests from the dev/core project using -Dtest.dev.disable=true, and you can run tests for only user/ using "ant user -Dtarget=test" (after an "ant dist-dev") instead of "ant test". You can further filter tests by setting the gwt.junit.testcase.includes property (it defaults to "**/*Suite.class", you could set it to "**/com/google/gwt/emultest/**Suite.class" for example to only run tests of the "java runtime emulation").

Have a look at the command line we use on our continuous integration server: http://build.gwtproject.org/job/gwt/lastBuild/consoleFull
Currently we use: "ant clean elemental test dist -Dgwt.version=2.7.0-SNAPSHOT -Dgwt.threadsPerProcessor=1 -Dgwt.threadCount=1 -Dtest.web.htmlunit.disable=true -Dtest.nometa.htmlunit.disable=true -Dtest.emma.htmlunit.disable=true"

Also note the environment variables we set (TZ specifically).

We'd need to better document this, but we're floating the idea (for a few years now) of ditching Ant for a better build tool (not that Ant is bad, but it's severely limiting us as we want to get a more modular build)

[Jens]

Hmm instead of ant I mostly use Eclipse to run tests. To do so I enable the GPE plugin for gwt-user/gwt-dev projects in Eclipse and then run tests as usual. Since GPE picks up the GWT source project as SDK it works quite well for me. 

-- J.

[Manuel Carrasco Moñino]

Just for your information, I was writing a document explaining how to run tests from command line, I have taken advantage of this thread to commit a first version to gerrit, feel free to play with the examples and make suggestions. 

https://gwt-review.googlesource.com/#/c/9552/

On Wed, Oct 8, 2014 at 12:40 PM, Jens <jens.ne...@gmail.com> wrote:

Hmm instead of ant I mostly use Eclipse to run tests. To do so I enable the GPE plugin for gwt-user/gwt-dev projects in Eclipse and then run tests as usual. Since GPE picks up the GWT source project as SDK it works quite well for me. 

-- J.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/0916bc14-d1f7-4b9a-be89-f0e2ffb56369%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Scott Morgan]

Hi All,

   Thanks for all of you replies, I will start looking at this again in a few days.  Also what build tools are you looking at?

Historically I have favored ant over maven because it is more flexible, however I have spent a good chunk of the last week

thinking about writing a build system that can concurrently build a large set of projects based on dependencies.   I haven't taken

any time for discovery of other build tools.

Cheers,

Scott

[Thomas Broyer]

Maven is a no-go for GWT (I lost way too much time trying that road).

Contenders for now are Buck and Gradle.

While modularizing GWT, Gradle would require us to move files around, or start to create a big, hard-to-maintain build script.

Buck is closer to what Google is using, but has no (or limited) IDE support, and no support for deploying artifacts to Maven repositories; but contrary to Ant we could make scripts to generate IDE configuration and Maven-deployable bundles from the build files (by analyzing them with "buck audit"; Gerrit is already doing such things and it works well)

I had thought about using Pants but it unfortunately doesn't run on Windows, and I don't think we're ready to block Windows users from contributing to GWT (or force them to run Linux in a VM; although we could probably create a Docker or VirtualBox image with everything installed and ready to use, so maybe it's not a showstopper actually…). Pants is likely to collide with Google BUILD files, but I've been told that could be fixed (telling Pants to use files with other names).

Pants is similar to Buck but has IDE support and support for deploying artifacts to Maven repositories. I haven't tried it yet though.

[Scott Morgan]

Thanks for the build tools comments, I have a few questions.

Do any of these have source control (git/tag) support for building specific versions (ie git clone, checkout)?
Do any of these use massive concurrency for the compiling/testing runs?

The legacy adligo.org build (in ant) did things this way in a fairly modular way, but was hard to set-up
and takes a good chunk of time because it's on a single thread.

Obtain other projects from source control
Compile/Jar
Share Jars (it used its own ~ repository concept)
Test

Also each project could be built solo in less than a minute.

I was thinking it would be nice(fast) if did this all concurrently (by default)
based on the project dependencies, with customizable waves/stages. 

Cheers,
Scott

[Scott Morgan]

Hi,

Some other issues;
read
https://gwt-review.googlesource.com/#/c/9552/8/README.md
this worked
$ ant clean elemental dist
It might be nice to have the following on the ci server...
${uname -r}
3.10.0-123.8.1.el7.x86_64
${python -V}
Python 2.7.5
${gcc -v}
...
gcc version 4.8.2 20140120 (Red Hat 4.8.2-16) (GCC)


  Working on a OutOfMemory PermGen space from
cd dev
ant test
     mucking with ANT_OPTS in the shell :)_
     [echo] scott ${ANT_OPTS}

I don't see any posts on the user group about this, and I have 8Gb RAM
on this machine, so hopefully that's enough. 
I will try to contribute to the doc sometime later;
www.gwtproject.org/makinggwtbetter.html

TIA,
Scott

[Scott Morgan]

Ok dev ant test is working :)
solved by the obvious
$ which ant
modify $PATH to the $ANT_HOME/bin/ant  script


ANT_OPTS="-Xmx4g -XX:MaxPermSize=4g"

quotes are important :)

[Scott Morgan]

yea

ant clean elemental test dist -Dgwt.version=2.7.0-SNAPSHOT -Dgwt.threadsPerProcessor=1 -Dgwt.threadCount=1 -Dtest.web.htmlunit.disable=true -Dtest.nometa.htmlunit.disable=true -Dtest.emma.htmlunit.disable=true

BUILD SUCCESSFUL
Total time: 56 minutes 51 seconds

14min off the gwt ci server :)_

[Scott Morgan]

hmm I added the three classes LinkageError,

IncompatibleClassChangeError and NoSuchFieldError,

and got a large number of test failures....

ie;

[junit] Running com.google.web.bindery.requestfactory.gwt.RequestFactorySuite
    [junit] Tests run: 182, Failures: 0, Errors: 182, Time elapsed: 5.257 sec

[Scott Morgan]

After seeing some issues in my pristine/unmodified copy, I  am trying
ant clean elemental dist

ant clean elemental test dist -Dgwt.version=2.7.0-SNAPSHOT -Dgwt.threadsPerProcessor=1 -Dgwt.threadCount=1 -Dtest.web.htmlunit.disable=

true -Dtest.nometa.htmlunit.disable=true -Dtest.emma.htmlunit.disable=true

Not sure why this would matter but it should be consistent.

[Scott Morgan]

Hmm,  still failures on the pristine copy getting new copies trying
to modify the directory before the first build?

[Scott Morgan]

Since my change causes so many test failures, it would be nice if I
could run the full build failing fast on the first test failure.  Is this possible?

This would help me determine the root cause of the other test failures.

Cheers,
Scott

[Scott Morgan]

Most of the errors appear to be caused by UnableToCompleteException;

Exception in constructor: testGetMethods (com.google.gwt.core.ext.UnableToCompleteException: (see previous log entries)
        at com.google.gwt.dev.cfg.ModuleDef.checkForSeedTypes(ModuleDef.java:1031)
        at com.google.gwt.dev.cfg.ModuleDef.getCompilationState(ModuleDef.java:670)
        at com.google.gwt.dev.javac.typemodel.ModuleContext.getTypeOracleFor(ModuleContext.java:44)
        at com.google.gwt.dev.javac.typemodel.ModuleContext.<init>(ModuleContext.java:55)

[Scott Morgan]

Added the three classes to /user/src/com/google/gwt/user/server/rpc/impl/LegacySerializationPolicy.java
trying another full run...

[Manuel Carrasco Moñino]

What is exactly the part of GWT you want to contribute with?

What are the tests you have to run for that specific part?

What are your OS and tools (java, ant, python, g++) versions?

I discourage you to try to run all the gwt test suite in your environment, I would focus on those tests relevant for your changes, if 'ant clean dist-dev' works ok, don't worry about tests because they are run continuously in our CI

Normally if you are working with your code in eclipse, you might want to run specific tests from eclipse, to do that you only have to run like any other JUnit test except that you would have to edit the configuration and include source folders in it's class path,  src, super, test* under gwt-user/core and gwt-dev/core.

On Fri, Oct 17, 2014 at 10:00 AM, Scott Morgan <sc...@adligo.com> wrote:

Added the three classes to /user/src/com/google/gwt/user/server/rpc/impl/LegacySerializationPolicy.java
trying another full run...

--

You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/b9b5c45b-7aa2-4d74-ad01-98c862f8a779%40googlegroups.com.

[Rene Hangstrup Møller]

At GWT.create Ray Cromwell mentioned he had created some rough gradle files that enabled him to generate an IntelliJ project.

I seem to recall I tried it out shortly after the conference. Now I can't find them. It also sound like Thomas Broyer has attempted something.

I might be tempted to give it a shot again. Can I find some publicly available repository of your attempts, or do I have to start from scratch?

Best regards
Rene

[Scott Morgan]

Hi,

   I do have all the tests passing in my environment just like the ci server, in a unmodified trunk :)
[gwt@localhost trunk]$ ant clean elemental test dist -Dgwt.version=2.7.0-SNAPSHOT -Dgwt.threadsPerProcessor=1 -Dgwt.threadCount=1 -Dtest.web.htmlunit.disable=true -Dtest.nometa.htmlunit.disable=true -Dtest.emma.htmlunit.disable=true
...
BUILD SUCCESSFUL
Total time: 88 minutes 18 seconds

I am trying to add java.lang.NoSuchFieldError and it's parent classes.
https://code.google.com/p/google-web-toolkit/issues/detail?id=8954

   I seem to be missing something that I need to do when adding JSE classes.
I have done the following;
added LinkageError, IncompatibleClassChangeError, NoSuchFieldError to
trunk/user/super/com/google/gwt/emul/java/lang

Ran the full build a huge number of the tests fail.

Tried to fix the tests by added the three classes to;
/user/src/com/google/gwt/user/

server/rpc/impl/LegacySerializationPolicy.java

Ran the full build and had a huge number of tests failing.

Again most of the test failures/errors look like;
Exception in constructor: testGetMethods (com.google.gwt.core.ext.

UnableToCompleteException: (see previous log entries)
        at com.google.gwt.dev.cfg.ModuleDef.checkForSeedTypes(ModuleDef.java:1031)
        at com.google.gwt.dev.cfg.ModuleDef.getCompilationState(ModuleDef.java:670)
        at com.google.gwt.dev.javac.typemodel.ModuleContext.getTypeOracleFor(ModuleContext.java:44)
        at com.google.gwt.dev.javac.typemodel.ModuleContext.<init>(ModuleContext.java:55)

I have done a full file content search for AnnotationFormatError since I am using it as a example,
but I don't find any mention of it in any of the other files in the trunk.

Does anyone have any idea how to fix these test failures/errors that occur when adding
a new JSE class?

Since the classes aren't referenced anywhere I figured it would just pass,
however it doesn't and I am out of ideas.

Cheers,
Scott

[Scott Morgan]

Nevermind,  it started passing, YEA! 
I am not sure what changed, it could be the updated code (tools/trunk) that someone checked in this morning that I pulled in.

[Scott Morgan]

What are your OS and tools (java, ant, python, g++) versions?

Here are my versions, and a script to get them.   Is there a good place to check this script in, perhaps under tools?

Also it would be nice if we could echo the ANT_OPTS in the ant script, since I set it to this in ${ANT_HOME}/bin/ant;
ANT_OPTS="-Xmx4g -XX:MaxPermSize=4g -Dfile.encoding=UTF-8"

[gwt@localhost bin]$ ./system_info.sh
This script simply prints the versions software
used by the gwt build.
unix is
Linux localhost.localdomain 3.10.0-123.8.1.el7.x86_64 #1 SMP Thu Oct 9 23:53:48 CDT 2014 x86_64 x86_64 x86_64 GNU/Linux

java is
java version "1.7.0_65"
OpenJDK Runtime Environment (rhel-2.5.1.2.el7_0-x86_64 u65-b17)
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)
ant is
/home/gwt/ant/apache-ant-1.9.4/bin/ant
Apache Ant(TM) version 1.9.4 compiled on April 29 2014

JAVA_HOME is
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.65-2.5.1.2.el7_0.x86_64

ANT HOME is
/home/gwt/ant/apache-ant-1.9.4

python is
Python 2.7.5

gcc is
gcc (GCC) 4.8.2 20140120 (Red Hat 4.8.2-16)
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


GWT_TOOLS is
/home/gwt/tools

TZ is
America/Los_Angeles

ANT_OPTS may be set in you /home/gwt/ant/apache-ant-1.9.4/bin/ant shell script


Cheers,
Scott

[Thomas Broyer]

On Friday, October 17, 2014 5:25:20 PM UTC+2, Rene Hangstrup Møller wrote:

At GWT.create Ray Cromwell mentioned he had created some rough gradle files that enabled him to generate an IntelliJ project.

I seem to recall I tried it out shortly after the conference. Now I can't find them. It also sound like Thomas Broyer has attempted something.

I might be tempted to give it a shot again. Can I find some publicly available repository of your attempts, or do I have to start from scratch?

Ray's original branch is at https://github.com/cromwellian/gwt-sandbox/tree/gradle-build

My fork of it at https://github.com/tbroyer/gwt-sandbox/tree/gradle-build

But note that both are out-of-date; external dependencies have changed since then.

[Scott Morgan]

Thanks for the comments on various build systems.  I have been scanning all of their docs. 
The strange thing I am noticing is that while they all allow concurrency, it generally seems to be a after thought
for multiple projects.  The buck facebook comment says 2x faster, however I am thinking a well organized build
with massive concurrency would be a much faster.   For example if the concurrency
unit were the project, and it had to do the following for four projects A,B,C and D.

git
compile/jar
test

If B, C and D depend on A, massive
concurrently this would look like.
git A B C D   (4x faster)
compile/jar A  (same)
compile/jar B C D, compile/jar C (3x faster)
test A B C D (4x faster)

If the single threaded approach took 12 minutes (1 minute for each step)
the massive concurrent approach would take 4 minutes or 3x faster. 
If your had a X_tests project for each other project it would enforce
a well optimized build.

I am thinking of wrapping the java part of ant and maven in java to accomplish this,
but it would be a huge time commitment/amount of work.  Although
it would be backward compatible with anything ant can do now :)_

Cheers,
Scott

[Vit Hampl]

Hi,

I've created the environment as described in http://www.gwtproject.org/makinggwtbetter.html, build works fine, but when I try to run tests using ant user -Dtarget=test, I end up with numerous "[ERROR] Hint: Check that your module inherits 'com.google.gwt.core.Core' either directly or indirectly (most often by inheriting module 'com.google.gwt.user.User')" and "Could not find the GWT compiler jarfile. Serialization errors might occur when accessing the persistent unit cache." messages. Any ideas, what may cause the trouble?

Vit