GWT 2.7 : precompile and incremental flags

[Jérémie Gottero]

Hello,
I have tried GWT 2.7 beta today, and first of all you guys made an awesome job to reduce compile time (x4/5 faster on my 500k LoC project).
There is however a CodeServer behavior that I don't understand: why the precompile flag is always turned off in incremental mode?

    if (incremental && !noPrecompile) {
      System.out.println("Turning off precompile in incremental mode.");
      noPrecompile = true;
    }

In my dev env, I never use the DevMode On/Off bookmarklets: I always load js files from the CodeServer url, and I wrote a small GWT utility class to display a "recompile" button which is embedded in my app UI (it does the same thing as the bookmarklet, ie calling codeserverurl/recompile). It's easier to use for others devs: no need to explain them the bookmarklet stuff, they just have to launch the CodeServer. With 2.7 I can't use this trick anymore: I need the bookmarklet to do a first compilation to be able to load my app. Is there a reason to prevent precompilation in incremental mode?

Regards,
Jeremie

[Daniel Kurka]

With GWT 2.7 you do not need any bookmarklet stuff anymore. You can just bring it up through the old dev mode integration in eclipse, see:

https://www.youtube.com/watch?v=qpCSbj36O44

PS: You do not need the -superDevMode anymore its now default.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/46bc27ce-d8a3-40ad-9629-b2c6d3bfe961%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Google Germany GmbH

Dienerstr. 12
80331 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Graham Law, Katherine Stephens

[Jérémie Gottero]

It works like a charm, thanks. I still have ~15sec of "compilation" to reload the page even when nothing has changed, but I guess it would be faster with a ssd drive.

Le lundi 20 octobre 2014 16:51:19 UTC+2, Daniel Kurka a écrit :

With GWT 2.7 you do not need any bookmarklet stuff anymore. You can just bring it up through the old dev mode integration in eclipse, see:

https://www.youtube.com/watch?v=qpCSbj36O44

PS: You do not need the -superDevMode anymore its now default.

On Mon, Oct 20, 2014 at 3:38 PM, Jérémie Gottero <jeremie...@gmail.com> wrote:

Hello,
I have tried GWT 2.7 beta today, and first of all you guys made an awesome job to reduce compile time (x4/5 faster on my 500k LoC project).
There is however a CodeServer behavior that I don't understand: why the precompile flag is always turned off in incremental mode?

    if (incremental && !noPrecompile) {
      System.out.println("Turning off precompile in incremental mode.");
      noPrecompile = true;
    }

In my dev env, I never use the DevMode On/Off bookmarklets: I always load js files from the CodeServer url, and I wrote a small GWT utility class to display a "recompile" button which is embedded in my app UI (it does the same thing as the bookmarklet, ie calling codeserverurl/recompile). It's easier to use for others devs: no need to explain them the bookmarklet stuff, they just have to launch the CodeServer. With 2.7 I can't use this trick anymore: I need the bookmarklet to do a first compilation to be able to load my app. Is there a reason to prevent precompilation in incremental mode?

Regards,
Jeremie

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/46bc27ce-d8a3-40ad-9629-b2c6d3bfe961%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Kurka]

Hi Jeremie,

what do you mean by 15s reload? Does the compiler take 15s to detect that you do not have a change? (this would be bad).

-Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/de430b92-1afd-4843-8b24-83b33ae3179c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jérémie Gottero]

Yes, that was 15sec to detect that nothing has changed. But after some tries, I actually got better times (~3sec). See logs below, I got 12sec at first then 3sec. I suspect that my app server load somehow impacts compile time. Should definitely be a problem on my side.

GET /recompile/adminfront
   Job com.exalead.mercury.admin.ui.AdminFront_1_0
      starting job: com.exalead.mercury.admin.ui.AdminFront_1_0
      Compiling module com.exalead.mercury.admin.ui.AdminFront
         Ignored 22 units with compilation errors in first pass.
Compile with -strict or with -logLevel set to TRACE or DEBUG to see all errors.
         Unification traversed 148143 fields and methods and 10734 types. 10695 are considered part of the current module and 10695 had all of their fields and methods traversed.
         Compiling 1 permutation
            Compiling permutation 0...
            Linking per-type JS with 10679 new types.
               prelink JS size = 30989214
               prelink sourcemap = 30989214 bytes and 708768 lines
               postlink JS size = 30934781
               postlink sourcemap = 30934781 bytes and 707368 lines
            Source Maps Enabled
         Compile of permutations succeeded
         Compilation succeeded -- 43,734s
      Linking into D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\tmp\com.exalead.mercury.admin.ui.AdminFront\compile-2\war\adminfront; Writing extras to D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\tmp\com.exalead.mercury.admin.ui.AdminFront\compile-2\extras\adminfront
         Link succeeded
         Linking succeeded -- 5,735s
      52,810s total -- Compile completed
GET /recompile/adminfront
   Job com.exalead.mercury.admin.ui.AdminFront_1_1
      starting job: com.exalead.mercury.admin.ui.AdminFront_1_1
      skipped compile because no input files have changed
      12,201s total -- Compile completed
GET /recompile/adminfront
   Job com.exalead.mercury.admin.ui.AdminFront_1_2
      starting job: com.exalead.mercury.admin.ui.AdminFront_1_2
      skipped compile because no input files have changed
      2,807s total -- Compile completed
[...]
GET /recompile/adminfront
   Job com.exalead.mercury.admin.ui.AdminFront_1_5
      starting job: com.exalead.mercury.admin.ui.AdminFront_1_5
      Compiling module com.exalead.mercury.admin.ui.AdminFront
         Ignored 22 units with compilation errors in first pass.
Compile with -strict or with -logLevel set to TRACE or DEBUG to see all errors.
         Unification traversed 585 fields and methods and 460 types. 9 are considered part of the current module and 9 had all of their fields and methods traversed.
         Compiling 1 permutation
            Compiling permutation 0...
            Linking per-type JS with 9 new types.
               prelink JS size = 13769
               prelink sourcemap = 13769 bytes and 338 lines
               postlink JS size = 30934783
               postlink sourcemap = 30934783 bytes and 707368 lines
            Source Maps Enabled
         Compile of permutations succeeded
         Compilation succeeded -- 3,147s
      Linking into D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\tmp\com.exalead.mercury.admin.ui.AdminFront\compile-7\war\adminfront; Writing extras to D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\tmp\com.exalead.mercury.admin.ui.AdminFront\compile-7\extras\adminfront
         Link succeeded
         Linking succeeded -- 1,753s
      8,204s total -- Compile completed
GET /recompile/adminfront
   Job com.exalead.mercury.admin.ui.AdminFront_1_6
      starting job: com.exalead.mercury.admin.ui.AdminFront_1_6
      skipped compile because no input files have changed
      2,782s total -- Compile completed

Jeremie

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/de430b92-1afd-4843-8b24-83b33ae3179c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Daniel Kurka]

These times are incredible slow. The worst I have seen was around 700ms for a huge chunk of code. Normally these times are around ~100ms.

Something is seriously wrong. Lets try a couple of things:

1. Up the ram of SDM to see if this makes any difference.

2. Are there any other things running on your system that make it slow (lots of other disk IO, other heavy cpu tasks?)

3. How much source code are we talking here (LOC, files)?

-Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/f8c3e8c7-70f6-45b9-a475-5484a55aaa9b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jérémie Gottero]

I run the CodeServer on my profiler to see what's going on (see attached screenshot). Most time is spent in java.io, meaning that we lost time with many disk I/O. Then I looked why I got so much I/O (10k listFiles!), and I think it's probably caused by .svn directories. For each directory, "descentToFindResources" method enters in 9 folders:

Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\prop-base
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\props
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\text-base
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\tmp
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\tmp\prop-base
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\tmp\props
Descending into dir: D:\ng\src\jgottero\svn\mercury\mercury-adminui\trunk\client-tmpsrc\com\exalead\.svn\tmp\text-base

Do we have an easy way to exclude these folders to see if it improves things?

Jeremie

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/f8c3e8c7-70f6-45b9-a475-5484a55aaa9b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jens]

These times are incredible slow. The worst I have seen was around 700ms for a huge chunk of code. Normally these times are around ~100ms.

Can you define "huge"? Our client side code is about 150 KLOC and with SSD a reload takes about 800-1000 ms.

-- J.

[Jérémie Gottero]

Do we have an easy way to exclude these folders to see if it improves things?

Quick and dirty: I overrode the descendToFindResources method to exclude .svn files (note that the file is excluded before isDirectory/isFile calls, avoiding an extra i/o):

    File[] children = dir.listFiles();
    for (File child : children) {
      if (child.getName().equals(".svn")) // hardcoded filter here
          continue;
      String childPath = dirPath + child.getName();
      if (child.isDirectory()) {
 
Reload time goes down from 2,8s to 1,1sec. The ability to filter files here could lead to interesting speed improvements.

Another possible improvement: I got 4874 calls to file.lastModified() in a single browser refresh, for 2088 unique files. Caching the result per run could avoid 50% i/o and reload time would go below 1s. I have no idea if it's realistic/easy to do though, I havent dug in the code ;)

[Juan Pablo Gardella]

Thanks for sharing Jérémie. Good to know.

--

You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/b052327f-8538-4598-b4fb-740cd16d2deb%40googlegroups.com.

[John Stalcup]

I think this is a reasonable change to make since it's simple to implement and a noticeable improvement for SVN users. I'll go ahead and get it in and we can cherry pick it into this release.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CA%2BkiFse_4M3ZqrOa2D%2BdQfVhmWvxXABFH7F7y-KkNXKu5HLxcA%40mail.gmail.com.

[Stephen Haberman]

> if (child.getName().equals(".svn")) // hardcoded filter here

Just curious, but didn't svn change (maybe a year ago or so?) to only
using one top-level .svn folder, similar to Git's top-level .git
folder, and not having .svn littered throughout the file system?

- Stephen

[John Stalcup]

change in review at https://gwt-review.googlesource.com/#/c/9850/

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141021174343.46a191a4%40sh9.

[Jérémie Gottero]

Just curious, but didn't svn change (maybe a year ago or so?) to only
using one top-level .svn folder, similar to Git's top-level .git
folder, and not having .svn littered throughout the file system?

- Stephen

 
You're right, but we have scripts that still rely on .svn folders, so we can't update to a newer version at the moment.