Large GIN Modules and DevMode performance

[Nuno Rosa]

Hi,

At the moment we've a large scale application that reached a bottleneck at DevMode startup time.

It takes ~120s to hit onModuleLoad() call and spends most of the time generating and compiling GIN Injector ~85%.

Our best result was to target gwt-UnitCache to a virtual disk in RAM, this speeded things up but still not under 90s to reach onModuleLoad().

Does exist any other way to improvide GIN processing without trying to isolate groups of screens on their own GWT modules (by consequence smaller GIN modules) and run only the one a developer is working on.

I think we could not reach an acceptable startup time (<20s) without it.

This only brings some small concerns about mantaining dev mode code only (besides gwt.xml files), at least ActivityMapper, HistoryMapper and GIN module to switch.

How do you tackle this issue on your large scale projects?

[dominikz]

Maybe the easiest way is to wait for GWT 2.5 and super-draft mode?

[Thomas Broyer]

FYI, the Wave team recommended (2 years ago) creating "harnesses"; i.e. smaller "standalone apps".

In Wave, they made an EditorHarness for their operation-transform-aware rich-text editor and an UndercurrentHarness for the "wave panel" (the one that displays a wave, with threaded wavelets, and incremental data and feature loading).

In your case, create small and modular GinModules, and small EntryPoints and Ginjectors specific to a particular set of "screens".

http://www.google.com/events/io/2010/sessions/gwt-continuous-build-testing.html (starts around 42:30 in the video)

[Nuno R]

Thanks Thomas, that was what i suspected.

[Joseph Lust]

We've run into the same issue with out project (~80 rich app screens). We're currently breaking it out into many independent modules for just this reason.

We also tried to make many application components common, and then moved these out into our common component module. This way we don't keep recompiling them for each module run, since they are precompiled and served up from Maven.

Sincerely,

Joseph

[Jens]

Also thought about it how to make reloading a page faster in dev mode.

If I understand you correctly, one should create multiple EntryPoints + Host Pages for development where each EntryPoint only shows a subset of the complete application?

So it would look like:

- libfeature1.gwt.xml (no entry point and is included in ProductionApp.gwt.xml)

- devfeature1.gwt.xml (inherits libfeature1 and has an entry point that is able to present feature1)

and each feature/screen/whatever has its own GinModule, right?

@Joseph: How do you precompile modules to JavaScript and then reuse it? Or did I misunderstood you?

-- J.

[Thomas Broyer]

On Friday, May 25, 2012 1:18:38 PM UTC+2, Jens wrote:

Also thought about it how to make reloading a page faster in dev mode.

If I understand you correctly, one should create multiple EntryPoints + Host Pages for development where each EntryPoint only shows a subset of the complete application?

So it would look like:

- libfeature1.gwt.xml (no entry point and is included in ProductionApp.gwt.xml)

- devfeature1.gwt.xml (inherits libfeature1 and has an entry point that is able to present feature1)

and each feature/screen/whatever has its own GinModule, right?

Basically, yes.

But you don't have to go as small as a "screen" per "test harness".

 

@Joseph: How do you precompile modules to JavaScript and then reuse it? Or did I misunderstood you?

Maybe he's talking about com.google.gwt.dev.CompileModule, which is basically running the Precompile phase of a GWT compilation and serializing the resulting AST into a *.gwtar file. You'll find such *.gwtar files in gwt-user.jar starting with GWT 2.4 (maybe 2.3), but the *.gwtar files are very dependent upon the version of GWT, so it's not a portable thing. AFAIK, it was thought out to speed up gwt-user.jar, not for general consumption.

[Joseph Lust]

Jens,

For clarification, I mean that in Maven we have a GWT project with all of our screens, a Server project with the serverside Java code, and a GWT-Common project with all of our common components.

So all of our common widgets are compiled in the GWT-Common project and wrapped up into a Jar as our common lib. Then our GWT project inherits these (via module.gwt.xml and a Maven dependency). AFAIK this reduces the size of the main code base we usually work in (just the GWT project) and thus Eclipse has less to work with/validate/etc and saves some overhead. We also have automation and GWTTestCases that run against the common lib as a separate project, so we save time during each build on our CI server (Team City).

I am sure Thomas knows more about the GWT compiler than I ever will, so I cannot speak to whether building in this way saves time building intermediate component code, or whether the code for the common components is still compiled with the main GWT project.

Sincerely,

Joseph

[Jens]

For clarification, I mean that in Maven we have a GWT project with all of our screens, a Server project with the serverside Java code, and a GWT-Common project with all of our common components.

So all of our common widgets are compiled in the GWT-Common project and wrapped up into a Jar as our common lib. Then our GWT project inherits these (via module.gwt.xml and a Maven dependency). AFAIK this reduces the size of the main code base we usually work in (just the GWT project) and thus Eclipse has less to work with/validate/etc and saves some overhead. We also have automation and GWTTestCases that run against the common lib as a separate project, so we save time during each build on our CI server (Team City).

Ah ok makes more sense ;)

 

I am sure Thomas knows more about the GWT compiler than I ever will, so I cannot speak to whether building in this way saves time building intermediate component code, or whether the code for the common components is still compiled with the main GWT project.

 The GWT compiler just pulls in any of your source files you have made visible using <source> / <super-source> in your module.gwt.xml and compiles it. So you won't save time in dev mode or during compilation.

-- J.

[Thomas Broyer]

On Friday, May 25, 2012 4:23:02 PM UTC+2, Jens wrote:

 The GWT compiler just pulls in any of your source files you have made visible using <source> / <super-source> in your module.gwt.xml and compiles it. So you won't save time in dev mode or during compilation.

Actually even the contrary! The more JARs and directories you add to your classpath, the more time it needs to find a class/resource in it. (it's negligible here however, with only a single dependency).

I know someone made a modified gwt-maven-plugin that first unpacks every dependency in a single temporary directory and launching GWT with only that directory, gwt-user.jar and gwt-dev.jar in the classpath; he hasn't contributed it though (but said he likely would, IIRC).

[Nuno R]

The startup time improved while breaking down into smaller parts,  but this explode the number of source to maintain only related with dev env.

After a couple of tests it seems the environment has big impact, I'm using a windows 7 machine and after a defrag startup time improved significantly and are now more closer to the results in a Linux environment that it is still faster then w7. 

On Thursday, May 24, 2012 8:58:06 AM UTC+1, Thomas Broyer wrote:

[Jens]

After a couple of tests it seems the environment has big impact, I'm using a windows 7 machine and after a defrag startup time improved significantly and are now more closer to the results in a Linux environment that it is still faster then w7. 

On Win7 you should also clean your temp files regularly to make Eclipse happy, see: http://code.google.com/p/google-web-toolkit/issues/detail?id=5261 . Not sure if all these temp files also slow down dev mode but maybe it helps.

-- J.