odd super dev mode times

[Stephen Haberman]

Hi,

I'm trying out GWT 2.7.0-beta1, and am seeing kind of odd recompile
times. (Our app is ~70k LOC, and the main/only generators are GWT-RPC
and UiBinder.)

The first compile take 40s. Then, only changing a single char in
a .java file, the first recompile takes 20s. Then each next one takes
consistently ~10-15s.

However, eventually something magical happens, and we hit 3s.

The 3s is great. The first 40s is known/understandable. But I can't
figure out what's happening with these ~10-20s recompiles in the
middle. ...and, right now, I can't even get the 3s time to kick in
again.

Here is the compile output from two attempts:

https://gist.github.com/stephenh/9eecf293570b8d5c5eee

I assume the idea it should immediately go from 40s to 3s? Any ideas
about how to debug the "faster but not quite fast" compiles? (I suppose
an obvious one is to kick up the log level; I should have tried that
already, and will later.)

Thanks,
Stephen

[Stephen Haberman]

> ...and, right now, I can't even get the 3s time to kick in again.

Ah ha...seems to be something with the PersistentUnitCache.

When the recompile is 10-20s every time, I see output of:

Wrote 4944 units to persistent cache
Wrote 1 units to persistent cache
(reload)
Wrote 4944 units to persistent cache
Wrote 1 units to persistent cache
(reload)
Wrote 4944 units to persistent cache
Wrote 1 units to persistent cache

I nuked my gwt-unitCache dir, and then see:

(on startup)
Wrote 4944 units to persistent cache
(first load)
Wrote 1 units to persisent cache
...this (Wrote 1 unit) repeated 100s of times...
Purging cache files
Wrote 6166 units to persistent cache
Wrote 4945 units to persistent cache
Wrote 1 units to persistent cache
...few more 1 units...
(reload)
Wrote 1 units to persistent cache
(reload)
Wrote 1 units to persistent cache

I'm now getting the 3s reload. I restarted devmode 2-3x and it still
worked great. However, on the 4th-5th restart, the old behavior came
back. Nuking my gwt-unitCache re-fixed it.

- Stephen

[Roberto Lublinerman]

The time it takes to compile depends not only on how many files are modified but also how big is the invalidation caused by the modification. Depending what gets invalidated it might require running more generators or (re) compiling many more types.

Also the first few compiles are also warming up the JVM, jit and all.

[Roberto Lublinerman]

John detected that behavior in the persistent unit cache and has a fix for it.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141019211649.6a1a0307%40sh9.
For more options, visit https://groups.google.com/d/optout.

[Stephen Haberman]

> depends on how many files are modified [+ invalidations]

Yeah, sorry, I should have mentioned I've only been changing one file,
just adding/removing a character in a string.

> John detected that behavior in the persistent unit cache and has a
> fix for it.

Great! I'll try it out when it hits master.

- Stephen

[John Stalcup]

I think it's in master now (commit 14f27064497f1171907d0ecbe01a4d2991a7a855)

- Stephen

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141019215142.6e6e53f1%40sh9.

[Daniel Kurka]

Stephen can you verify that this solved your problem and if so cherry pick it to the branch and add me as a reviewer?

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/CAFw3gJ-j4KGj%3DG-sSda%2Bei9M3X%2BDx4%2BzZGW2joogbjoRkSe-Sg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Google Germany GmbH

Dienerstr. 12
80331 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Graham Law, Katherine Stephens

[Stephen Haberman]

> I think it's in master now (commit 14f27064497f1171907d0ecbe01a4d2991a7a855)

Oh shoot; I was wondering if that's what Roberto was referring to.

I assumed 2.7-beta1 already had that fix, because I saw it on the
release/2.7 branch in gerrit (and a few commits down/non-cherry picked).

However, right, it didn't actually make it into 2.7-beta1. I'll test
with a snapshot of the 2.7 branch later today/tonight and report back.

Thanks,
Stephen

[Stephen Haberman]

> However, right, it didn't actually make it into 2.7-beta1.

Crap. Eclipse was lying to me, and had a 2.6.x source jar hooked up to
gwt-dev-2.7-beta1.jar (don't ask) when I pulled up PersisentUnitCache
in my project to check for the change.

So: a) 2.7-beta1 does have John's 14f2706 fix, and b) even trying with
a locally built snapshot of release/2.7, the behavior is still
happening.

I'll spend quality time with the debugger tonight and try and see
what's going on. (But if a CL were to magically hit master before then,
I would not mind that either.)

- Stephen

[Daniel Kurka]

I think that CL is only on master. Thats why I said try master and if it works send me a cherry pick for the CL :)

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141020172605.0c3895aa%40sh9.

For more options, visit https://groups.google.com/d/optout.

[Stephen Haberman]

> I think that CL is only on master. Thats why I said try master and if
> it works send me a cherry pick for the CL :)

$ git branch -a --contains 14f27064497f11
master
* release/2.7
remotes/gerrit/HEAD -> gerrit/master
remotes/gerrit/master
remotes/gerrit/release/2.7

I also opened the 2.7.0-beta1 jar and it's .java file has the change as
well (checks UnitOrigin.ARCHIVE instead of PERSISTENT).

- Stephen

[Stephen Haberman]

Well, this is somewhat anti-climatic, but, AFAICT, it was just
MemoryUnitCache's maps using soft values.

Instead of turning on TRACE everywhere, I upgraded
MinimalRebuildCache's "known modified type" output to WARN. (It would
be great to have log4j-style package-level knobs in the GWT log
output.) That was always right (only 1 .java file with it's 3 inner
types).

After trying a few other things, I also upgraded
CompilationStateBuilder's doBuildFrom, where it reads from the unit
cache, "Used x/y units from cache" line.

And that's it; whenever I have a slow load, I'm loading only ~3k out of
~4k units from cache. Or less (I saw zero once, after I had
particularly annoyed the GC somehow). Whenever it's fast, all ~4k
(minus the newly invalidated one) are loaded from the unit cache.

I removed MemoryUnitCache's maps' .softValues() and that seems to fix
it. Upping the Xmx from 1g to 2g also fixes it, and provides a nice
speed up anyway.

Perhaps 1g was a naively low setting, but, AFAICT, without poking in
the code (or turning on TRACE and knowing exactly what to look for),
there are no visible hints about "hey, the cache isn't keeping up,
that's why your reloads are slow, so up your Xmx".

Given how important the unit cache is now, perhaps the maps should no
longer use soft values? In my case, I would have preferred an OOME as
an blatant "needs more RAM" than silent sub-optimal behavior.

- Stephen

[John Stalcup]

Ah, thanks for getting to the bottom of it. I agree that we should opt to use more RAM rather than have randomly slower compiles. And it would be nice to cherry pick the change into the release branch.

--
You received this message because you are subscribed to the Google Groups "GWT Contributors" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit-contributors+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-web-toolkit-contributors/20141020233903.2a9a93d1%40sh9.