Incremental compiler refreshes triggered by IDE touching files

[James Nelson]

Whilst digging around trying to make some stuff go faster, I noticed something odd... I was getting a bunch of extra types getting marked stale at the beginning of a recompile, and at work, recompiles triggered even when no changes were made to Gwt-related files.

So, a little debugging and I noticed that we're computing the initial stale types using timestamps.  And, my IDE (yes, sigh, eclipse) is actually touching the files even when I'm not explicitly saving them.  This causes these types to look stale despite there being no changes.

When I close my IDE and manually edit a file via text editor, there are no longer extraneous files being marked stale.

It is understandable to use timestamps when recalculating stale types after a rebind, but for the initial staleness check in UnifyAst, I think paying a little extra for file hashes instead of time stamps might save a lot of extra time recompiling files that did not actually change.  If we want to save some wall time, we could calculate all the initial file hashes in a background thread on the initial compile (or store them as we load them), and then only look at the hashes if there is a timestamp difference.  This would let us use timestamps as a heuristic for checking hashes.

I'm not sure if IntelliJ has better respect for touching files than eclipse, but it seems like a pretty common use case for a file to get touched without changing. 

[Stephen Haberman]

Hi James,

This seemed like a pretty interesting thread. Do you still
suspect it's an issue?

I remember turning on the right logging ~6-12 months ago to see
files get incrementally compiled/etc. with SDM first came out,
but it's been awhile...

I use Eclipse, so if you can refresh my memory with some steps to
reproduce this, I'll see if it's happening on my end.

Thanks,
Stephen

[James Nelson]

Ok.  It doesn't always happen to me, but I am using eclipse Luna on Ubuntu 14.04.

I noticed it personally while testing magic method injection, but it has also happened at work with a standard 2.7 SDK.

What I noticed is when I have a lot of files open, even if I am editing another file in a project not included in GWT, it is marked stale.
I know this because I added a bunch of debugging logs while working on magic method injection, and noticed the timestamp updates.

A co-worker has also reported the same issue with running full builds on a unit cache.

So, it's not in SDM in particular, it's in MinimalRebuildCache (which looks only at timestamps).

Once you can get enough files open that eclipse decides to touch your GWT files when saving other files,

then a rebuild always happens.  Then I close eclipse, and it stops happening.

With my debug logging, and eclipse open, editing a file that is included, a bunch of other files were marked stale.
With eclipse closed, editing the same file, and only that file was marked stale.

I don't have a 100% reliable set of reproduction steps, but I have seen it on different machines using different builds >= 2.7

[James Nelson]

Oh, any my work machine is Windows, and one coworker who reported it is on mac.  
Both work builds are on plain 2.7