SOYC generates more dependency trees

[sp...@google.com]

Reviewers: kathrin,

Description:
Currently, SOYC shows dependency information for a whole-program
dependency trace. This patch additionally shows the various dependency
graphs used by the code splitter to decide what code goes into what
fragment.

The display of this information can use work. Since the patch is
getting large, though, I was thinking we might want to go ahead with it
and at least get the information included at all. Thoughts welcome on
that plan; it would also be possible to spend time polishing up the UI
further.

This patch has several parts:
1. The CodeSplitter can record its dependencies as it goes.
2. Correspondingly, DependencyRecorder is updated to handle recording
multiple dependency graphs to the same file.
3. The dashboard now has "dependency linkers" that specify what
dependency page to link to from each size breakdown.
4. For the total-size breakdown, there is a "split status" page added,
because it's ambiguous which dependency graph to show.
5. For the leftovers size breakdown, there is a "leftovers" page added,
again because it's ambiguous which dependency graph to show.
6. To improve the text within the UI, the initial split point sequence
is now recorded to splitPointsN.xml.gz.

Please review this at http://gwt-code-reviews.appspot.com/50806

Affected files:
dev/core/src/com/google/gwt/core/ext/soyc/impl/DependencyRecorder.java
dev/core/src/com/google/gwt/core/ext/soyc/impl/SplitPointRecorder.java
dev/core/src/com/google/gwt/dev/jjs/JavaToJavaScriptCompiler.java
dev/core/src/com/google/gwt/dev/jjs/ast/JProgram.java
dev/core/src/com/google/gwt/dev/jjs/impl/CodeSplitter.java
dev/core/src/com/google/gwt/dev/jjs/impl/ControlFlowAnalyzer.java
tools/soyc-vis/src/com/google/gwt/soyc/GlobalInformation.java
tools/soyc-vis/src/com/google/gwt/soyc/MakeTopLevelHtmlForPerm.java
tools/soyc-vis/src/com/google/gwt/soyc/SoycDashboard.java

[kpr...@google.com]

On 2009/07/01 17:56:26, Lex wrote:

Generally, LGTM. I had a few comments below.

Going forward, I notice that SOYC + the dashboard are growing in
complexity and functionality - there is nothing wrong with that, and
it's definitely a good step forward. However, would it make sense to
write up a number of questions that the user may ask, and then design
the UI that way? For instance, the new patch includes answers to
questions ranging from "what is downloaded in a typical initial load
sequence?" to "what split points include code belonging to a specific
class?" Both of these are very good questions, but they are very
different ways of looking at SOYC output. I see two options: a) a
full-blown query language (far off, but would be very cool), or b) a UI
driven by use cases and/or specific questions, which we spell out
explicitly for the user, either in the UI itself or else in an
accompaying doc. What do you think?

http://gwt-code-reviews.appspot.com/50806

[Bruce Johnson]

On Mon, Jul 6, 2009 at 10:11 AM, <kpr...@google.com> wrote:

I see two options: a) a
full-blown query language (far off, but would be very cool), or b) a UI
driven by use cases and/or specific questions, which we spell out
explicitly for the user, either in the UI itself or else in an
accompaying doc. What do you think?

We can break it into phases. For GWT 2.0, we ought to polish up the aesthetics of what we already have. Post-2.0, I think a query language makes the most sense, but only if it is a reasonable amount of effort (a few weeks). Once we have a query engine on SOYC data, then we can build any kind of out-of-the-box HTML reports we want, formulated as a set of canned queries.

[kpr...@google.com]

here the comments I was referring to.

http://gwt-code-reviews.appspot.com/50806/diff/1/5
File
dev/core/src/com/google/gwt/core/ext/soyc/impl/DependencyRecorder.java
(right):

http://gwt-code-reviews.appspot.com/50806/diff/1/5#newcode42
Line 42: * that wrap them.
"they throw" -> "it throws"?

http://gwt-code-reviews.appspot.com/50806/diff/1/5#newcode147
Line 147: * do repeated lookups into the dependencies table to follow
the chain.
Is the motivation here that otherwise we would get too much SOYC output?

http://gwt-code-reviews.appspot.com/50806/diff/1/3
File tools/soyc-vis/src/com/google/gwt/soyc/GlobalInformation.java
(right):

http://gwt-code-reviews.appspot.com/50806/diff/1/3#newcode61
Line 61: "Leftovers code", "leftovers");
I would leave the "neither initial nor exclusive to any split point" in
there (unless I missed it and it shows up somewhere else).

http://gwt-code-reviews.appspot.com/50806

[Lex Spoon]

I fully agree about the development pattern! That's why I started by
writing the CodeSplitting wiki page as if SOYC already supported the
use cases I was thinking about. The design doc was therefore a user
manual. As an aside, the code splitting document might sound like a
weird home for SOYC development, but in this case the use cases are
the same. If we add more kinds of information to SOYC, then that will
change.

At the risk of being redundant, here are those two use cases I thought about:

1. Size breakdowns of different parts of your code, so you know what
parts to work most on shrinking and/or splitting out.

2. Dependency information related to code splitting, so you can debug
what is happening when splitting doesn't go as expected.

For this scope, my remaining task list to get information complete is:

1. Dependency information for strings.

2. Depict the initial load sequence.

3. Deal with the divide between pre-optimization size breakdowns and
after-optimization dependencies. If we go with the "Soy Lite" size
breakdowns, then this issue will disappear.

For the UI itself, doubtless it can be improved. I only went so far
as to think about the work flow for common debugging tasks, which I
documented in the CodeSplitting page. However, the UI is still
sparse, and it could doubtless use more guidance and cross-linking.
In particular, the current implementation makes it nearly impossible
to re-sort the output. Additionally, it takes a lot of time to
generate even the relatively limited set of HTML files that are
currently supported. Both of these could be remedied by making it a
GWT app plus a servlet.

Finally, I like the idea of supporting more detailed queries. I
haven't looked into it because it's exhausted more than my available
time just to cover the basics. A good start would be... use cases
that aren't already covered. :)

Lex

[sp...@google.com]

Thanks, Kathrin! I'll update it as you request. One question is in
there about the wording for the leftover fragment; what do you think?

http://gwt-code-reviews.appspot.com/50806/diff/1/5
File
dev/core/src/com/google/gwt/core/ext/soyc/impl/DependencyRecorder.java
(right):

http://gwt-code-reviews.appspot.com/50806/diff/1/5#newcode42
Line 42: * that wrap them.

I'll reword it to parse better.

http://gwt-code-reviews.appspot.com/50806/diff/1/5#newcode147
Line 147: * do repeated lookups into the dependencies table to follow
the chain.

Yes, and more specifically, it takes a while to compute all that output.
I admit I didn't carefully time whether it helps, but it should.

http://gwt-code-reviews.appspot.com/50806/diff/1/3
File tools/soyc-vis/src/com/google/gwt/soyc/GlobalInformation.java
(right):

http://gwt-code-reviews.appspot.com/50806/diff/1/3#newcode61
Line 61: "Leftovers code", "leftovers");

Good eye. How about "Leftover code not in any other category"?

The reason I dropped it is because there is now a *third* category:
split points in the initial sequence. It was getting hard to fit on a
single header line in the output HTML if all the exceptions are listed!

http://gwt-code-reviews.appspot.com/50806

[Katharina Probst]

http://gwt-code-reviews.appspot.com/50806/diff/1/3#newcode61
Line 61: "Leftovers code", "leftovers");

Good eye.  How about "Leftover code not in any other category"?

SGTM.

[sp...@google.com]

Committed at r5676.

http://gwt-code-reviews.appspot.com/50806