Profiling support for GWT

[Toby Reyelts]

Inspired by some needs to perform application profiling, I've decided it would be nice if GWT had profiling support. I've put my thoughts down below and would love to have some feedback.

Goals:

- Profiling should generally NOT involve user-written code. It's standard practice that tools rewrite/instrument programs so that profiling is transparent to the program. A developer shouldn't have to rewrite code to add or remove profiling or collect profiling results.

- GWT's profiling support should include tools to visualize the data. Regardless of whether they're working with proprietary memory dumps, hprof dumps, or using a graphical runtime profiler, developers generally don't work directly with raw profiling data.

- Because profiling's primary goal is to report accurate performance statistics, it does not make sense to support profiling in pure hosted mode.

- Profiling should record timings that relate to the context of the original Java program. ( I.E. report timings for Java functions - not JavaScript counterparts).

- Profiling should capture the elapsed times of RPC invocations and their asynchronous completions.

- If possible, profiling should capture the elapsed time of all HTTP requests that occur through GWT.

- Profiling should capture memory statistics.

- Profiling results should be accurate and repeatable.

- An initial list of events I'd like to capture while profiling:
  methodEntered/Exited
  exceptionThrown/Caught
  moduleLoadBegin/End
  httpRequest/Response
  rpcRequest/Response

Basic Design:

- Modify the GWT compiler so that it optionally instruments GWT applications with profiling code.
- Implement a profiling service in the hosted browser that is accessible in hybrid mode. This profiling service is what the instrumented code would invoke to record the profiling results.
- Modify the dev shell so that it supports launching applications for profiling in a simple manner.

Advantages of this approach:

- Very precise timing measurements using high performance timers (precision in nanoseconds).
- Simple and fast communication between the profiling service and instrumented code. Access to the hosted browser is fast and easy while remote RPC would be slow and wieldy.
- Very accurate memory profiling.
- No server component required.
- Simple user setup.

Drawbacks:

- Can only perform profiling using the local hosted browser.

Thanks,
-Toby

[Dan Morrill]

As a minor comment:  I think it might be useful to track RPC codec time separate from HTTP async request time.  That is, I might be sending data that is fairly low bandwidth, but really nasty to parse, so it might be helpful to time the codec activities separately from the transmission.

Also, substitute "JSON" for "RPC" and you have a second use case that would also be useful to track.  Just something to keep in mind as this gets fleshed out...

- Dan

[Scott Blum]

One concern.. if you're going to get into profiling particular parts of the GWT user code base, I'd think that whould be something configurable rather than baked into the compiler directly.

[Toby Reyelts]

I guess I forgot to mention in the RR that I was planning on having support for filters, where the user specifies which classes/methods they want instrumented. The only thing "baked directly into the compiler" is support for filters. Did I understand your concern correctly, and does that address it?

Thanks,
-Toby

[Scott Blum]

I'm not sure.. it seems like your filter mechanism would have to be pretty sophisticated to be able to specify that you would to time how long RPC round trips take.

[Toby Reyelts]

I think I misunderstood your concern, Scott. I haven't totally planned out the implementation of timing of RPC/HTTP calls, but I've been leaning towards routing the calls through a lightweight proxy. So the timing of code execution would occur through instrumentation and the timing of the asynchronous I/O would occur through the proxy.

[Scott Blum]

Gotcha.  Okay, that makes total sense. :)

[kebernet]

Aside from the RPC stuff, I must be missing something here:

What does this get me that running GWTShell under the NetBeans
profiler now doesn't?

On Feb 5, 12:13 pm, "Scott Blum" <sco...@google.com> wrote:
> Gotcha. Okay, that makes total sense. :)
>

> On 2/5/07, Toby Reyelts <t...@google.com> wrote:
>
>
>
> > I think I misunderstood your concern, Scott. I haven't totally planned out
> > the implementation of timing of RPC/HTTP calls, but I've been leaning
> > towards routing the calls through a lightweight proxy. So the timing of code
> > execution would occur through instrumentation and the timing of the
> > asynchronous I/O would occur through the proxy.
>
> > Thanks,
> > -Toby
>
> > On 2/5/07, Scott Blum <sco...@google.com> wrote:
>
> > > I'm not sure.. it seems like your filter mechanism would have to be
> > > pretty sophisticated to be able to specify that you would to time how long
> > > RPC round trips take.
>

> > > On 2/5/07, Toby Reyelts <t...@google.com> wrote:
>
> > > > I guess I forgot to mention in the RR that I was planning on having
> > > > support for filters, where the user specifies which classes/methods they
> > > > want instrumented. The only thing "baked directly into the compiler" is
> > > > support for filters. Did I understand your concern correctly, and does that
> > > > address it?
>
> > > > Thanks,
> > > > -Toby
>
> > > > On 2/5/07, Scott Blum < sco...@google.com> wrote:
>
> > > > > One concern.. if you're going to get into profiling particular parts
> > > > > of the GWT user code base, I'd think that whould be something configurable
> > > > > rather than baked into the compiler directly.
>

> > > > > On 2/5/07, Dan Morrill <morri...@google.com> wrote:
>
> > > > > > As a minor comment: I think it might be useful to track RPC codec
> > > > > > time separate from HTTP async request time. That is, I might be sending
> > > > > > data that is fairly low bandwidth, but really nasty to parse, so it might be
> > > > > > helpful to time the codec activities separately from the transmission.
>
> > > > > > Also, substitute "JSON" for "RPC" and you have a second use case
> > > > > > that would also be useful to track. Just something to keep in mind as this
> > > > > > gets fleshed out...
>
> > > > > > - Dan
>

[Toby Reyelts]

Meaningful timing results. With the NetBeans profiler, you're profiling a Java application running in a JVM + JSNI, rather than a JavaScript application running in a browser.

[bg]

Exactly. I actually made a post regarding this in the user forum
here: http://groups.google.com/group/Google-Web-Toolkit/browse_thread/
thread/309cbb8f843ed531/55ce3772b9541f8e#55ce3772b9541f8e

Basically what it comes down to is profiling a GWT app in hosted mode
has very little correlation to how a GWT app will run once compiled
down to Javascript. Besides the fundamental differences in how the
stack, heap and garbage collection are used/handled in the two
different environments, you're profiling a strongly typed in hosted
mode but ultimately compiling down to an untyped language. Someone
recommended using a firefox plug-in FireBug to profile a GWT App in a
compiled state, however I haven't tried yet. It is open source and
perhaps it can be adopted to profile GWT apps specifically.

[kebernet]

Aah, I gotcha. Yeah, I have used FireBug to do profiling in JavaScript
mode, but I guess I always see that as a two-step process. I usually
work with the NB profiler just to make sure I am not "leaking" objects
and the basics are OK, and only really start using Firebug if I see a
problem area in the difference in functionality.

[charlie...@gmail.com]

I think your ideas look very solid, and this would be a huge plus for
GWT.

There has been, as you all are likely aware, some discussion about
profiling in the user group several times and it basically comes down
to what kebernet said: profile the java as a "reference" to make sure
there is nothing obviously amiss, and then fire up firebug/venkman and
do some poking around in web mode.

A well thought out way to actually profile things, which this
"proposal" certainly looks like, would be great.

On Feb 5, 10:11 am, "Toby Reyelts" <t...@google.com> wrote:
> Inspired by some needs to perform application profiling, I've decided it
> would be nice if GWT had profiling support. I've put my thoughts down below
> and would love to have some feedback.
>
> Goals:
>
> - Profiling should generally NOT involve user-written code. It's standard
> practice that tools rewrite/instrument programs so that profiling is
> transparent to the program. A developer shouldn't have to rewrite code to
> add or remove profiling or collect profiling results.
>
> - GWT's profiling support should include tools to visualize the data.
> Regardless of whether they're working with proprietary memory dumps, hprof
> dumps, or using a graphical runtime profiler, developers generally don't
> work directly with raw profiling data.
>
> - Because profiling's primary goal is to report accurate performance
> statistics, it does not make sense to support profiling in pure hosted mode.
>
> - Profiling should record timings that relate to the context of the original

> Java program. (I.E. report timings for Java functions - not JavaScript

[Bruce Johnson]

This feature and design makes good sense for a few reasons:

- It's essentially browser-independent (at least, it works on the hosted mode browser for each GWT development platform).

- Native JS profilers won't necessarily do a good job of directing you to the right bit of Java source. In particular, if you're using a native JS profiler ( e.g. firebug or venkman), you would probably not be able to profile obfsucated code. However, in a scripting language, the length of idents can have a big effect on speed. So, intrinsic GWT profiling support is going to be much more accurate than any other approach.

- It's a step towards the cool vision of the hosted browser as a sort of "performance dashboard" for the apps it's running. You could imagine hints that pop up as your apps runs, saying things like "It looks like you forgot to turn on permanent caching for your compiled JS" or "You seem to be requesting lots of little images...consider using an ImageBundle to optimize this". It's easy to imagine a lot more of these kinds of messages.

Very nice!
-- Bruce