Defining "time to first frame"

[Rick Byers]

One of our goals for this quarter to is add measurements for "time to first frame".  I'd like to make a proposal for how exactly we define this.

I think the main case we were thinking of was apps that have a touchstart handler and do some painting in response to first being touched.  Here we should measure the time from the touchstart event being generated to the swap.

Other scenarios may do their handling on touchend instead of touchstart (eg. button clicks that have no affordance).  So what if the touchstart doesn't generate any frames, but the touchend does?

A similar scenario I think is very important to measure is time to scroll start (we've got 3 distinct features to try to reduce this - has-touch-handlers flag, ACK timeout, and cc touch hit testing).  In this case I don't think we want the time from the touchstart event as that would make the measurement sensitive to scroll velocity (slower scrolls have more time spent in the 'slop').  We really want the time from the first non-slop touchmove.

I propose that the measurement we add should handle all three of the above cases.  Basically it's "for any touch sequence, how long did it take between the first frame being drawn and the input event that triggered that frame".  One way to implement this as a telemetry measurement would be to look for the first latency info at or after a touchstart that has a swap component and report it.

Thoughts?

   Rick

[Jared Duke]

That seems very reasonable, particularly if we can use the same approach for all relevant cases.  There may be some subtle complexities, like ensuring the TapDown event doesn't generate a swap (link highlighting?) when we're really interested in GestureScrollUpdate, but nothing comes to mind immediately that can't be addressed with care.

How would we feel about including "time to first fling frame" under the general "time to first frame" umbrella?  This is another very important metric that's completely off the current telemetry radar.  It's cropped up several times in the last short while, e.g., fling debounce on Aura, main-thread fling jank with touch absorption, and would also test basic compositor animation scheduling responsiveness.  The measurement here may be a bit trickier, but if we want to be jank free in the best case, that should include jank free touch to fling transitions.

[Nat Duca]

I wonder if we start with tools/perf/metrics/smoothness.py --- this has the latency events and the frame drawn events in it, and is what we use for our current smoothness metrics.

This metric begins its life with a parsed version of an about:tracing  recording, and an "interest range" --- a start and end time during which the page was doing the smooth thing.

For various reasons, it doesn't know that a fling or whatever was happening. I think those various reasons are good and I'd like to keep that separation.

So, with that in mind, two questions:

1) can we come up with a way to get a "time to first frame" by examining a trace to figure out what happened rather than having advance knowledge of what was happening? E.g. same code reports the right thing for a trace where there was (a) a fling, (b) a regular touch scroll, (c) a mouse scroll, and (d) javascript ran independent of input and eventually updated the screen.

2) for a given time range where only one user gesture ocurred, is there more than one "time to first X"? Or, is there just a "time to first frame"? E.g. you'd never have a scroll and a tap in the range. You can probably guess, I'd prefer one metric, maybe computed slightly differently depending on what is happening during the interval.

Note: 1d is important. Lets not forget it... time to first frame is valid for, e.g. chat windows that appear. :) Doing that one is ... tricky too, and possibly worth thinking through first: its constraints might shape a lot of the remaining thinking.

Any progress on this would be huge. Even a simplistic version of time to first frame that, for an active range, reported the time when the main thread frame's data made it up onto the compositor thread would be, even though not completely accurate, a HUGE step forward in terms of test coverage.

- N

[Rick Byers]

I didn't realize we were hoping to capture the non-input cases as part of this metric too, but I agree they're important and that we should try to handle them the same if possible.  I'm OK with trying to rely on the "interest range" as our start point without using InputLatency at all - it's certainly simplest and we can see where that takes us and iterate as necessary.

As a first step I think it would be interesting to try testing this approach on one simple page that does a synthetic tap on something that responds (eg. changes opacity) on touchstart.  We should be able to get the time to first frame very low (~16ms?) there as a proof of concept (proving, for example, that we can rely on the "interest range" being precise).  Then we can start looking at the more interesting existing smoothness cases and expand it for cc touch hit testing (scrolling at different co-ordinates within and outside touch handler regions with a synthetic main thread delay).

Rick

[Rick Byers]

On Tue, Apr 8, 2014 at 9:50 PM, Jared Duke <jdd...@chromium.org> wrote:

That seems very reasonable, particularly if we can use the same approach for all relevant cases.  There may be some subtle complexities, like ensuring the TapDown event doesn't generate a swap (link highlighting?) when we're really interested in GestureScrollUpdate, but nothing comes to mind immediately that can't be addressed with care.

How would we feel about including "time to first fling frame" under the general "time to first frame" umbrella?  This is another very important metric that's completely off the current telemetry radar.  It's cropped up several times in the last short while, e.g., fling debounce on Aura, main-thread fling jank with touch absorption, and would also test basic compositor animation scheduling responsiveness.  The measurement here may be a bit trickier, but if we want to be jank free in the best case, that should include jank free touch to fling transitions.

I would like to track "time to first fling frame" as some point, but I think it's in conflict with the goal of trying to have a single generic "time to first frame" measurement (since there will always be some other frame before the fling).  I think we'd have to create a special metric just for it (time between last scroll frame and first fling frame), and even then I wonder if that's really all we want here.  The fling stutter issue Ian filed (https://code.google.com/p/chromium/issues/detail?id=353702) appears to be actually generating frames consistently but just with the wrong scroll velocities across the transition.  So maybe it's more about tracking scroll velocity across a fling making sure that it's a smooth transition?  Regardless I think this is important but probably not something we can fit into Q2 (although I'd still like to see some targeted fixes without all the perf infrastructure to verify them).

Rick

[Nathaniel Duca]

Hmm @jdduke this concept of fling or velocity continuity reminds me of studying the scroll position graph  in about:tracing over time. I wonder if there's a generalization about this that we can use in the long term ---- e.g. anything we want to be smooth should be smooth in its value curve as well, when graphed with the value that is used for interpolation and the time that value went onscreen. Discontinuities become jank. Would apply to any property that was changing over N frames, not just scrollTop. Not quite sure how to stitch that a benchmark though. :)