Exposing input hardware timestamps to the web

[Rick Byers]

Hey Ilya,

As we discussed briefly today, for developers to measure their own response times (and for input-linked animation cases) they really need an input timestamp that takes (at least) the queuing time (before being handled by the renderer main thread) into account.

We've been tracking this here.  Basically the debate has been stuck between two options:

 1) Change the semantics of Event.timestamp

  - WebKit has now done this in a largely compatible way

  - Firefox is attempting something more radical - changing the field to be a DOMHighResTimestamp

  - abarth@ and james@ had concerns about the compatibility / rationality of such a change, so instead we focused on:

2) Add a 'systemTime' property to Event (or UIEvent)

 - I pushed on this a few times, getting bounced back and forth between the web-perf WG (and HighResolutionTime spec) the DOM WG.  But couldn't really get any traction.

 - IE has a non-standard hwTimestamp field for this on some events

Our current plan is to wait for Mozilla to complete their experiments.  Either they'll find it's too breaking and join us in attempting #2, or find it works alright and we can just copy them.

Any opinion?  Is it worth trying to push more in WebPerf on defining a new property?  Or perhaps we should stop blocking on Mozilla to attempt their experiment and try it ourselves - see how many sites break.

Thanks,

   Rick

[Ilya Grigorik]

Thanks for the context.. this is really useful!

Whichever route we take, I would definitely like to see it as a DOMHighResTimestamp. All of the time's and durations in various perf-related specs are expressed as DOMHighResTimestamp's, so having the same type (and timebase) to work with is important -- e.g. I want to do compare event's timeStamp value with Frame Timing's startTime. 

Regarding the actual strategy.. That's a tricky one:

1. I'm all for updating current timeStamp to DOMHighResTimestamp. That said, not sure how we'd even track the potential breakage?
2. If we think that (1) is not doable or too risky, then extending Event with a new value is simple enough.

Reading the Mozilla bug, it sounds like it might take them a while to gather data. Would we be willing to drive it on our own? Do we have any data on how often that property is accessed? 

ig

[Rick Byers]

Yep, sounds good.

 

Reading the Mozilla bug, it sounds like it might take them a while to gather data. Would we be willing to drive it on our own?

Yes, if we think it's important enough.  I've wanted this forever, but have been nudging it along as a Pri-2.  At a minimum we could probably make it a priority for input dev in Q4 (might be too late to find someone to really drive it in Q3 unless we think it's worth replacing something else).

 

Do we have any data on how often that property is accessed? 

Nope, but we know that LOTS of apps have eventing frameworks that blindly copies event objects by reading all properties, so such "accessed" data probably wouldn't be useful anyway :-(

ig

[Ilya Grigorik]

Is there a "something else"? :) Getting this underway would go a long way towards making "R" measurable. 

  

Do we have any data on how often that property is accessed? 

Nope, but we know that LOTS of apps have eventing frameworks that blindly copies event objects by reading all properties, so such "accessed" data probably wouldn't be useful anyway :-(

Fair enough. Re-reading the whatwg thread [1]: Anne's argument is that between Safari, Chrome and FF we already have two different timebases and two different points at which timestamps are recorded... so changing to DOMHighResTimestamp shouldn't make things any more broken? The only wildcard is that we're switching the type.

I think it's worth trying. At a minimum, hack up a custom build and see if we can spot any obvious breakage ourselves before we inflict it on users?

ig

p.s. as one further +1 for DOMHighResTimestamp, we've recently clarified "time origin" for workers in HRT2 and defined translateTime which makes it possible to translate timestamps between different time origins (e.g. frames, workers, etc). 

[1] https://lists.w3.org/Archives/Public/public-whatwg-archive/2014May/0046.html

[Timothy Dresser]

Firefox nightlies on Linux are now using a DOMHighResTimestamp for Event.timestamp, so we could hunt for breakage there.

[Rick Byers]

Oh wow, I somehow missed the fact that Gecko has ALWAYS had an epoch-based event timestamp, that definitely lowers the risk.  Ok, I agree - we should just try making this change (behind --enable-experimental-web-platform-features for now). Majid has volunteered to drive this for M47 - I've assigned the bug to him.

I think it'll take a little more work to ship (eg. ensuring we're handling timestamps correctly on all platforms and all types of input, trying to get the edge case behavior specified well enough to address Adam and James's original concerns, etc.).  But I think the change should be easy enough that we might as well try behind a flag first and see what we can learn before worrying too much about this additional stuff. 

Dave raises a good point that analytics frameworks may be using event timestamps just to track "when" (in real time) a user did something, and breaking that may have no user-visible impact but still cause a lot of pain for developers.  But given the difference in timebases across browsers already, any such use case is already better off with Date.now() than relying on 'timestamp'.  So I still think we should try making this change.

Of course we'll ultimately need to do an intent-to-ship with the details of the tradeoff here and let the API owners decide.

[Ilya Grigorik]

Awesome!

 

I think it'll take a little more work to ship (eg. ensuring we're handling timestamps correctly on all platforms and all types of input, trying to get the edge case behavior specified well enough to address Adam and James's original concerns, etc.).  But I think the change should be easy enough that we might as well try behind a flag first and see what we can learn before worrying too much about this additional stuff. 

Dave raises a good point that analytics frameworks may be using event timestamps just to track "when" (in real time) a user did something, and breaking that may have no user-visible impact but still cause a lot of pain for developers.  But given the difference in timebases across browsers already, any such use case is already better off with Date.now() than relying on 'timestamp'.  So I still think we should try making this change.

  

Of course we'll ultimately need to do an intent-to-ship with the details of the tradeoff here and let the API owners decide.

Looking a bit ahead, what's the argument *for* updating the current timeStamp instead of introducing a new attribute? 

- existing code receives more accurate timestamps; no need to rewrite apps.

- normalized behavior between browsers (well, FF and Chrome... would Safari follow?)

- ... :)

[Rick Byers]

I'll ask Dean Jackson (Safari) his opinion on this when I see him in a couple days.  But presumably we'd aim to get this benefit regardless of the solution we choose.  Eg. if we decide this approach isn't sufficiently web compatible, then I suspect we could convince FF to treat 'timestamp' as legacy (return basically just Date.now()).

 

- ... :)

I believe the primary motivation is that it's only misleading / confusing to have a real-time based event timestamp.  It can't really be used perfectly correctly in any scenario.  So ideally we'd avoid confusion by having only a single timestamp property that makes sense and is universally useful.

[Ilya Grigorik]

Yep. I'm just trying to anticipate feedback from API owners.. Having Safari on board would be a compelling win.

  

- ... :)

I believe the primary motivation is that it's only misleading / confusing to have a real-time based event timestamp.  It can't really be used perfectly correctly in any scenario.  So ideally we'd avoid confusion by having only a single timestamp property that makes sense and is universally useful.

Sold! But I'm not the one we have to convince.