Touch Action Timeout

[David Bokan]

Hi input-dev,

I've been looking at some touch-action browser tests and noticed we have lots of per-platform differences that originate from the fact that we only have a touch-action timeout on Android. Would anyone object if I removed the difference and enabled touch-action timeouts on all platforms?

Platform differences like this have a non-trivial cost in productivity and testing complexity. Additionally, if we think the timeout improves the touch experience on Android then I don't see why the same reasoning wouldn't apply on other platforms.

Is there anything I'm missing? If not, I'll send out a PSA to blink-dev.

Thanks,

David

[Timothy Dresser]

Just confirming, we're talking about this?
What timeout value would you use on desktop?

[David Bokan]

Correct. We'd use the desktop value of 200ms.

Sadly, it seems like this was planned to be temporary/an intermediate step and alludes to bug 504904 but that's since been closed without action :(. It'd be great if we could go the other way and remove the timeout altogether. I don't see any metrics on how often this gets used, perhaps step 1 would be to add those to see how often we hit it.

[Timothy Dresser]

Seeing how often we hit this sounds like a great first step!

I think applying a 200ms timeout could break a lot of sites.

[David Bokan]

On Tue, Apr 16, 2019 at 10:24 AM Timothy Dresser <tdre...@chromium.org> wrote:

I think applying a 200ms timeout could break a lot of sites.

 

I think you're right; looking at ScrollBegin.TimeToHandled_Main on Windows, the 99th percentile is over 200ms (the metric caps at 200ms so not sure how high it goes). But this metric is even higher on Android. I guess one reason why this is ok on Android is we expect many desktop pages to play poorly on a mobile device so breaking functionality for responsiveness is a worthwhile tradeoff.

We use a 1 second timeout on "mobile" pages assuming they should be well tested on mobile so we lean towards predictability for authors, is that an accurate assessment? WDYT about using the same timeout on desktops (i.e. a "native page" timeout). Looking at ScrollBegin.Touch.TimeToScrollUpdateSwapBegin4, we hit 1 second ~99.7th percentile. So I'd expect the timeout to hit about 3/1000 scrolls. 99.9th is ~2.5 seconds. I expect once we're over a second in delay the user experience would be improved by abandoning the touch action and allowing the gesture.

Note also: Compositor-touch-action is now enabled by default so at least some touch action queries can be answered from the compositor thread without consulting main. So these should be upper bounds.

[Xida Chen]

Hi,

I'd like to have a better understanding on this.

1. The touch action timeout is currently enabled on Android only, not enabled on desktop.

2. The timeout on mobile site is 1000ms and desktop site is 200ms.

Are these correct?

So David is proposing:

1. Enable touch action on desktop too? and

2. Set the time out for desktop site to 1000ms?

Or is it just 2?

[David Bokan]

I'd like to have a better understanding on this.

1. The touch action timeout is currently enabled on Android only, not enabled on desktop.

2. The timeout on mobile site is 1000ms and desktop site is 200ms.

Are these correct?

 

Yes 

So David is proposing:

1. Enable touch action on desktop too? and

2. Set the time out for desktop site to 1000ms?

I'm proposing enabling touch action timeout on desktop browsers as well and using 1000ms for all pages on desktop browsers.

i.e. In this case, the timeouts would be better called NativePageTimeout (1000ms) and LegacyPageTimeout (200ms). i.e. NativePageTimeout is used when pages are "designed for the platform", (viewport meta on mobile, all pages on desktop). Legacy is used when viewing a "desktop page" on Android.

[Rick Byers]

Sigh, this is a sad situation. In general I agree desktop and Android should be unified. But how exactly to best do so is IMHO non-trivial.

Ideally the passive touch listener intervention would be the right replacement for the touch ACK timeout. In particular, are there really still many cases of desktop pages on Android which have touch listeners which aren't covered by the intervention (and so can trigger the timeout)? Perhaps we can show that the intervention largely solved the issue (much better than the loose-loose situation of a timeout) for that specific case and so we could stop special-casing it at 200ms. Eg. maybe finch-trial the impact of removing the 200ms special-case?

Then, 1000ms is, to me, SO long, that (although it's hard to reason about and unpredictable), I'd be OK with applying it universally everywhere (if we block scrolling for a whole second, the whole UX is already so broken that the additional breakage caused by timeouts are arguably not fundamentally worse). There is some unique compat risk though, so I think it would still need to go through an intent process with metrics and probably some UKM analysis.

To be a legitimate intervention though it should really be avoidable by following best practices. In particular, is it already the case today that if all your touch listeners are passive, you're guaranteed not to hit the timeout? Eg. in rare cases where touch-action hit-testing needs to block on main and the device is over-subscribed due to no fault of the web site, there IMHO really shouldn't be a timeout. When I knew it I think the timeout logic was written to apply to any reason the browser might block on a touch-ACK from main - including chrome bugs. I'd feel a lot better if it applied only when there are non-passive touch listeners since the best practice (perhaps enforced by some future never-slow feature policy) is to use pointer events or passive touch listeners exclusively.

Rick

[Ojan Vafai]

What is the behavior of the timeout? I had it in my head that it would force the handlers to be passive. Does it actually prevent delivery of the event? If so, can much of the compat concern and the specific concern around not penalizing passive listeners be address this way?

[Rick Byers]

I believe the timeout still sends 'touchcancel' as if the app lost focus or something. Sometimes preventing double-handling is better than preventing missed events. But I agree it's inconsistent with the passive listener interventions we've done - there we found it was less breaking to force passive. I could see an argument for aligning.

[David Bokan]

Thanks for the detailed response. As usual, the situation is more nuanced and complex than I initially thought. Thanks for helping me understand :)

Rick Byers wrote:

Sigh, this is a sad situation. In general I agree desktop and Android should be unified. But how exactly to best do so is IMHO non-trivial.

Ideally the passive touch listener intervention would be the right replacement for the touch ACK timeout. In particular, are there really still many cases of desktop pages on Android which have touch listeners which aren't covered by the intervention (and so can trigger the timeout)? Perhaps we can show that the intervention largely solved the issue (much better than the loose-loose situation of a timeout) for that specific case and so we could stop special-casing it at 200ms. Eg. maybe finch-trial the impact of removing the 200ms special-case?

Unfortunately I couldn't find any metrics/counters on this. As a proxy though, it looks like ~18% of all scrolls on Android still fallback to the main thread due to a blocking touch handler (that'd be a lower-bound since it won't include handlers that called preventDefault). We can't tell how that distributes between mobile/desktop pages but that's higher than I expected. As a first step, I think it'd be good to add a metric on how often we hit the timeout and whether we hit the desktop or mobile timeout.

 

To be a legitimate intervention though it should really be avoidable by following best practices. In particular, is it already the case today that if all your touch listeners are passive, you're guaranteed not to hit the timeout? Eg. in rare cases where touch-action hit-testing needs to block on main and the device is over-subscribed due to no fault of the web site, there IMHO really shouldn't be a timeout. When I knew it I think the timeout logic was written to apply to any reason the browser might block on a touch-ACK from main - including chrome bugs. I'd feel a lot better if it applied only when there are non-passive touch listeners since the best practice (perhaps enforced by some future never-slow feature policy) is to use pointer events or passive touch listeners exclusively.

I like this idea. If, and only if, you have a blocking touch handler somewhere on the page, you'll be subject to the timeout. Using scroll metrics as a proxy again, "hit testing" reasons for falling to main account for about ~0.5% of cases and those are being actively driven down. It's much harder for an author to reason about how to avoid those ("how do I prevent my 'layers' from getting 'squashed'?") so perhaps no timeout in that case is preferable?

On Thu, Apr 18, 2019 at 5:03 PM Rick Byers <rby...@chromium.org> wrote:

I believe the timeout still sends 'touchcancel' as if the app lost focus or something. Sometimes preventing double-handling is better than preventing missed events. But I agree it's inconsistent with the passive listener interventions we've done - there we found it was less breaking to force passive. I could see an argument for aligning.

Seems like this would be something that requires looking case-by-case. Did this involve just looking at common cases and making a judgement call or was there something more quantitative?

 

tl;dr:

This'll require more care and attention than I initially thought and can spare right now so in the short term all I'll do is add some metrics that might be helpful for when I get some cycles in the future.

To confirm, Tim or Rick: are you aware if we track anything related? e.g. touch timeouts, main-thread touch-action checks, or how often we hit a blocking touch handler? If not I'll think through what'll be useful to measure and add.

Thanks!

[Dave Tapuska]

David did you look at: Event.Touch.TimedOutOnMobileSite/TimedOutOnDesktopSite ? That gives the number of times the timeout actually fired.

Likewise Event.Touch.FilteredAtPassthroughQueue also tracks the result codes and has similiar information.

dave.

[David Bokan]

Thanks, that's what I was looking for! The graphs look encouraging, they're falling steadily; we're below 0.1% for mobile pages and 0.6% for desktop pages where we activate the timeout.

[David Bokan]

Reviving this old thread as I recently discovered (for unrelated reasons) that the timeout is broken (or I don't understand how it works).

I'm using https://rbyers.github.io/touch-timeout.html - setting the delay to 3000ms and delaying touchstart. Scrolling remains blocked until the end of the delay which is much longer than the timeout. (touch-action: none has an exception that the timeout doesn't apply to it but you can change it to touch-action: pan-x and see the same effect over touch action). Do I understand the timeout behavior correctly? I bisected a behavior change back to Re-land "Enabled rAF aligned touch input" which seems unintentional and happened in M58 over three years ago. Prior to that change, scrolling would begin fairly quickly even with a large delay and a touchcancel is seen which is how I expect the timeout to work abased on the documentation from the chromestatus entry.

The metrics mentioned by Dave in the previous message are both low but non-0 (0.35% and 0.04% for desktop and mobile, respectively). Interestingly, they didn't appear to change at the time this shipped in 59.0.3036.0 but appear to correctly report non-timeouts in my test above.

Could someone confirm what I'm seeing and that my understanding of the timeout is correct? 

Thanks,
David