How Pointer Events solves the "scroll blocks on main thread" problem

1,132 views
Skip to first unread message

Rick Byers

unread,
Mar 18, 2015, 4:55:46 PM3/18/15
to Timothy Dresser, Chris Harrelson, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?
   Rick

On Wed, Mar 18, 2015 at 2:28 PM, Rick Byers <rby...@chromium.org> wrote:
I certainly think this is an idea worth exploring.  We want to make performance footguns more obvious to developers, this may be a good way to help in this case.  Note that the choice here is between scroll-blocks-on and an async event API.  touch-action is orthogonal (even with an async event API, you may need to use touch-action to declaratively disable certain gestures since you can no longer use preventDefault for that).

However, one of the use cases scroll-blocks-on was designed to address is where the incentives between components are misaligned and you want some form of "action at a distance".  Eg. we know various libraries (ad networks, etc.) attach touch event handlers, when they'd work perfectly fine with async events.  They may not be properly incentivized to update their code to switch to async handlers, but the page owner wants scrolling to be fast so we should give them the ability to disable scroll-blocking capability of components they're hosting.  Perhaps this is a false use-case though - we can only ever truly fix this sorts of problems through aligning the incentives by fixing our tooling to enable blame attribution.

The other trade off is ease of incremental adoption.  I think our guidance would be one of the following:

1) Find all code in your page (including frameworks etc.) that adds 'touch*' event handlers and evaluate whether each can be conditionally replaced with a 'touch*-async' handler when feature detection indicates your browser supports it.  Also update all code that consults Event.type to be aware of the -async variants.
 
-or-

2) Put 'scroll-blocks-on: none' on your document.  Test the various uses of touch events and if you find one where scrolling is occurring where you don't want it, apply 'touch-action' to that element to disable the unwanted scrolling (or, less desirably, apply 'scroll-blocks-on: start-touch' to re-enable scroll blocking for that element and it's descendants).

I think #2 is easier to apply incrementally to a large application (and so likely to have a bigger positive impact on real world performance) than #1, but I could be wrong.

Also, just from an optics perspective, if we tell people they need to listen to a new type of event, I'm sure we'll get the typical "Why is Google introducing a 3rd type of input events rather than just embracing pointer events which already solved this problem".  Perhaps an API shape like addAsyncEventHandler('touchstart',...) would be better anyway and avoid this concern.

Regarding wheel events.  Why does theverge have a wheel listener at all instead of just a scroll event listener?  What useful capability does a wheel event offer that a scroll event doesn't if you're not going to call preventDefault to disable scrolling?  If there's nothing, then we should be able to argue just as well that people should switch from consuming wheel events to consuming scroll events, rather than introduce an async wheel event.

Rick


On Wed, Mar 18, 2015 at 1:59 PM, 'Timothy Dresser' via input-dev <inpu...@chromium.org> wrote:
We were previously thinking that the scroll customization methods would only be called when "scroll-blocks-on: scroll-event" was set. It's a little more explicit that way, instead of transparently opting into main thread scrolling when overriding applyScroll or distributeScroll.

If all you're trying to do is synchronously respond to scrolling, you shouldn't really need to deal with the complexity of overriding the applyScroll method. It's a bit annoying to have to write:

var defaultScrollBehavior = el.applyScroll;
el.applyScroll = function(scrollState) {
  defaultScrollBehavior.call(el, scrollState);
  // Scroll response logic here.
}

That will get a bit cleaner with ES6 classes, but it's still more complicated than would be ideal. 
I suppose there is some benefit to keeping opting in to synchronous scrolling non-trivial though.

On Wed, Mar 18, 2015 at 1:48 PM Chris Harrelson <chri...@chromium.org> wrote:
I think the proposed scroll and distributeScroll methods would handle that case, right? Those are direct extensions of scroll functionality, and so need to be executed synchronously as part of scroll.

On Wed, Mar 18, 2015 at 10:45 AM Timothy Dresser <tdre...@google.com> wrote:
Would we consider adding a "scroll-sync" event, in the same vein as the "*-async" events? Synchronous scroll response is another problem which "scroll-blocks-on" addressed.

On Wed, Mar 18, 2015 at 1:26 PM Chris Harrelson <chri...@chromium.org> wrote:
Async events sounds really nice.

I think we need to move in general to a model where any computation which might take lots of time or is not high priority is async and schedulable.

This will unblock a lot of de-jank innovations within the browser, such as throttling, delaying during scroll gestures, etc.

Chris

On Wed, Mar 18, 2015 at 10:01 AM 'Nat Duca' via input-dev <inpu...@chromium.org> wrote:
Suppose we had "async flavors" of the classic touch events that are never preventDefaultable?

addEventListener('touch-*-async', ...)

And same for mouse, and especially wheel events.


There are a lot of snippets out there for presence monitoring, user interaction heatmapping, that add global handlers on touch and wheel. They only want passive indication that these things have happened. But because of the cancellable semantics of touch and mouse/wheel events, these handlers require us to go to the main thread.


We have things like touch-action: none. But, IMO, this is action at a distance: people add an event listener: asking them to think at the adding-site what semantics they want seems very intuitive, as compared to saying "oh, please go define your semantics over in css."

This seems nice in that its easily-plausibly compatible with other vendors' implementations...



(Fun fact: theverge has a wheel listener. So it can't scroll async on desktop. But we could probably very easily ask them to switch to an async wheel listener. Then theverge would fast scroll!)



~shrug~ insane? Thanks for reading this far!



- N


Chris Harrelson

unread,
Mar 19, 2015, 1:41:03 AM3/19/15
to Rick Byers, Timothy Dresser, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
On Wed, Mar 18, 2015 at 1:55 PM, Rick Byers <rby...@chromium.org> wrote:
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?

Yes, this makes a lot of sense. Thanks for explaining. More questions below...

This table uses the "sync" and "async" terms when describing the events. I'm not sure what meaning is intended, but based on the "sync" in that table and the MUST words elsewhere it sounds like it's required for those events to happen ASAP in the event loop. Therefore there would only a limited opportunity to slow down or schedule the processing of these events.

Rick Byers

unread,
Mar 20, 2015, 11:44:02 AM3/20/15
to Chris Harrelson, Timothy Dresser, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
On Thu, Mar 19, 2015 at 1:40 AM, Chris Harrelson <chri...@chromium.org> wrote:
On Wed, Mar 18, 2015 at 1:55 PM, Rick Byers <rby...@chromium.org> wrote:
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?

Yes, this makes a lot of sense. Thanks for explaining. More questions below...

This table uses the "sync" and "async" terms when describing the events. I'm not sure what meaning is intended, but based on the "sync" in that table and the MUST words elsewhere it sounds like it's required for those events to happen ASAP in the event loop. Therefore there would only a limited opportunity to slow down or schedule the processing of these events.

I'm a little fuzzy on the details of the terminology here.  In general "sync"/"async" is only meaningful relative to something else.  We were talking about events relative to scrolling (and so, in Chrome's architecture, relative to gesture event generation).  In that context, pointerdown/pointermove are definitely async with respect to scrolling / gesture production.

But that's not what the spec is discussing here.  Here's the definition the spec is referring to.  Basically "pointerdown" is guaranteed to be consistently ordered with respect to other events / DOM changes, but "gotpointercapture" is not.

So when there are no touch event handlers, we'd send all touch pointer* events to blink without blocking gesture production on the ACK back from the renderer.  This is the key thing that would ensure scrolling wouldn't be blocked by the page's Javascript.

Chris Harrelson

unread,
Mar 20, 2015, 12:21:14 PM3/20/15
to Rick Byers, Timothy Dresser, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
On Fri, Mar 20, 2015 at 8:43 AM, Rick Byers <rby...@chromium.org> wrote:
On Thu, Mar 19, 2015 at 1:40 AM, Chris Harrelson <chri...@chromium.org> wrote:
On Wed, Mar 18, 2015 at 1:55 PM, Rick Byers <rby...@chromium.org> wrote:
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?

Yes, this makes a lot of sense. Thanks for explaining. More questions below...

This table uses the "sync" and "async" terms when describing the events. I'm not sure what meaning is intended, but based on the "sync" in that table and the MUST words elsewhere it sounds like it's required for those events to happen ASAP in the event loop. Therefore there would only a limited opportunity to slow down or schedule the processing of these events.

I'm a little fuzzy on the details of the terminology here.  In general "sync"/"async" is only meaningful relative to something else.  We were talking about events relative to scrolling (and so, in Chrome's architecture, relative to gesture event generation).  In that context, pointerdown/pointermove are definitely async with respect to scrolling / gesture production.

But that's not what the spec is discussing here.  Here's the definition the spec is referring to.  Basically "pointerdown" is guaranteed to be consistently ordered with respect to other events / DOM changes, but "gotpointercapture" is not.

It also says ", and to user interaction". Do you know what that means? (Sorry if I sound pedantic. I only care in order to understand the flexibility available to us for this API.)

Also, since scroll body.scrollTop changes on scroll, isn't that a DOM change that must be respected? I guess that means that the event may be delayed, but its relative ordering to updates to scrollTop must be preserved? Does that mean that according to spec we can only update scrollTop at most once before processing the event (assuming it's spec'd that scrollTop updates before the event, don't know if that is true).

Rick Byers

unread,
Mar 20, 2015, 12:37:45 PM3/20/15
to Chris Harrelson, Timothy Dresser, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
On Fri, Mar 20, 2015 at 12:20 PM, Chris Harrelson <chri...@chromium.org> wrote

On Fri, Mar 20, 2015 at 8:43 AM, Rick Byers <rby...@chromium.org> wrote:
On Thu, Mar 19, 2015 at 1:40 AM, Chris Harrelson <chri...@chromium.org> wrote:
On Wed, Mar 18, 2015 at 1:55 PM, Rick Byers <rby...@chromium.org> wrote:
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?

Yes, this makes a lot of sense. Thanks for explaining. More questions below...

This table uses the "sync" and "async" terms when describing the events. I'm not sure what meaning is intended, but based on the "sync" in that table and the MUST words elsewhere it sounds like it's required for those events to happen ASAP in the event loop. Therefore there would only a limited opportunity to slow down or schedule the processing of these events.

I'm a little fuzzy on the details of the terminology here.  In general "sync"/"async" is only meaningful relative to something else.  We were talking about events relative to scrolling (and so, in Chrome's architecture, relative to gesture event generation).  In that context, pointerdown/pointermove are definitely async with respect to scrolling / gesture production.

But that's not what the spec is discussing here.  Here's the definition the spec is referring to.  Basically "pointerdown" is guaranteed to be consistently ordered with respect to other events / DOM changes, but "gotpointercapture" is not.

It also says ", and to user interaction". Do you know what that means? (Sorry if I sound pedantic. I only care in order to understand the flexibility available to us for this API.)

No, I don't.  I've never fully understood this.  Frankly I care more about developer-exposed interoperability with the other engines (and compat impact on real-world sites) than I do about spec word-smithing here.  I suspect there's a reason the WhatWG DOM spec is so much simpler in this regard.

Also, since scroll body.scrollTop changes on scroll, isn't that a DOM change that must be respected? I guess that means that the event may be delayed, but its relative ordering to updates to scrollTop must be preserved? Does that mean that according to spec we can only update scrollTop at most once before processing the event (assuming it's spec'd that scrollTop updates before the event, don't know if that is true).

You'd think so, eh <grin>. Yet AFAIK browsers switched to threaded scrolling without much (any?) official spec impact.  Sigh.

Basically I believe this is all technically unspec'd (or full of spec bugs).  Eg. the ordering between scroll, RAF and input events is definitely not spec'd anywhere (not even consistent between browsers), and it's a known spec bug that we don't define hit testing at all anywhere!  The TouchEvents spec in particular is quite loose on a number of things to ensure it captured the variety of behavior of existing browsers.

Anyway, I'm not trying to convince you these aren't important questions.  Just that the behavior of the other browsers and expectations of web developers are the primary issue, and incrementally improving the specs to better reflect that is a secondary ongoing concern.

Chris Harrelson

unread,
Mar 20, 2015, 12:48:05 PM3/20/15
to Rick Byers, Timothy Dresser, Nat Duca, input-dev, Alex Russell, Paul Lewis, Matt Gaunt, Paul Irish
On Fri, Mar 20, 2015 at 9:37 AM, Rick Byers <rby...@chromium.org> wrote:
On Fri, Mar 20, 2015 at 12:20 PM, Chris Harrelson <chri...@chromium.org> wrote

On Fri, Mar 20, 2015 at 8:43 AM, Rick Byers <rby...@chromium.org> wrote:
On Thu, Mar 19, 2015 at 1:40 AM, Chris Harrelson <chri...@chromium.org> wrote:
On Wed, Mar 18, 2015 at 1:55 PM, Rick Byers <rby...@chromium.org> wrote:
Chris asked me to expand on my comment that pointer events was designed from the beginning to solve this problem, so I figured I might as well do that here.

I got this sentence added to the spec introduction:
"An additional key goal is to enable multi-threaded user agents to handle default touch actions, such as scrolling, without blocking on script execution."  But it doesn't spell out exactly how.  The key bit is this:
"For touch input, the default action of any and all pointer events must not be a manipulation of the viewport (e.g. panning or zooming)."

This means that the browser doesn't need to wait to see if the page calls 'preventDefault' on a pointer event before scrolling, because scrolling isn't a "default" action.  Instead scrolling is enabled / disabled only declaratively through the touch-action property

We already support touch-action in Chrome, but for compatibility we need some knob for developers to tell the browser to depend only on it.  We considered just adding a new value to touch-action (to differentiate the default 'auto' from 'all enabled'), but for various technical reasons that wasn't going to work well.  So instead I proposed "touch-action-delay" boolean CSS property for this (saying just whether touch-actions should be delayed on touch event handlers or not).  Then we realized this was a special case of a more general problem not limited strictly to touch, hence scroll-blocks-on.

Make sense?

Yes, this makes a lot of sense. Thanks for explaining. More questions below...

This table uses the "sync" and "async" terms when describing the events. I'm not sure what meaning is intended, but based on the "sync" in that table and the MUST words elsewhere it sounds like it's required for those events to happen ASAP in the event loop. Therefore there would only a limited opportunity to slow down or schedule the processing of these events.

I'm a little fuzzy on the details of the terminology here.  In general "sync"/"async" is only meaningful relative to something else.  We were talking about events relative to scrolling (and so, in Chrome's architecture, relative to gesture event generation).  In that context, pointerdown/pointermove are definitely async with respect to scrolling / gesture production.

But that's not what the spec is discussing here.  Here's the definition the spec is referring to.  Basically "pointerdown" is guaranteed to be consistently ordered with respect to other events / DOM changes, but "gotpointercapture" is not.

It also says ", and to user interaction". Do you know what that means? (Sorry if I sound pedantic. I only care in order to understand the flexibility available to us for this API.)

No, I don't.  I've never fully understood this.  Frankly I care more about developer-exposed interoperability with the other engines (and compat impact on real-world sites) than I do about spec word-smithing here.  I suspect there's a reason the WhatWG DOM spec is so much simpler in this regard.

Also, since scroll body.scrollTop changes on scroll, isn't that a DOM change that must be respected? I guess that means that the event may be delayed, but its relative ordering to updates to scrollTop must be preserved? Does that mean that according to spec we can only update scrollTop at most once before processing the event (assuming it's spec'd that scrollTop updates before the event, don't know if that is true).

You'd think so, eh <grin>. Yet AFAIK browsers switched to threaded scrolling without much (any?) official spec impact.  Sigh.

Basically I believe this is all technically unspec'd (or full of spec bugs).  Eg. the ordering between scroll, RAF and input events is definitely not spec'd anywhere (not even consistent between browsers), and it's a known spec bug that we don't define hit testing at all anywhere!  The TouchEvents spec in particular is quite loose on a number of things to ensure it captured the variety of behavior of existing browsers.

Anyway, I'm not trying to convince you these aren't important questions.  Just that the behavior of the other browsers and expectations of web developers are the primary issue, and incrementally improving the specs to better reflect that is a secondary ongoing concern.

I definitely agree. (aside: That was the intent of my pedantic-referencing aside...LOL)
Reply all
Reply to author
Forward
0 new messages