Rail Scrolling

[Timothy Dresser]

I think we should move all scroll rail application into our input handling logic, and remove it from the scroll bubbling logic.

For touch scrolling, we currently restrict the axis of movement in two places: during gesture recognition and during scroll bubbling.

With touchpad we only restrict the axis of movement during scroll bubbling.

The logic for applying rails during scroll bubbling is inconsistent between the main thread and the compositor thread. (crbug.com/457765, crbug.com/466991)

I think that moving all rail logic into our input handling code, and removing it from the scroll bubbling logic would have several advantages:

• This portion of the input handling logic is common for main and impl thread scrolling, so we'd remove some inconsistencies.
• The scroll bubbling logic is complicated, and there are a number of cases where diagonal scrolling across multiple scrollable areas feels jumpy. I believe this simplification would improve these cases.
• I think scroll rails should be input device specific.

Does this sound reasonable? 

[Christopher Cameron]

I like this idea (and am getting irked by their absence -- users are noting this, e.g, with https://code.google.com/p/chromium/issues/detail?id=452592#c37, https://code.google.com/p/chromium/issues/detail?id=459347#c7, some other places too).

I hope that we'll have sufficient signals at the input event level to be able to implement effective rails (e.g, do natural-feeling rails need to know something about the page and its scroll limits).

I also agree that this is likely to be device-specific -- for instance, on Mac we have "phase events" that bracket each scroll gesture (which can help out a lot), and I hear that some trackpads have rails somewhere in the OS stack.

Do you think this "filtering" belongs in the browser or in the renderer?

[Timothy Dresser]

For touch, it definitely belongs in the browser, with the rest of gesture recognition.
I'm not particularly familiar with the event flow for touchpad, but I'm inclined to think that rail application belongs in the browser for touchpad as well.

I have a hunch that we'll be able to get good results without knowing about the scroll limits, but I have no data to back that up. We'll need to do some experimentation. Are there any specific cases we're aware of where page topology should potentially impact how rails are applied?

[Alexandre Elias]

We do intentionally rail more aggressively depending on the page topology.  The code that makes hard diagonal scrolling "jumpy" across bubbled layers is there (https://code.google.com/p/chromium/codesearch#chromium/src/cc/trees/layer_tree_host_impl.cc&q=layer_tree_Host_impl&sq=package:chromium&l=2704 ) because when scrolling a sublayer vertically while zoomed in, we found it's really annoying when the parent moves slightly horizontally.  I don't think we can change this, but when we do the big scroll refactoring I proposed, that should solve the problem of the inconsistent behavior on main and impl thread for the bubbling-related logic.

On Android I believe the non-topology-related railing is performed somewhere in the java code in the browser process.  I'm not sure right now whether it's in the system library code or not, if it is we'll need to have a separate implementation for other platforms, otherwise we could remove it from there and have a unified implementation in C++.

[Christopher Cameron]

[Hijacking this for Mac a bit, and adding some Mac folks]

TL;DR: Some simple filtering of trackpad events on Mac makes trackpad scrolling (especially when zoomed in or on a page that can scroll in both directions) much more pleasant. Should I go ahead and clean up this & submit it, or is way too hacky for peoples' tastes?

Longer version:

• "Rails" here means preferring scrolling just horizontally or vertically (instead of diagonally).

• I really don't like the rails that are applied in Safari -- they latch to one direction or the other too aggressively, so you end up with a staircase instead of diagonal motion. Of note is that, by comparison with all other Mac apps, rails are an anomaly (and Safari's are particularly bad).

• Blink also effectively applies ineffective rails by its non-independent bubbling of X and Y components of scrolls, which isn't much fun. This is bad with and without this sort of patch. How high on the list of priorities is doing X and Y independently in Blink? Is this in the deprecated/not-ready-yet trap?

• I spent a while this afternoon prototyping various rails systems ... and I really like the feel of the following very simple one (https://codereview.chromium.org/1008993002). It just has a running average of deltaX and deltaY (with a decay applied), snaps to one axis initially, allows you to break off if your running-average vector is sufficiently off-rails, and allows you to snap-back-on if your running-average vector gets back.
• This is after having tried a lot more complicated types of hysteresis).
• I didn't find that the LTHI code that Alexandre pointed to interfered with this.

[Timothy Dresser]

Alex: do we need to rail more aggressively depending on page topology, or can we just always rail that aggressively?

"when scrolling a sublayer vertically while zoomed in, we found it's really annoying when the parent moves slightly horizontally."

Is it any less annoying when you're scrolling a single layer vertically and you accidentally scroll horizontally slightly? It seems to me that we should rail aggressively enough that we prevent accidental scrolling, regardless of page topology.

For touch, we do non-topology related railing during gesture recognition, which is shared across all platforms.

Chris: I'm still hopeful that we can get rid of the topology dependent railing logic, which would likely change which variety of topology independent railing logic feels best. We may want to hold off until we've definitively established whether or not the topology dependent railing logic is necessary.

Trackpad railing logic should be cross platform, shouldn't it? Is there any reason that trackpad rails on mac should feel different than they do on ChromeOS or Windows?

[Christopher Cameron]

Most of the rest of this email reads "Mac is different, Mac is weird, Mac is special" -- it may be that Mac should take on this issue differently from other platforms.

On Mon, Mar 16, 2015 at 5:40 AM, Timothy Dresser <tdre...@google.com> wrote:

Alex: do we need to rail more aggressively depending on page topology, or can we just always rail that aggressively?

"when scrolling a sublayer vertically while zoomed in, we found it's really annoying when the parent moves slightly horizontally."

Is it any less annoying when you're scrolling a single layer vertically and you accidentally scroll horizontally slightly? It seems to me that we should rail aggressively enough that we prevent accidental scrolling, regardless of page topology.

(Actually, it was the unidirectional bit of this that was the most helpful -- I'll try some experiments not eliminating the horizontal component).

 

For touch, we do non-topology related railing during gesture recognition, which is shared across all platforms.

Of note is that Mac gestures are somewhat different (fling in particular) in that we do not recognize the fling -- the OS does, and it sends us additional input events after the fingers have been lifted, representing the "momentum phase" of the gesture.

 

Chris: I'm still hopeful that we can get rid of the topology dependent railing logic, which would likely change which variety of topology independent railing logic feels best. We may want to hold off until we've definitively established whether or not the topology dependent railing logic is necessary.

Trackpad railing logic should be cross platform, shouldn't it? Is there any reason that trackpad rails on mac should feel different than they do on ChromeOS or Windows?

On Mac, we get a signal when each new gesture begins. For instance, if you scroll for a bit, lift your fingers, then scroll again, Mac actually tells us about the lifted fingers in between, which allows us to reset all of the hysteresis. IIUC Windows and CrOS don't have this (which makes life a lot harder).

I've also been led to believe that some Windows/CrOS devices implement rails inside the driver.

[Alexandre Elias]

]On Mon, Mar 16, 2015 at 5:40 AM, Timothy Dresser <tdre...@google.com> wrote:

Alex: do we need to rail more aggressively depending on page topology, or can we just always rail that aggressively?

"when scrolling a sublayer vertically while zoomed in, we found it's really annoying when the parent moves slightly horizontally."

Is it any less annoying when you're scrolling a single layer vertically and you accidentally scroll horizontally slightly? It seems to me that we should rail aggressively enough that we prevent accidental scrolling, regardless of page topology.

I think there really does need to be two different kinds of railings.  The normal non-topological railing is carefully tuned on Android to have a light touch, giving just enough resistance to allow a perfectly vertical scroll while also making it easy to start moving diagonally at will.  Meanwhile, when scrolling a sublayer, scrolling the sublayer and the parent layer on different axes is not at all useful and looks almost buggy when it happens so we suppress that scenario very aggressively.

[Timothy Dresser]

I hadn't considered that in order to implement something like -ms-scroll-rails, we can't apply rails without paying attention to page topology.

Perhaps the most flexible approach would be if when input comes, we don't apply rails to the delta at all, but we annotate the event with a "rail state", which is one of VERTICAL, HORIZONTAL, or NONE.

Then when we're actually applying a scroll delta to a scroller, we can choose whether to respect or disregard the event's rail state, and apply more strict railing due to page topology if necessary.

To fully make this change, we'd want to receive unrailed deltas from touchpad/touchscreen drivers as much as possible.

Does this approach sound reasonable? It doesn't completely centralize the railing logic (as some of that logic is device dependent), but it would move the actual application of the rails to a single place in the pipeline.

[Christopher Cameron]

On Wed, Mar 18, 2015 at 5:47 AM, Timothy Dresser <tdre...@google.com> wrote:

I hadn't considered that in order to implement something like -ms-scroll-rails, we can't apply rails without paying attention to page topology.

Perhaps the most flexible approach would be if when input comes, we don't apply rails to the delta at all, but we annotate the event with a "rail state", which is one of VERTICAL, HORIZONTAL, or NONE.

Then when we're actually applying a scroll delta to a scroller, we can choose whether to respect or disregard the event's rail state, and apply more strict railing due to page topology if necessary.

To fully make this change, we'd want to receive unrailed deltas from touchpad/touchscreen drivers as much as possible.

Does this approach sound reasonable? It doesn't completely centralize the railing logic (as some of that logic is device dependent), but it would move the actual application of the rails to a single place in the pipeline.

I agree that this is the right thing to do.

We had discussed applying rails to the input events themselves, but I now think that to be a bad idea, as wheel listeners may want to receive non-railed deltas.

In Safari, listeners receive non-railed deltas, but the scrolls are very aggressively railed (it appears that they just make the smaller component zero).

I'm tracking some work on this in crbug.com/468454 -- I feel really-not-great about releasing the pinch-zoom work with this issue left un-resolved.

[Christopher Cameron]

I've implemented this in

Blink: https://codereview.chromium.org/1018183002/

Chromium: https://codereview.chromium.org/1019153002/

Note that the Chromium version is intentionally sloppy (it applies the rails state in InputHandlerProxy instead of InputHandler). The reason for that is that the full fix which adds a rails argument to InputHandler::ScrollBy (which I'll put up next) touches tons of callers, and won't be suitable to merge.

Does this seem like a reasonable starting point to everyone?

[Rick Byers]

I skimmed the CLs and left a few comments. This general approach seems reasonable to me.  I don't think there's any escaping two key points:

1) Rail definition may always be device and platform dependent somehow (eg. on chromeOS I think each touchpad has configuration for this)

2) When and how exactly rails are applied should be up to the consumer.  Eg. JS handling wheel events, or a custom scroller being used to pan an image.

These are what led Tim and me to conclude this design was the way to go.  And these CLs seem like a step in the right direction (again I didn't review all changes in detail).

Rick