Intent to Implement: Native Caret Browsing

[Bruce Long]

Contact emails

Bruce...@microsoft.com, Amit...@microsoft.com, Grisha....@microsoft.com

 

Explainer

Native Caret Browsing Explainer

 

Design Doc

Native Caret Browsing Design Document

 

A TAG review is not requested, because the feature is not intended to be introduced as web standards. It is not expected that web developers will need to make changes to their content as a result of this work

 

Summary

We are proposing the implementation of native caret browsing in Chromium. In caret browsing a moveable cursor is placed on a web page, allowing a user to select and navigate text with just a keyboard. Caret browsing mode will be toggled by an activation key (F7), with a confirmation dialog displayed. The native implementation of this feature will obviate the need to install a browser extension. 

 

Motivation

Caret browsing enables users to navigate web content using the keyboard keys and common shortcuts for character, word and line level navigation. Caret browsing allows a full range of text navigation and selection functionality within web content without relying on additional pointing devices like mice, trackpads and touchpads, so is an important accessibility feature.

 

Today Chromium users can download a Caret Browsing extension from the Chrome Web Store. There are several problems with this approach:

 

• Extensions might be blocked in a work environment due to enterprise policies (see Issue 611798: Enterprise users can't install accessibility extensions).
• Extensions might not be available in "Incognito" tabs, or when a Guest profile is used.
• There are additional barriers for users of all abilities, since they need to find and install an extension and the functionality is not readily available when it might be needed.
• Applications that use Chromium don't have the platform capability to enable native caret browsing.

 

Risk

Interoperability and Compatibility

Mozilla Firefox, Microsoft Edge and Internet Explorer already natively support caret browsing. Native caret browsing doesn't aim to replace extensions; they would continue to work as they do today having the first opportunity to handle the default activation shortcut. 

 

• IE: Shipped
• Edge: Shipped
• Chromium: In progress
• Firefox: Shipped
• Safari: N/A (there is system-wide assistive support)
• Web/Framework developers: N/A

 

Ergonomics

Performance should not be significantly impacted. Native caret browsing will rely on the same implementation for rendering a caret and moving it around as already used within editable content in Chromium.

 

Activation

The feature is not exposed to the web API layer. The feature will initially be behind a runtime flag and disabled by default.

 

Debuggability

No special DevTools support is required to debug this feature. 

 

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes, with the caveat that function keys such as F7 may not be available and so alternative shortcuts might be needed.

 

Link to entry on the feature dashboard

TBD--but not a web-facing change in Blink. 

 

Requesting approval to ship?

No. The feature will initially be implemented behind a runtime flag.

 

 

 

Thanks.

 

--Bruce

[flo...@rivoal.net]

Hi,

I know the way caret browsing works is different from the way spatial navigation works, but I'm under the impression that they are largely motivated by similar use cases. I am less familiar with caret browsing than with spatial navigation, so I may be wrong about that though.

Maybe the two are motivated by sufficiently different use cases that we need both, but I would suggest looking into this before going to far.

Here are some links:

(slighlty out of date) explainer: https://drafts.csswg.org/css-nav-1/explainer

specification: https://drafts.csswg.org/css-nav-1/

—Florian (co-editor of the spatial navigation specification)

[Dominic Mazzoni]

I'm glad to see this!

Several years ago I tried implementing native caret browsing. One of the things I did was modify

the web tests (then layout tests) so that at the end of every test it'd try to start with the cursor

at the top of the document and arrow all the way to the bottom. The test would pass if the

cursor reached the bottom. I hit thousands of cases where the cursor got stuck, indicating a bug

in the text editing code. That led me to implement the extension instead.

I definitely think native caret browsing would be the best, though.

Another issue was that I couldn't figure out how to make it possible for users to get in and

out of iframes. I didn't see you address that issue in your proposal. I don't believe other

browsers necessarily handle this, but I think it'd be good to at least be explicit as to what's

in and out of scope.

Along those lines, which form controls ought to be possible to work with and which ones are

out of scope?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MWHPR2101MB0874DFEE466AAD72ADCA8E75865F0%40MWHPR2101MB0874.namprd21.prod.outlook.com.

[Manuel Rego Casasnovas]

On 26/03/2019 19:26, 'Bruce Long' via blink-dev wrote:
> We are proposing the implementation of native caret browsing in
> Chromium. In caret browsing a moveable cursor is placed on a web page,
> allowing a user to select and navigate text with just a keyboard. Caret
> browsing mode will be toggled by an activation key (F7), with a
> confirmation dialog displayed. The native implementation of this feature
> will obviate the need to install a browser extension.

I guess caret navigation will follow DOM order, this might cause
confusion to the user as you can see the caret jumping to "weird" places
when you have different layout methods like: floats, absolute positions,
grid layout, ...

A few days ago talking to Alice Boxhall we were thinking about the
possibilities to improve TAB navigation on grid layout so it doesn't
follow DOM order always, instead it could follow the "visual" order
defined by grid layout (first one row, then the next one, etc.).
Probably Alice can provide more information about these ideas.

Have you think about this issue related to caret navigation?
Would be possible to make it closer to "visual" order than DOM order?

Bye,
Rego

[Dominic Mazzoni]

On Wed, Mar 27, 2019 at 5:49 AM Manuel Rego Casasnovas <re...@igalia.com> wrote:

A few days ago talking to Alice Boxhall we were thinking about the
possibilities to improve TAB navigation on grid layout so it doesn't
follow DOM order always, instead it could follow the "visual" order
defined by grid layout (first one row, then the next one, etc.).
Probably Alice can provide more information about these ideas.

Have you think about this issue related to caret navigation?
Would be possible to make it closer to "visual" order than DOM order?

Keep in mind that any change you make for caret navigation

would probably also affect caret movement in contenteditable,

unless you deliberately wanted to decouple them.

Also, not following the DOM order would make it really hard to

implement copying to the clipboard.

[Aaron Leventhal]

Hi, I implemented caret browsing in Mozilla ages ago. Here are some things I wish I had thought of.

Some users get caret browsing turned on by accident, even when there is a dialog. They might hit F7 but not read the dialog, and just hit OK. People do this all the time. Or they don't know what a caret is. Finally, they might be curious to try it but don't remember the key to turn it off. I've seen users who have caret browsing on but didn't realize they had it on at all, because they weren't technical enough to realize arrow keys were no longer doing the same thing.

Therefore, it's best if caret browsing is turned off when the browser relaunches, or only sticks to the current tab, or something so that when it's turned on accidentally, the damage doesn't last long.

Aaron

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFz-FYyzVt8KE2m34Gi_CFjep14Fy_ECL3R0B8_pXPHpbuOwPA%40mail.gmail.com.

[Dominic Mazzoni]

Good point. I think how to actually enable it will be a separate discussion with Chrome UX.

The scope of what Microsoft could contribute at this point would be to implement

full support for caret browsing behind a flag.

[brl...@microsoft.com]

Navigating across iframes using just the keyboard would be out of scope for this proposal. To my knowledge there is no means to do this in EdgeHtml or FireFox either. I will take a deeper look at the question regarding form controls.

On Tuesday, March 26, 2019 at 10:59:11 PM UTC-7, Dominic Mazzoni wrote:

I'm glad to see this!

Several years ago I tried implementing native caret browsing. One of the things I did was modify

the web tests (then layout tests) so that at the end of every test it'd try to start with the cursor

at the top of the document and arrow all the way to the bottom. The test would pass if the

cursor reached the bottom. I hit thousands of cases where the cursor got stuck, indicating a bug

in the text editing code. That led me to implement the extension instead.

I definitely think native caret browsing would be the best, though.

Another issue was that I couldn't figure out how to make it possible for users to get in and

out of iframes. I didn't see you address that issue in your proposal. I don't believe other

browsers necessarily handle this, but I think it'd be good to at least be explicit as to what's

in and out of scope.

Along those lines, which form controls ought to be possible to work with and which ones are

out of scope?

On Tue, Mar 26, 2019 at 2:27 PM 'Bruce Long' via blink-dev <blin...@chromium.org> wrote:

Contact emails

Bruc...@microsoft.com, Amit...@microsoft.com, Grisha...@microsoft.com

[brl...@microsoft.com]

Thanks for sharing the information on spatial navigation, a proposal that I was not aware of. It does look like the use cases for spatial navigation and caret browsing are similar. And the spatial navigation spec would allow page authors to customize behavior. However, we already have developed a working implementation of native caret browsing with navigation behavior aligned with that already familiar from Original Edge and Firefox. And as the design doc indicates, the navigation behavior also matches that currently found for editable content in Chromium. What is the timeframe for implementing spatial navigation in Chromium? If it is some time off, then I would suggest that taking this native caret navigation implementation (which is a relatively small code change) would provide an improved user experience in the short term, particularly with regards to accessibility.

[Daniel Bratell]

Hi, I implemented caret browsing in Opera Presto ages ago and I approve of this but:

* Depending on the use case it might be best to keep it on/off per document (which was how we did it). If the user wants to arrow around in the document I think they can handle prefixing it with F7 once per document. This would solve the problem aleventhal mentions too.

* I also found it to be a really good stress case of the editing code, caret code and selection code. I imagine you will have to fix a lot of bugs in that area.

* Compared to proper spatial navigation this might be a feature that won't get that many users but if my experience from Presto maps to Blink, it will be 90% improving editing and 10% the rest so it will probably be a light weight feature and the way there will be valuable in itself.

(I think my main use case was shift-arrow to manipulate selections when it was hard to do with the mouse)

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFz-FYyX6Lj5Tx61gkTycDCNR%3D-maEturCDQ5hBSMG6PgH%3D58Q%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[Dominic Mazzoni]

On Thu, Mar 28, 2019 at 9:20 AM Daniel Bratell <bra...@opera.com> wrote:

* I also found it to be a really good stress case of the editing code, caret code and selection code. I imagine you will have to fix a lot of bugs in that area.

That was the main reason I gave up on it. Easy to get it working, hard to make it robust. People rarely test editing code on complex layouts.

As for spatial navigation, I see some overlap but I don't think they're the same. We shouldn't ship caret browsing if we plan to replace it with spatial navigation later, but I'd rather think about how they could play nicely together. There are plenty of things you can do with caret browsing that you can't do any other way - like selecting text to copy and paste without using the mouse.

 

[jh....@lge.com]

Hi,

I think there are several differences between caret navigation and spatial navigation.

• Need for the modifier key:
    Caret navigation uses the modifier key (F7) to activate the feature, but spatial navigation doesn't.
  
  
• The target element for navigation:
    Caret navigation can reach to the non-focusable elements (e.g. text element),
    but spatial navigation only navigates to the focusable elements.
  
  
• The purpose of the feature:
    Spatial navigation aims to reach the target element with the least amount of directional input,
    but caret navigation seems to reach every element in the page. (Am I understanding right?)

I think there aren't many dependencies between caret navigation and spatial navigation,

but It would be interesting to think about how to combine those feature.

I wonder how the caret navigation manages the input element or web components. 
(Because I'm figuring out how to solve this issue in spatial navigation.)

> What is the timeframe for implementing spatial navigation in Chromium?

There isn't a concrete plan yet, but there are implementations in progress:

https://chromium-review.googlesource.com/q/%2522spatial+navigation%2522+OR+%2522SpatialNavigation%2522+OR+snav

Thanks,

Jihye

[Petter Nilsen]

On Fri, 29 Mar 2019 08:36:05 +0100, <jh....@lge.com> wrote:

Hi,

I think there are several differences between caret navigation and spatial navigation.

• Need for the modifier key:
    Caret navigation uses the modifier key (F7) to activate the feature, but spatial navigation doesn't.

It's easy to argue that spat nav should use a modifier key with the arrows keys as it does get "interesting" when navigating down/up in a document where it scrolls while trying to go to the next target.  

--

Petter Nilsen, Senior Developer, Vivaldi Technologies AS

[Daniel Bratell]

Spatial navigation is already implemented and is (as far as I used) by various Chromium based products that lack a pointer device.

/Daniel

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6be56edf-c630-4510-b867-59246609864e%40chromium.org.

[Rick Byers]

+bokan and Chao on the input team who I think have been working with the spatial navigation code lately.

In general I believe being able to effectively navigate the web without a pointing device is increasingly important, so I'm excited to see this, and for folks working on spatnav and caret browsing to be collaborating closely.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.zzmmn8t9rbppqq%40cicero2.linkoping.osa.

[chao...@chromium.org]

I just tried the caret browsing mode on Edge today. I am seeing some problem for current design on Edge:

- caret trapped when it moved into input element

- caret can never reach some element, eg, In https://www.bing.com/search?q=lol caret can never reach the info card on right side and the sign in button

- caret on same element, press up not always back to same element

I see caret browsing is good for text selection. I am thinking is it caret browsing considering as additional function of tab navigation? 

For the document provided,

1. It would be better to have the details of caret movement algorithm

2. Design need to consider about OOPIF, caret need be able to move in/out OOPIF

3. Design need to consider how caret browsing works with tab navigation and spatial navigation

[Yoshifumi Inoue]

Thanks for reporting chaopeng@!

On caret sticks inside <input> is also happened for content editable.

It seems arrow key caret movement should change behavior on caret browsing mode and non-browsing mode.

Or having special key sequence, e.g. ESC key, to move out from <input>/content editable.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/39a68d66-58be-42d2-8376-babae9e0886c%40chromium.org.

[brl...@microsoft.com]

Thanks for calling this to my attention. Do you know where in the code base this feature is implemented, and how it is enabled by the user?

Thanks.

--Bruce

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6be56edf-c630-4510-b867-59246609864e%40chromium.org.

[Jianpeng Chao]

Yes, esc and focusless spatial navigation mode is what bokan@ did for input element. 

I am thinking it is possible make caret browsing as sub mode of spatial navigation. Make spatnav able to do text selection.

[David Bokan]

As mentioned, Spatial Navigation is an already shipped configuration (it's always behind a flag though) and attempts to solve similar use cases. You can try it out using --enable-spatial-navigation.

I think supporting two separate modes that are trying to accomplish effectively the same thing would be a maintenance burden. However, I can see some advantages to the cursor mode over spatial navigation, particularly around working with text in a page and finer-grained control. OTOH, spatial navigation seems more user friendly and intuitive. Additionally, it accounts better for accessibility issues. As Chao mentioned above, perhaps we could merge the two - I could imagine spatial navigation that allows dropping down into a cursor for finer grained selections. I'd be interested to hear use cases.

FYI: +Hyojin Song has setup a breakout session on the first day of BlinkON next week to discuss spatial navigation issues. If any of you will be attending it'd be a great opportunity to sync up!

[Daniel Bratell]

Maybe the instructions from Florian's blog still works:

https://florian.rivoal.net/blog/2015/04/controlling-spatial-navigation/

If so you should be able to find that flag in the code and see what is toggled by it.

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/84977192-5f0f-4501-9507-82defff57cd0%40chromium.org.

[hyojin...@lge.com]

At a glance, seems difficult to support both caret browsing mode and spatial navigation mode at the same time. If both enables, the UA would work as the caret browsing mode. The major use cases of the caret browsing are accessibility, but spatial navigation isn't. The spatial navigation is mainly targeting for several products that lack a pointer device such as TV and feature phone without touchscreen as Daniel Bratell mentioned.

I guess the two modes have same problems. I wonder if the cursor reaches inside a <input type="text"> in the caret browsing mode, how it works. <input> elements have their own default behavior of arrow keys, so it conflicts with the caret browsing mode as well as spatial navigation. It's recently one of the major issues in terms of the design of spatial navigation. I checked the behavior with the caret browsing mode in IE, Edge, and FF, and all are a bit buggy for now.

[Bruce Long]

Updated links to moved Explainer and Design doc below (no change in their current content)

On Tuesday, March 26, 2019 at 2:27:10 PM UTC-7, Bruce Long wrote:

Contact emails

Bruc...@microsoft.com, Amit...@microsoft.com, Grisha...@microsoft.com

[Bruce Long]

Thank you for all the insightful commentary on this i2i. The Native Caret Browsing Design Document has been updated in response to feedback on this thread.  Summarizing the principle concerns:  

• Easy to forget you are in caret browsing mode and not know how to disable it 

Aaron, thanks for raising this issue.  The design has been updated so that the caret browsing mode preference is not persisted between browser sessions. Caret browsing mode is always turned off when the browser relaunches, and the user needs to explicitly toggle it on if desired. The session caret browsing state is respected by all open tabs, and is applied on creation of a new tab. 

• Details of the caret movement algorithm should be documented 

Thanks to the several reviewers asking for clarification on the caret movement model. The design doc now specifies that caret movement for native caret browsing using the arrow keys follows the same model as caret movement in editable regions or the selection focus in editable or read-only regions of the document.  After discussion with my colleagues, we feel that deriving a new independent model for moving the caret would lead to a disjoint experience when, for example, the user extends the caret into a selection by holding shift while pressing an arrow key, and thus is a non-goal for the introduction of this feature. 

This movement model constrains caret movement within the same regions that establish natural boundaries for selection, i.e. the caret can move from position X to position Y if a selection can start at X and end at Y. This precludes moving into or out of controls, across regions with different content-editabilities, or across iframes. 

• Inability to move into or out of controls or iframes 

Movement between selection boundaries established by controls and other elements that are intrinsically focusable is generally possible with tab navigation unless the author has explicitly declared these elements to not be tab stops.  Moving focus also moves the caret into the focusable element, so in this way, tab navigation complements caret navigation. 

 

There is currently no way to navigate between iframes using the keyboard alone. The goal of caret browsing, however, is to facilitate precision selection and not facilitate navigation to every possible insertion point in the document using only the keyboard. The typical scenario is for a user to click to place the caret and then further position the caret and subsequently extend it into a selection using the keyboard. 

 

Given this understanding I’m inclined to say this isn’t a problem we need to solve in v1, and if other usage patterns emerge, we can pursue a new affordance for navigating across selection boundaries.  

• Relation to spatial navigation 

The primary overlap with spatial navigation and caret browsing is just that both features consume the arrow keys to control movement of the caret and focus.  They aren’t solving the same problem so I don’t see any overlap otherwise.  Caret browsing would typically be used to place the caret, extend it into a selection, and copy content to the clipboard.  Spatial navigation moves focus between focusable elements; it is to caret browsing as tab navigation is to caret browsing.  It should be complementary, but because (for the moment) they consume the same input (arrow keys) you really need to use just one or the other and not both at the same time. 

 

The situation is similar to how caret browsing interacts with scrolling: caret browsing handles the arrow key and as a result the caret moves instead of the page scrolling (unless the caret would navigate out of the viewport in which case it is scrolled back into view). 

 

I think caret browsing should also take priority over spatial navigation.  You can toggle caret browsing on and off with F7 so it’s easy to get back into a mode that moves between focusable elements with the arrow keys instead of moving the caret. 

 

It seems like there’s plenty of room for us to define exactly what triggers each of these features.  Per Dominic’s suggestion we can talk with Chrome UX about that.  FWIW it’s also not clear that on devices where Spatial Navigation shines (like navigating with a TV remote) that you would ever want to enable caret browsing – so maybe the flags controlling the features will never be enabled by default on the same device.