Intent to Prototype: HTML handwriting attribute

[Chromestatus]

Contact emails

Adam.Ett...@microsoft.com

Explainer

https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/Handwriting/explainer.md

Specification

None

Summary

This feature provides a standard mechanism to indicate whether an element or its subtree should allow user agent handwriting input. User agents that support handwriting recognition as a means to input text using a pen/stylus will behave differently than a user agent that doesn't support handwriting. However, this handwriting may not be desirable for all supported input fields. By specifying the HTML handwriting attribute, authors can indicate when the user agent should not allow handwriting.

Blink component

UI>Input>Text

Motivation

To disable handwriting input without this feature, developers would need to use preventDefault() on pen touch events, assuming that was an option for the user agent implementation of handwriting input. Android uses a non-standard `touch-action` configuration to disable handwriting input, this proposal will provide a standardized mechanism.

Initial public proposal

None

TAG review

None

TAG review status

Pending

Risks

Interoperability and Compatibility

None

Gecko: No signal

WebKit: No signal

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None

Debuggability

None

Is this feature fully tested by web-platform-tests?

No

Flag name on chrome://flags

None

Finch feature name

None

Non-finch justification

None

Requires code in //chrome?

False

Estimated milestones

No milestones specified

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5186620366782464?gate=5076052179943424

This intent message was generated by Chrome Platform Status.

[Gregg Tavares]

I'm just curious. Why is it a DOMString and not a boolean? I didn't see that in the explainer

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/000000000000faa1ae061e677989%40google.com.

[Adam Ettenberger]

Chose DOMString for a few reasons, primarily some technical differences between Boolean and Enumerated attributes.

The intent is for this attribute to be opt-out, i.e., true by default and developers may specify when handwriting should not be available.

Additionally this attribute will be inherited, so developers could specify handwriting="false" on the lowest common ancestor to affect a subtree of content.

This behavior mirrors recent work regarding the "writingSuggestions" attribute.

---

Boolean attribute:
Specifying the attribute always evaluates to true. It's only possible to have a value of false by omitting the attribute.

This makes it impossible to implement an inheritance-like behavior, and would require the attribute to either be an opt-in (developers need to specify handwriting on all inputs), or to rename it so it reads as an opt-out attribute (e.g., disableHandwriting) and specify on all inputs that shouldn't have handwriting.

Keywords "true", "false", "on", "off" are not valid for boolean attributes.

Enumerated attribute:
Is much more flexible than Boolean attributes, may specify "true" / "false" keywords, and may specify behavior for "missing value default" and "invalid value default".

This makes it possible to implement both "true by default" and inheritance in the attribute specification.

[Adam Ettenberger]

cc+: fla...@chromium.org, pec...@chromium.org, mahe...@samsung.com

Was wondering if you could share some background / motivation into why `touch-action` was selected for allowing/disallowing handwriting for Android Stylus Writing.

This attribute aims for a similar effect, providing developers a mechanism to allowing/disallowing handwriting, and could replace `touch-action` as a qualifier.

Thanks!

[Robert Flack]

On Tue, Jul 30, 2024 at 7:05 PM 'Adam Ettenberger' via blink-dev <blin...@chromium.org> wrote:

cc+: fla...@chromium.org, pec...@chromium.org, mahe...@samsung.com

Was wondering if you could share some background / motivation into why `touch-action` was selected for allowing/disallowing handwriting for Android Stylus Writing.

touch-action is used by authors to define for direct manipulation devices whether direct manipulation input devices (touch and pen) should execute their direct manipulation behavior. When the spec was written this only included panning and zooming. Authors are used to the recommended practice of adding touch-action: none to elements over which they wish to handle all events themselves.

In order to allow sites for which authors following this recommended practice to continue working, we treat stylus handwriting as a "direct manipulation" action, which is similarly prevented by touch-action.

This attribute aims for a similar effect, providing developers a mechanism to allowing/disallowing handwriting, and could replace `touch-action` as a qualifier.

It's unclear to me whether implementing this would mean we don't need to honor touch-action: none. I would not expect someone authoring a drawing application to not have to set handwriting to false, unless perhaps handwriting also is preventable by their event handlers. However, today even the demo from the pointer-events spec doesn't actually call preventDefault on the events.

Thanks!

On Tuesday, July 30, 2024 at 11:42:01 AM UTC-7 Adam Ettenberger wrote:

Chose DOMString for a few reasons, primarily some technical differences between Boolean and Enumerated attributes.

The intent is for this attribute to be opt-out, i.e., true by default and developers may specify when handwriting should not be available.

Additionally this attribute will be inherited, so developers could specify handwriting="false" on the lowest common ancestor to affect a subtree of content.

It seems to me to be quite similar to the spellcheck attribute which presents its IDL value as a boolean, even though the html attribute effectively supports three values (true, false, unset / invalid).

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9521a161-75c9-4740-b936-b9e0b7655c97n%40chromium.org.

[Robert Flack]

I'm concerned this may not be the right property. For use cases where the author wants to handle the pointerevents themselves (e.g. in order to accept free-form drawing) they should be using touchaction or preventDefault on the events to tell the browser they are handling them. They shouldn't have to recognize that if there happens to be an input field underneath or nearby that they need to disable handwriting on it. The developer authoring the drawing widget may not be aware that it may be on top of or near an input element, and it seems bad if they need to find such elements and disable handwriting on them.

IMO this covers these two use cases from your explainer as well as other drawing applications:

• Document editor that wants to temporarily disable handwriting input while certain tools are selected, to support using a stylus to seamlessly draw, place, or size non-text content overtop an editable text region.
  
• Application with custom text inputs or editing experiences that override default browser behaviors by observing and handling input events and editing experiences, doesn't support input method editor (IME) or composition{start|end|update} events, or if for any reason the experience designed by website authors doesn't behave as they intend when handwriting input is available.

The declarative pointer-event solution could be better augmented by spec'ing and implementing pointer-action and/or something similar to the direct-manipulation property I proposed on that issue or one that otherwise selects which default action a pointing device should take.

For the non-custom pointer-event handling use cases, I think there are better alternate mechanisms. E.g. from those use cases you mentioned:

• Application with custom form controls that accept sensitive input may also want to avoid using a custom system IME, so they should have a hint to be treated like a password field
• Applications which have a strict format should have that format listed as a pattern value and the browser can decide if the handwriting IME is capable of honoring the pattern.
  
• I'm not sure what the special characters use case you were thinking of, perhaps you could elaborate on this one?

I'm further concerned that offering a mechanism to disable handwriting not because they're replacing it but just to disable handwriting may encourage authors to do so even if it reduces the accessibility of users visiting their site. E.g. it will prevent users from using their device's normal handwriting input capability.

[Adam Ettenberger]

> The developer authoring the drawing widget may not be aware that it may be on top of or near an input element, and it seems bad if they need to find such elements and disable handwriting on them.

The goal was that developers wouldn't have to specify handwriting for each element individually unless they had a need to.

I guess they'd likely set this on the same element that would have `touch-action`, probably the top-level "canvas", "document", or "form" element then all of the text controls within would inherit the state.

Among the existing CSS/HTML properties/attribute, I think `touch-action` is the best candidate for disabling handwriting.

Maybe instead of a `handwriting` attribute, a `handwriting` keyword could be added to `touch-action` and included in the set of `manipulation` actions?

A canvas, document editor, or form input might want granular control based on which editing tool is selected. For example:

- [🖱️] pointer-tool: auto.

- [✋] panning-tool: disables handwriting.

- [🖌️] drawing-tool: disables both panning and handwriting.

- [🔠] text-tool: disables panning.

I don't think preventDefault will be an option if they want to use the default touch/stylus scrolling behavior while preventing handwriting when handwriting is tied to pan-x and pan-y.

If it were possible to distinguish between touch and stylus inputs for `touch-action`, a developer might want to always allow panning with touch input but selectively for stylus input for those tools.

> Applications which have a strict format should have that format listed as a pattern value and the browser can decide if the handwriting IME is capable of honoring the pattern.

There could be a legal form that requires a strict but not standardizable format.

For example, a field that needs to be an exact character-to-character match with the same the capitalization, spacing, and punctuation as an existing legal document.

Handwriting may not provide the best experience for such an input.

> I'm not sure what the special characters use case you were thinking of, perhaps you could elaborate on this one?

I don't remember exactly what I had in mind for this one when I added that bit 😅.

I can't think of a good example for this now.

---

Is there a better forum for this discussion?

It sounds like introducing an HTML `handwriting` attribute may no be the right approach, but I also think the current iteration of `touch-action` may not be sufficient for allowing developers filter default touch/stylus behaviors granularly.

[Adam Ettenberger]

Hey Robert,

Just checking in, wanted to see what your thoughts are for introducing a `touch-action: handwriting` keyword instead, or if you had a strong preference for one of the other proposals you mentioned?

Also, is there another forum that's be better for this discussion?

Thanks,

~Adam

[Adam Ettenberger]

Some of the pros/cons of `touch-action: handwriting` keyword over the `handwriting` attribute:

Pros:

• Matches existing expectations around the `touch-action` CSS property, developers can choose to enable handwriting or not.
• Requires a relatively small code change, both for Chromium implementation and for web developers to control where handwriting should be allowed.
• Allows developers to individually specify whether stylus handwriting or touch scrolling should be enabled.
  • This wouldn't be possible without using preventDefault to bypass handwriting and then manually handle stylus scrolling with something like scrollTo.
  • For example, developers could specify:
    • `touch-action: manipulation` to allow stylus handwriting, and both touch scrolling and touch zoom
    • `touch-action: pinch-zoom handwriting` to allow stylus handwriting and touch zoom, but disallow scrolling
    • `touch-action: pinch-zoom pan-x pan-y` to allow scrolling and touch zoom, but disallow handwriting

Cons:

• ~40% of sites specify `touch-action` and may need treatment to enable handwriting if used in the context of editable text.
  • Chrome Platform Status (chromestatus.com)
  • Property keywords are opt-in when any are specified (default is opt-out by including all keywords). So sites specifying any keywords other than `auto`, `inherit`, or `manipulation` on editable text or ancestors of them will need to be considered by site developers.
  • i.e., Sites that may want handwriting could unintentionally have it disabled once the experience becomes available, needing to update the site to allow the experience.
  • It's unclear from the chrome status page which keywords are being used or if they're being used in editable context, so not enough information to gauge the full impact.

---

The main issue with the existing non-standard behavior where handwriting is tied to having both `pan-x pan-y` specified is developers cannot choose whether a stylus stroke should be interpreted as handwriting or scrolling when applicable. By specifying any `touch-action` that doesn't include both, the developer loses both handwriting and scrolling (one or both directions).

The existing implementation is more difficult for developers, for example full-screen and/or paginated text editors, to control whether stylus should handwrite or scroll, since it necessitates taking control of all relevant stylus inputs to manually handle scrolling instead of handwriting.

Including a keyword to allow developers to make this distinction will at least make it easier for developers to disable handwriting or scrolling independently for scenarios where the only viable alternative is to manually handle all stylus inputs.

[Adam Ettenberger]

Created a new thread in the w3c group:

[touch actions] handwriting manipulation type to distinguish panning · Issue #568 · w3c/charter-drafts (github.com)

[Adam Ettenberger]

Edit: Migrated this issue from w3c/charter-drafts to w3c/pointerevents:

[touch actions] handwriting manipulation type to distinguish panning · Issue #516 · w3c/pointerevents (github.com)