Intent to Implement: EditContext API

[Anupam Snigdha]

Intent to Implement: EditContext API

Contact emails

sni...@microsoft.com, gl...@microsoft.com, pc...@microsoft.com

Explainer

explainer

Design doc

design doc

TAG review: https://github.com/w3ctag/design-reviews/issues/416

Summary

The EditContext API simplifies the process of integrating a web app with advanced text input methods (IME), improves accessibility and performance, and unlocks new capabilities for web-based editors.

Motivation

The web platform provides out-of-the-box editing experiences for single lines of plain-text (input), small amounts of multi-line plain-text (textarea) and a starting point for building an HTML document editing experience (contenteditable elements).  

Each of the editable elements provided by the web platform comes with built-in editing behaviors that are often inadequate to power the desired editing experience. As a result, web-based editors don't incorporate the web platform's editable elements into their view. Unfortunately, the only API provided by the web platform today to enable advanced text input experiences is to place an editable element in the DOM and focus it.

This contradiction of needing an editable element, but not wanting it to be visible, leads web-based editors to create hidden editable elements to facilitate text input.  This approach negatively impacts accessibility and increases complexity, leading to buggy behavior.

An alternative is to incorporate a contenteditable element into the view of the editor, regardless of whether the editor is editing an HTML document.  This approach limits the editor's flexibility in modifying the view, since the view is also powering the text input experience.

The EditContext addresses the core issue by decoupling text input from the DOM. Rather than having the web platform infer the data required to enable sophisticated text input mechanisms from the DOM, the author will provide that data explicitly through the API surface of the EditContext.

See the explainer for more details.

Risks

Interoperability and Compatibility

Web authors opt-in to using the EditContext. It can co-exist with the existing approaches that leverage contenteditable and textarea and be used instead when feature detected. 

There are no known interop or compat risks.

Edge:In development

Chrome: In development

Firefox: No signals

Safari: No signals

Web / Framework developers: Positive support from Google docs, Word online and VS Code teams

Challenges with offscreen contenteditable element usage faced by Google docs.

Ergonomics

N/A

Activation

Most sites will consume the EditContext through editing frameworks and component libraries.  Outreach to popular editing frameworks and libraries will be needed for broad adoption.

Debuggability

Existing DevTools should be sufficient to debug EditContext APIs.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Link to entry on the feature dashboard

https://chromestatus.com/feature/5041440373604352

Requesting approval to ship?

No

[Joe Medley]

What are "advanced text input methods (IME)" and why isn't the abbreviation "TIM" or "ATIM"?

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/38a58c98-c2ef-410e-91ce-97a67d53af28%40chromium.org.

[Kenneth Rohde Christiansen]

I believe IME is a common abbreviation of Input Method Editor

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJUhtG8BSTkV_yTXhyw8hiob%2BFKc0%2BY2in1MQmcbnykQ2oq%3Dfg%40mail.gmail.com.

[Anupam Snigdha]

Kenneth is correct. IME is an abbreviation for Input Method Editor. Mostly East Asian languages rely on IMEs to compose texts. By advanced text input methods, I meant text input methods such as VK shape-writing, Handwriting panels, speech recognition, IME Compositions etc.

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/38a58c98-c2ef-410e-91ce-97a67d53af28%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

[Shimi Zhang]

+Changwan Ryu 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/45bcb868-5608-4887-8e37-60336292c167%40chromium.org.

[Joe Medley]

Instead of "advanced text input methods (IME)" you should probably say "advanced input methods such as VK shape-writing, Handwriting panels, speech recognition, IME Compositions etc." 

I'm trying to clarify the summary on Chrome Status. If that language works, I'll change it.

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/45bcb868-5608-4887-8e37-60336292c167%40chromium.org.

[Anupam Snigdha]

That sounds good to me. I'll update the explainer too. Thanks!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/45bcb868-5608-4887-8e37-60336292c167%40chromium.org.

[Joe Medley]

Footnote: I mean to write "advanced text input..." 

Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195

If an API's not documented it doesn't exist.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5a4405b5-60ee-4057-8f5d-64e3b6e04e42%40chromium.org.

[Mustaq Ahmed]

Thanks for the detailed design, Anupam.  While I am taking some time to absorb all the details, I will ask the two questions I have so far:

Event dispatching:  I see that the view forwards "focus" events to the editContext object, but apparently not "textupdate" events.  Am I right to assume that the browser would keep track of all EditContext instances to be able to dispatch events to them directly?  In that case, would an EditContext need to be attached to DOM (not done in the example) to have access to an event loop?  Or, if having both the view and the editContext in DOM is problematic (e.g. focusing ambiguity), perhaps the EditContext would be attached to a Worker instead?  Or may be have its own event loop?

EditContext.inputMode:  Are we planning to make this attribute immutable?  I am trying to understand why.  Without any details on this, the one EditContext per inputmode part of the example seems like introducing a complicated syncing problem between too many buffers.  What am I missing here?

[Anupam Snigdha]

@Joe In case its helpful to you I provided some links in the explainer to some animated GIFs demonstrating what I'm referring to wherever I mention advanced text input methods.

@Mustaq These are really good questions, thanks for asking! 

RE Focus and event dispatching: The view doesn't forward the focus in any automatic way... the author needs to call editContext.focus() whenever they want to enable "advanced text input methods".  Per the explainer that can be when some editable part of their application's view has been focused, or it could be when some custom widget is focused like in the case of the select type-to-search behavior.  So I expect that when some focused element gets its focusin event handler called that it will in turn call editContext.focus() on an appropriate editContext instance to turn on advanced text input methods.

Note that one point of confusion is likely the reuse of the word "focus".  editContext.focus() doesn't steal focus from the DOM.  We've had some conversations internally on whether we should call it something different like activate to avoid this confusion.  I don't believe there is any problem with regards to focus ambiguity if I understand your question correctly.  The activeElement is unchanged after calling editContext.focus().  Focusing (or activating) an editContext is just a way to tell the OS that the author wants advanced text input methods enabled and that the metadata required for suggestions, where text input UI should appear, what the input mode is, etc. should all come from this particular editContext instance.

RE the association with an event loop.  To my knowledge there is no requirement to be associated with a DOM node to participate in a window event loop.  The focused (or activated) EditContext (if any) associated with window of the active document will receive events from the OS text input services.  Probably some examples with multiple EditContexts in different iframes would help clear this up in the explainer.

RE EditContext.inputMode: I think we should make it mutable just like text, selection, and layout bounds properties of the EditContext. There is one caveat to this which also exists in the editable elements: if an EditContext had password inputMode and now changed to text, then we want to make sure we clear the text to avoid suggestions from appearing on VKs that might reveal some password details.

Please let me know your thoughts.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5a4405b5-60ee-4057-8f5d-64e3b6e04e42%40chromium.org.

[Dave Tapuska]

Is there a reason that you used APIs like focus/blur instead of having an "activeEditContext" variable on the document?

It seems that the Document (in its impl) would need to maintain a weak reference to the edit context. Then if the main reference to the EditContext is thrown out how does that work? How does the document know it isn't assigned. Or are you expecting focus() to assign the a strong reference to the EditContext and it would remain for how long?

Would it ever be useful to lookup what the current edit context is in a standard way? Perhaps from an embedded library. Or knowing there is an edit context alive?

dave.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6b01b0f0-3ba0-4117-84b7-cc15c9647f75%40chromium.org.

[Mustaq Ahmed]

On Fri, Sep 6, 2019 at 11:02 AM Dave Tapuska <dtap...@chromium.org> wrote:

Is there a reason that you used APIs like focus/blur instead of having an "activeEditContext" variable on the document?

It seems that the Document (in its impl) would need to maintain a weak reference to the edit context. Then if the main reference to the EditContext is thrown out how does that work? How does the document know it isn't assigned. Or are you expecting focus() to assign the a strong reference to the EditContext and it would remain for how long?

Would it ever be useful to lookup what the current edit context is in a standard way? Perhaps from an embedded library. Or knowing there is an edit context alive?

dave.

On Thu, Sep 5, 2019 at 10:26 PM 'Anupam Snigdha' via blink-dev <blin...@chromium.org> wrote:

@Joe In case its helpful to you I provided some links in the explainer to some animated GIFs demonstrating what I'm referring to wherever I mention advanced text input methods.

@Mustaq These are really good questions, thanks for asking! 

RE Focus and event dispatching: The view doesn't forward the focus in any automatic way... the author needs to call editContext.focus() whenever they want to enable "advanced text input methods".  Per the explainer that can be when some editable part of their application's view has been focused, or it could be when some custom widget is focused like in the case of the select type-to-search behavior.  So I expect that when some focused element gets its focusin event handler called that it will in turn call editContext.focus() on an appropriate editContext instance to turn on advanced text input methods.

Note that one point of confusion is likely the reuse of the word "focus".  editContext.focus() doesn't steal focus from the DOM.  We've had some conversations internally on whether we should call it something different like activate to avoid this confusion.  I don't believe there is any problem with regards to focus ambiguity if I understand your question correctly.  The activeElement is unchanged after calling editContext.focus().  Focusing (or activating) an editContext is just a way to tell the OS that the author wants advanced text input methods enabled and that the metadata required for suggestions, where text input UI should appear, what the input mode is, etc. should all come from this particular editContext instance.

RE the association with an event loop.  To my knowledge there is no requirement to be associated with a DOM node to participate in a window event loop.  The focused (or activated) EditContext (if any) associated with window of the active document will receive events from the OS text input services.  Probably some examples with multiple EditContexts in different iframes would help clear this up in the explainer.

My question was on "textupdate" event which is sent by the browser.  But I got my answer: I see that creating an EditContext using "var e = new EditContext" (instead of "let e =...") attaches "e" to "window", which makes "textupdate" dispatch from browser very similar to "beforeunload" event dispatch.  Thanks for clarifying.

 

RE EditContext.inputMode: I think we should make it mutable just like text, selection, and layout bounds properties of the EditContext. There is one caveat to this which also exists in the editable elements: if an EditContext had password inputMode and now changed to text, then we want to make sure we clear the text to avoid suggestions from appearing on VKs that might reveal some password details.

I tested both Chrome and Firefox, it turns out that there is no need to worry about the change from "password" to "text" because a similar change in "<input type=password>" also reveals the text.  In fact, Chrome actually let's the user switch <input type> this way (by clicking on the "eye" icon in password field) to make the password readable.  I believe this is designed for accessibility.  Therefore, mutable EditContext.inputmode should be fine.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHgVhZWbVNvzj5goJM9n033wRPrsARFYWA9L2o_9K894zNe7ZA%40mail.gmail.com.

[Anupam Snigdha]

@Dave 

The current thinking is that editContext.focus() will create a strong reference (internally) from document to that editContext. It remains until blur is called on that EditContext or until focus is called on a different editContext.

The typical owner of an editContext instance will be the web app or widget that needs to enable text input. I have been thinking of it as an implementation detail of the app or widget that owns it and that other widgets in the app don't need a way (and maybe shouldn't have a way) to obtain a reference to some other widget's EditContext.

If there are scenarios though where it would be useful to have a mechanism to look up the active EditContext then we can add it.

I've filed an issue to discuss more about it.

@Mustaq Thanks for confirming the "password" to "text" scenario in input elements. I've filed the below issues for the scenarios that you mentioned in your other reply:

1. Make inputMode property mutable

2. Change "focus" to "Activate" to avoid confusion with the focus in the activeElement

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6b01b0f0-3ba0-4117-84b7-cc15c9647f75%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHgVhZWbVNvzj5goJM9n033wRPrsARFYWA9L2o_9K894zNe7ZA%40mail.gmail.com.

[Anupam Snigdha]

Ping. Is there any other concern regarding this proposal? We discussed about this API in TPAC and wanted to implement a prototype of this API on Chromium (behind a flag) so the web authors could play with it and give us feedback.

Here is the prototype implementation of this feature:

https://chromium-review.googlesource.com/c/chromium/src/+/1591870

[TAMURA, Kent]

I think it's ok to proceed the experimental implementation if no one objects to the concept of API regardless of the details of the API.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2067e47a-874b-47dc-b365-923f8d6c3b70%40chromium.org.

--

TAMURA Kent
Software Engineer, Google

[smaug]

Has the proposal been updated at all? At TPAC there were still lots of open issues and there wasn't any spec-like text ready, so
it was still very hard for other browser vendors to give feedback.


-Olli

On 10/16/19 8:34 PM, 'Anupam Snigdha' via blink-dev wrote:
> Ping. Is there any other concern regarding this proposal? We discussed about this API in TPAC and wanted to implement a prototype of this API on
> Chromium (behind a flag) so the web authors could play with it and give us feedback.
> Here is the prototype implementation of this feature:
> https://chromium-review.googlesource.com/c/chromium/src/+/1591870
>
> On Tuesday, September 3, 2019 at 11:13:47 PM UTC-7, Anupam Snigdha wrote:
>
> Intent to Implement: EditContext API
>
>
> Contact emails
>

> sni...@microsoft.com <mailto:sni...@microsoft.com>, gl...@microsoft.com <mailto:gl...@microsoft.com>, pc...@microsoft.com <mailto:pc...@microsoft.com>
>
>
> Explainer
>
> explainer <https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/EditContext/explainer.md>
>
>
> Design doc
>
> design doc <https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/EditContext/dev-design.md>
>
> TAG review: https://github.com/w3ctag/design-reviews/issues/416 <https://github.com/w3ctag/design-reviews/issues/416>

>
>
> Summary
>
> The EditContext API simplifies the process of integrating a web app with advanced text input methods (IME), improves accessibility and
> performance, and unlocks new capabilities for web-based editors.
>
>
> Motivation
>
> The web platform provides out-of-the-box editing experiences for single lines of plain-text (input), small amounts of multi-line plain-text
> (textarea) and a starting point for building an HTML document editing experience (contenteditable elements).
>
>
> Each of the editable elements provided by the web platform comes with built-in editing behaviors that are often inadequate to power the desired
> editing experience. As a result, web-based editors don't incorporate the web platform's editable elements into their view. Unfortunately, the only
> API provided by the web platform today to enable advanced text input experiences is to place an editable element in the DOM and focus it.
>
>
> This contradiction of needing an editable element, but not wanting it to be visible, leads web-based editors to create hidden editable elements to
> facilitate text input.  This approach negatively impacts accessibility and increases complexity, leading to buggy behavior.
>
>
> An alternative is to incorporate a contenteditable element into the view of the editor, regardless of whether the editor is editing an HTML
> document.  This approach limits the editor's flexibility in modifying the view, since the view is also powering the text input experience.
>
>
> The EditContext addresses the core issue by decoupling text input from the DOM. Rather than having the web platform infer the data required to
> enable sophisticated text input mechanisms from the DOM, the author will provide that data explicitly through the API surface of the EditContext.
>
>

> See the explainer <https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/EditContext/explainer.md>for more details.

>
>
> Risks
>
> Interoperability and Compatibility
>
> Web authors opt-in to using the EditContext. It can co-exist with the existing approaches that leverage contenteditable and textarea and be used
> instead when feature detected.
>
>
> There are no known interop or compat risks.
>
>
>
> Edge:In development
>
> Chrome: In development
>
> Firefox: No signals
>
> Safari: No signals
>
> Web / Framework developers: Positive support from Google docs, Word online and VS Code teams
>
>
> Challenges with offscreen contenteditable element usage

> <https://docs.google.com/document/d/1KtmXy9kGwADyP5x1tfLtGt8v86XetAiZwgcOTnDyTOY/edit?ts=5cbe26c1#>faced by Google docs.

>
>
> Ergonomics
>
> N/A
>
>
> Activation
>
> Most sites will consume the EditContext through editing frameworks and component libraries.  Outreach to popular editing frameworks and libraries
> will be needed for broad adoption.
>
>
> Debuggability
>
> Existing DevTools should be sufficient to debug EditContext APIs.
>
>
> Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
>
> Yes
>
>

> Link to entry on the feature dashboard <https://www.chromestatus.com/>
>
> https://chromestatus.com/feature/5041440373604352 <https://chromestatus.com/feature/5041440373604352>

>
>
> Requesting approval to ship?
>
> No
>
>

> --
> You received this message because you are subscribed to the Google Groups "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org

> <mailto:blink-dev+...@chromium.org>.

> To view this discussion on the web visit
> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2067e47a-874b-47dc-b365-923f8d6c3b70%40chromium.org

> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/2067e47a-874b-47dc-b365-923f8d6c3b70%40chromium.org?utm_medium=email&utm_source=footer>.

[Bo Cupp]

I talked offline with Anupam and wanted to clarify that he was pinging the thread because he was under the impression that he needed an LGTM from this thread before asking others to review his CL and make an initial check-in.  Kent already replied to indicate that isn't necessary.

Olli, our next steps are to address the open issues we discussed at TPAC and to share a prototype with interested customers.  We aren't seeking another round of implementor feedback at this time - not that it isn't welcome - but we have a good number of issues to work through already.

Thank you,

Bo

> To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org
> <mailto:blin...@chromium.org>.