Intent to Ship: VirtualKeyboard API

130 views
Skip to first unread message

Anupam Snigdha

unread,
Jul 23, 2021, 7:42:35 PMJul 23
to blin...@chromium.org, Bo Cupp, keit...@google.com, Scott Low

Contact emails

sni...@microsoft.compc...@microsoft.com

Explainer

Note there are two explainers and two TAG reviews since the parts of the VirtualKeyboard interface dealing with hiding/showing the keyboard and controlling whether the Visual Viewport resizes when the virtual keyboard changes visibility were originally developed separately.  The specification below, however, covers both parts of the API.

https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/VirtualKeyboardPolicy/explainer.md
https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/VirtualKeyboardAPI/explainer.md

Specification

https://w3c.github.io/editing/docs/virtualkeyboard/

Design docs
https://docs.google.com/document/d/1I0LUNxK_gP5IaNQsbYN6gL6Zpm71XzduCKkublU5o5Q/edit?usp=sharing

Summary

The virtual keyboard (VK) is the on-screen keyboard used for input in scenarios where a hardware keyboard may not be available. The VirtualKeyboard interface has APIs to provide authors with more control over when the VK is shown or hidden. It also fires events that describe the intersection of the VK and layout viewport and can opt the browser out of resizing its visual viewport in response to VK visibility changes.

Blink component

Blink

Search tags

VirtualKeyboardVK APIs

TAG review

https://github.com/w3ctag/design-reviews/issues/498

https://github.com/w3ctag/design-reviews/issues/507

TAG review status

https://github.com/w3ctag/design-reviews/issues/498 - Resolution: Satisfied
https://github.com/w3ctag/design-reviews/issues/507 - Resolution: Ambivalent

Risks

Interoperability and Compatibility

This API offers sites the ability to enhance how the VK behaves in response to user interaction. The API can be feature detected using navigator.virtualKeyboard. No significant interoperability or compatibility risks have been identified.



Gecko: No signal (
https://github.com/mozilla/standards-positions/issues/531)

WebKit: No signal (
https://lists.webkit.org/pipermail/webkit-dev/2021-June/031878.html)

Web developers: Positive

Signals from webdevs: https://github.com/w3c/editing/issues/308 

In the I2P thread developers also expressed interest.

 

Ergonomics

Are there any other platform APIs this feature will frequently be used in tandem with?

No.

 

Could the default usage of this API make it hard for Chrome to maintain good performance (i.e. synchronous return, must run on a certain thread, guaranteed return timing)?

No.

 

Activation

Will it be challenging for developers to take advantage of this feature immediately, as-is?

No.

 

Would this feature benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it to make it easier to use?

No, as the web APIs need to interact with system VK APIs which is currently not possible through JS.

Security

https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/VirtualKeyboardPolicy/security-privacy.md

 

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

 

The API is supported for browsers on touch platforms that have virtual keyboards: Windows, Chrome OS and Android.  It is not supported on Mac, Linux or Android WebView.

 

Demo link

https://bocupp-microsoft.github.io/VirtualKeyboard/(https://github.com/BoCupp-Microsoft/VirtualKeyboard)

 

Debuggability

The VK APIs have basic tooling support as described in this doc.

Is this feature fully tested by web-platform-tests?

Yes (https://github.com/web-platform-tests/wpt/pull/29523)

Flag name

VirtualKeyboard

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=856269

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5680057076940800

 

Link to “Intent to Prototype” blink-dev discussion

https://groups.google.com/a/chromium.org/g/blink-dev/c/q80uCrMgiTM/m/nF3mo-7zBAAJ

 

Edge OT feedback

https://docs.google.com/document/d/1iNmFRHLXAviKopvywutbvOAirOdGJskNupnnlK0stdA/edit?usp=sharing

This intent message was generated by Chrome Platform Status.

 

Chris Harrelson

unread,
Jul 30, 2021, 12:58:57 PMJul 30
to Anupam Snigdha, blin...@chromium.org, Bo Cupp, keit...@google.com, Scott Low
Hi,

Could you respond to the points raised in this comment? They raised issues with lack of multi-implementer support, and unaddressed design points. Could you say what the open issues and points of contention are?

Thanks
Chris

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MN2PR00MB068507116D7CFD6ADF0D5FCFCFE59%40MN2PR00MB0685.namprd00.prod.outlook.com.

Anupam Snigdha

unread,
Jul 30, 2021, 4:08:09 PMJul 30
to chri...@chromium.org, blin...@chromium.org, Bo Cupp, keit...@google.com, Scott Low

Hi Chris, 

 

To my knowledge there is no disagreement over the behavior of the API.  There was only disagreement over whether the part of the API which reports the intersection between the VK and the layout viewport should be made more generic.  Specifically, the debate was whether we should build an API that describes the intersection for any UI which occludes the layout viewport versus our current proposal which is specific to the VK. 

 

Our stance has been that the VirtualKeyboard is special enough to warrant its own cohesive API. For what it’s worth I think that same debate took place within the TAG and consensus wasn’t formed, and as a result we have an ambivalent status on this issue. 

 

Additionally, the back and forth on the TAG threads with Apple ended with Apple taking an action to follow-up internally.  After that we met virtually at TPAC with representatives from Google, TAG, Apple and Firefox.  As far as I recall we addressed all objections in the TPAC meeting and agreed that even if we want to build something more generic, there is no harm in starting with this more specialized (for the VK) API.  Minutes: https://www.w3.org/2020/10/30-editing-minutes.html

 

Hope this addresses the concerns. Please let me know if you have any other questions!

 

Thanks,

Anupam

Alex Russell

unread,
Aug 5, 2021, 4:49:10 PMAug 5
to blink-dev, snianu, blin...@chromium.org, Bo Cupp, keit...@google.com, Scott Low, chri...@chromium.org
Per today's OWNERs meeting and a follow-up conversation with Bo, a few points that are clearer to me now than when I started reading the thread:

  • Apple's mention about looking internally for feedback was ~9 months ago (from TPAC 2020), which feels pretty clearly timed out to me
  • I had the same misundnerstanding about the `overlaysContent` flag  that Kenneth had in the TAG thread but, basically, it makes sense to me now as a global setter because of the other ways that virtual keyboards can be triggered (e.g., by focus into an <input>).
  • This work was moved into the new Editing WG, but is at the ED stage, 404ing TR link in the ED not withstanding, so there's no WG consensus implied here
With all of that clarified, and with the thorough OT feedback in hand, LGTM1



On Friday, July 30, 2021 at 1:08:09 PM UTC-7 snianu wrote:

Hi Chris, 

 

To my knowledge there is no disagreement over the behavior of the API.  There was only disagreement over whether the part of the API which reports the intersection between the VK and the layout viewport should be made more generic.  Specifically, the debate was whether we should build an API that describes the intersection for any UI which occludes the layout viewport versus our current proposal which is specific to the VK. 

 

Our stance has been that the VirtualKeyboard is special enough to warrant its own cohesive API. For what it’s worth I think that same debate took place within the TAG and consensus wasn’t formed, and as a result we have an ambivalent status on this issue. 

 

Additionally, the back and forth on the TAG threads with Apple ended with Apple taking an action to follow-up internally.  After that we met virtually at TPAC with representatives from Google, TAG, Apple and Firefox.  As far as I recall we addressed all objections in the TPAC meeting and agreed that even if we want to build something more generic, there is no harm in starting with this more specialized (for the VK) API.  Minutes: https://www.w3.org/2020/10/30-editing-minutes.html

 

Hope this addresses the concerns. Please let me know if you have any other questions!

 

Thanks,

Anupam

 

From: Chris Harrelson <chri...@chromium.org>
Sent: Friday, July 30, 2021 9:59 AM
To: Anupam Snigdha <sni...@microsoft.com>
Cc: blin...@chromium.org; Bo Cupp <pc...@microsoft.com>; keit...@google.com; Scott Low <sc...@microsoft.com>
Subject: [EXTERNAL] Re: [blink-dev] Intent to Ship: VirtualKeyboard API

 

Hi,

 

Could you respond to the points raised in this comment? They raised issues with lack of multi-implementer support, and unaddressed design points. Could you say what the open issues and points of contention are?

 

Thanks

Chris

 

On Fri, Jul 23, 2021 at 4:42 PM 'Anupam Snigdha' via blink-dev <blin...@chromium.org> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Yoav Weiss

unread,
Aug 5, 2021, 4:54:55 PMAug 5
to Alex Russell, blink-dev, snianu, Bo Cupp, keit...@google.com, Scott Low, chri...@chromium.org
LGTM2

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ac55687f-cd7e-446e-a8eb-b655cc3bff87n%40chromium.org.

Chris Harrelson

unread,
Aug 5, 2021, 6:59:00 PMAug 5
to Yoav Weiss, Alex Russell, blink-dev, snianu, Bo Cupp, keit...@google.com, Scott Low
Reply all
Reply to author
Forward
0 new messages