Intent to Prototype: Handwriting Recognition API

[Jiewei Qian]

Contact emails

q...@chromium.org, mgi...@chromium.org

Explainer

https://github.com/WICG/handwriting-recognition/blob/main/explainer.md

Specification

None

Summary

An API for web applications to make use of advanced handwriting recognition services (e.g. those on operating systems) to recognize text from handwriting drawings (inks) in real time. In this context, handwriting drawing means the temporal and positional information used to describe a human handwriting process.

Blink component

Blink

Motivation

This is needed to make use of the latest advanced proprietary handwriting recognition algorithms, reduce the need for third-party integration. This API makes it easier to develop web applications that process handwriting inputs (e.g. note taking app).

Initial public proposal

https://github.com/WICG/proposals/issues/8

TAG review

None

TAG review status

Pending

Risks

Interoperability and Compatibility

Different browsers (and operating systems) will expose different underlying implementations and produce different outputs given the same input. We think this is acceptable given the implementation is machine-learning based, and it is not feasible or desirable to precisely specify the expected output for a given input in a standard. In the past, we have taken this approach for the Web Speech and Shape Detection APIs.

Gecko: No signal

WebKit: No signal

Web developers: No signals

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

Initially we support this on Chrome OS, since it has handwriting recognition service ready for use. We will need to do additional work to integrate with other platforms. Handwriting recognition capabilities already exist on all other platforms, with the exception that Linux doesn't have a built-in handwriting recognition library we can integrate with.

Is this feature fully tested by web-platform-tests?

No

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5263213807534080

This intent message was generated by Chrome Platform Status.

[Mike West]

Hey Jiewei!

On Thursday, November 19, 2020 at 6:33:16 PM UTC+1 Jiewei Qian wrote:

Contact emails

q...@chromium.org, mgi...@chromium.org

Explainer

https://github.com/WICG/handwriting-recognition/blob/main/explainer.md

Specification

None

Summary

An API for web applications to make use of advanced handwriting recognition services (e.g. those on operating systems) to recognize text from handwriting drawings (inks) in real time. In this context, handwriting drawing means the temporal and positional information used to describe a human handwriting process.

Blink component

Blink

Motivation

This is needed to make use of the latest advanced proprietary handwriting recognition algorithms, reduce the need for third-party integration. This API makes it easier to develop web applications that process handwriting inputs (e.g. note taking app).

Initial public proposal

https://github.com/WICG/proposals/issues/8

TAG review

None

The explainer looks robust enough to send to the TAG for review. You'll want to do so before shipping: perhaps you could file that review request now?

-mike

[Jiewei Qian]

Hi Mike,

There's still something we need to flesh out. We will request a TAG review soon after that (sometime in early December).

[Jiewei Qian]

FYI, we want to add a new optional return value attribute to describe the text segmentation. This attribute allows the application to know which strokes and points make up a character.