Intent to Ship: Handwriting Recognition API

Skip to first unread message

Jiewei Qian

Dec 8, 2021, 9:57:40 PM12/8/21
to blink-dev,

Contact emails,,


Specification [in review]

Design docs


An API for web applications to make use of advanced handwriting recognition services (e.g. those on operating systems) to recognize text from handwriting drawings (inks) in real time. In this context, handwriting drawing means the temporal and positional information used to describe a human handwriting process.

Blink component


TAG review

TAG review status

Issues addressed

Link to origin trial feedback summary


Interoperability and Compatibility

Different browsers (and operating systems) will expose different underlying implementations and produce different outputs given the same input. We think this is acceptable given the implementation is machine-learning based, and it is not feasible or desirable to precisely specify the expected output for a given input in a standard. In the past, we have taken this approach for the Web Speech and Shape Detection APIs.

Gecko: No signal ( Feedback requested, no reply yet.

WebKit: No signal ( Feedback requested, no reply yet.

Web developers: Positive ( Twitter:

Other signals:


N/A. Existing DevTools tooling works for this feature.

Is this feature fully tested by web-platform-tests?

No. Full coverage relies on platform-specific implementation. WPT only tests failure cases.

Flag name

Requires code in //chrome?


Tracking bug

Launch bug

Estimated milestones

OriginTrial desktop last


OriginTrial desktop first


Link to entry on the Chrome Platform Status

Links to previous Intent discussions

Intent to prototype:

Intent to Experiment:

This intent message was generated by Chrome Platform Status.

Glen Robertson

Dec 9, 2021, 12:07:09 AM12/9/21
to Jiewei Qian, blink-dev
Given that the Handwriting Recognition OT ends with M97, and M98 branches today, it seems like it will be difficult to ship in M98 - a merge request to enable the feature is unlikely to be approved.

Blink Owners: Would it be acceptable to extend the OT for 1 additional milestone (end in M98) and ship in M99?

Chris Harrelson

Dec 10, 2021, 12:50:58 AM12/10/21
to Glen Robertson, Jiewei Qian, blink-dev
I think it's ok. LGTM. Thanks for doing your best to ship a high-quality implementation!

Note that this will be a total of 8 milestones, including M98, but as was mentioned in a previous intent, there was also a breaking API change in M95, plus evidence of experimental feedback.

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit

Chris Harrelson

Dec 15, 2021, 11:57:53 AM12/15/21
to Glen Robertson, Jiewei Qian, blink-dev
Coming back to the intent-to-ship part: do you consider it ready for approval? Or are there more spec or other issues to iron out before we should consider approval?

Jiewei Qian

Dec 20, 2021, 11:38:42 PM12/20/21
to Chris Harrelson, Glen Robertson, blink-dev
The spec is currently being reviewed by the spec mentor. Otherwise we think the feature is ready for review.

Jiewei Qian

Dec 22, 2021, 8:02:28 PM12/22/21
to Chris Harrelson, Glen Robertson, blink-dev
Some context for security and privacy considerations (the reviewers suggest me including them here):
  • This intent is for shipping on Chrome OS.
  • We'll send separate I2S and security/privacy reviews for shipping on other platforms because their platform-specific recognizers might have different implications for privacy.
  • Chrome OS handwriting recognizers are stateless, and won't update themselves based on user behavior.
  • The list of available Chrome OS handwriting recognizers are hard-coded. So this API won't expose information about the user's language settings and preferences.

Chris Harrelson

Jan 5, 2022, 12:19:12 PMJan 5
to Jiewei Qian, Glen Robertson, blink-dev
LGTM1 from Alex came in a non-threaded message:


Per conversation in the API OWNERs, it'd be great to have a list in this thread of the bits that will be closed-source and potentially diverge by platform. Thanks!"

LGTM2 from me, but please reply with the bits Alex mentioned thanks.

Yoav Weiss

Jan 5, 2022, 12:23:47 PMJan 5
to Chris Harrelson, Jiewei Qian, Glen Robertson, blink-dev

Jiewei Qian

Jan 6, 2022, 12:32:43 AMJan 6
to Yoav Weiss, Chris Harrelson, Glen Robertson, blink-dev

We expect the following to potentially be closed source:

  • The OS-specific implementation of the handwriting recognizer and its models. In other words, how to convert a list of points to some recognized text. Example: the implementation of Microsoft Windows Ink, and Windows Ink’s English recognizer model.

We expect the following to potentially diverge by platform:

  • The handwriting recognizer implementation being used (depending on availability). Example: Uses Windows Ink on Microsoft Windows, Use MLService on Chrome OS, use Apple PencilKit on macOS.

  • Language fallback rules (depending on OS-specific models). Example: If the OS model recognizes both “zh-Hans” and “zh-Hant” at the same time, the Web recognizer supports “zh” language tag. If the model only supports “zh-Hans”, the Web recognizer doesn’t support “zh” language tag (because “zh” should also cover “zh-Hant”).

  • Availability of languages (depending on availability of OS-specific models). Example: Windows Ink doesn’t support az-Latn (Arabic in Latin alphabet), but Google MLKit supports this language.

The factors contributing to divergence:

  • Recognizer models could be large (the model for a single language can exceed 20MB), thus impractical to ship them with the browser

  • Allowing the use of proprietary recognizers (which could outperform open-source ones)

  • Allowing browsers and OSs to compete on recognizer quality

Thanks all!
Reply all
Reply to author
0 new messages