Intent to prototype and ship Web Speech: SpeechRecognition (on device)

[Paul Adenot]

Summary:
This is a _strictly_ on device implementation of the SpeechRecognition interface.
The speech synthesis counterpart defined in this specification is untouched.

This can turn audio, directly from the microphone or a same-origin
MediaStreamTrack into timed text, in real-time, with low-latency.

Bug:
Implementgation: https://bugzilla.mozilla.org/show_bug.cgi?id=webspeechondevice
Activation in Nighly: https://bugzilla.mozilla.org/show_bug.cgi?id=2069803
Shipping: https://bugzilla.mozilla.org/show_bug.cgi?id=2072585

Specification:
https://webaudio.github.io/web-speech-api/#speechreco-section

Standards Body:
Audio Community Group, to be adopted in the Audio Working Group next charter (currently being draft)

Platform Coverage:
All desktop platforms, Android tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1244242.
It's mostly a question of performance, we want to make sure we can use hardware
acceleration on mobile.

Preference:
media.webspeech.recognition.enable

DevTools Bug:
N/A

Extensions Bug:
N/A

Use Counter:
https://searchfox.org/firefox-main/search?q=&path=dom%2Fbase%2FUseCounters.conf&case=false&regexp=false

Standards-Positions Discussion:
https://github.com/mozilla/standards-positions/issues/170

Other Browsers:
    Blink: Shipped
    WebKit: No information

web-platform-tests:
Complicated. There are some manual tests, but this needs a rather large model
to run. I'm actively investigating.

[Tom Ritter]

Is Blink's implementation also on-device or is it using Google's backend (or maybe their JS AI API that goes to Gemini in the backend...?)

--
You received this message because you are subscribed to the Google Groups "dev-pl...@mozilla.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-platform...@mozilla.org.
To view this discussion visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CANWt0Wp5-8UEHVjycJiCmauvKU2ND4my%2B3dA6nuiT7wxvPsKZg%40mail.gmail.com.

[Paul Adenot]

Both, the had a cloud-based path for a long time, but they have shipped an on-device implementation for probably more than a year now.

In my own (limited) testing, the on-device implementation is generally preferred.

I'm not entirely certain what technical solution is employed to perform the actual recognition, and it is bound to evolve, with useful feature we'll add to the spec.

The standard allows for a wide variety of backends and some latitude in quality/features. Google certainly has nice in-house solutions for various performance envelopes.

Paul.