Intent to Ship: Add MediaStreamTrack support to the Web Speech API

296 views
Skip to first unread message

Chromestatus

unread,
Dec 17, 2024, 7:28:07 PM12/17/24
to blin...@chromium.org, ev...@google.com

Contact emails

ev...@google.com

Explainer

https://github.com/WICG/speech-api/pull/118

Specification

https://wicg.github.io/speech-api

Design docs


https://docs.google.com/document/d/1LHGtO6A-U4OFQpG9LXUf1_4Elerpv1rncNCbHnmA7zY/edit?usp=sharing

Summary

Add MediaStreamTrack support to the Web Speech API. The Web Speech API is a web standard API that allows developers to incorporate speech recognition and synthesis into their web pages. Currently, the Web Speech API uses the user's default microphone as the audio input. MediaStreamTrack support allows websites to use the Web Speech API to caption other sources of audio including remote audio tracks.



Blink component

Blink>Speech

Search tags

speech, recognition

TAG review

None

TAG review status

Not applicable

Risks



Interoperability and Compatibility

None



Gecko: Positive (https://github.com/WICG/speech-api/issues/66) Similar changes proposed by Firefox in 2019

WebKit: Positive (https://www.w3.org/Bugs/Public/show_bug.cgi?id=26336) Similar changes proposed by Safari in 2014

Web developers: Positive (https://stackoverflow.com/questions/67351357/web-speech-recognition-on-mediastream) Feature requested by web developers

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None



Debuggability

None



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

No

Initially supported on Windows, Mac, and Linux with ChromeOS support to follow.



Is this feature fully tested by web-platform-tests?

No

Flag name on about://flags

None

Finch feature name

MediaStreamTrackWebSpeech

Requires code in //chrome?

False

Estimated milestones

Shipping on desktop 133


Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

Spec changes will be made to the Web Speech API spec after the spec repo migrates over to the Audio Community Group. Tentative spec PR changes: https://github.com/WICG/speech-api/pull/118

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5178378197139456?gate=5091779878846464

This intent message was generated by Chrome Platform Status.

Daniel Clark

unread,
Dec 17, 2024, 8:27:42 PM12/17/24
to Chromestatus, blin...@chromium.org, ev...@google.com

The links supporting the browser Gecko and WebKit signals do not seem strongly indicative of a current positive position. https://github.com/WICG/speech-api/pull/118 from 2014 was resolved WONTFIX and skimming through https://github.com/WICG/speech-api/issues/66 the issue did not appear to reach any strong resolution. I think it would make sense to request formal positions to try to get a clearer picture.

 

-- Dan Clark

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6762170b.2b0a0220.38516f.0183.GAE%40google.com.

Yoav Weiss (@Shopify)

unread,
Dec 18, 2024, 10:26:28 AM12/18/24
to blink-dev, dan...@microsoft.com, ev...@google.com, Chromestatus
You also probably want to flip the relevant chromestatus review bits (for security, privacy, etc).

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Evan Liu

unread,
Dec 18, 2024, 6:48:25 PM12/18/24
to yoav...@chromium.org, blink-dev, dan...@microsoft.com, Chromestatus
Regarding interop support, this change was discussed at TPAC 2024 this year and was supported by Eric Carlson from Apple and Paul Adenot from Mozilla among others. I've submitted requests for formal permissions from Mozilla and WebKit.

I've flipped all of the chromestatus review bits as well.

Thanks!
Evan

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Chris Harrelson

unread,
Dec 30, 2024, 1:34:18 PM12/30/24
to Evan Liu, yoav...@chromium.org, blink-dev, dan...@microsoft.com, Chromestatus

PhistucK

unread,
Dec 31, 2024, 10:42:25 AM12/31/24
to Chris Harrelson, Evan Liu, yoav...@chromium.org, blink-dev, dan...@microsoft.com, Chromestatus
Very nice. Will I be able to record as well as transcribe at the same time?

This does not work at the moment (on some Android devices apparently, not all if I remember correctly, though I did not come across one that allows this) since I think both of the APIs try to get a(n exclusive?) handle of the microphone.

Basically get a stream and hand it to the MediaRecorder as well as the speech API. Will that no longer try to record simultaneously (and thus conflicting on some devices)?


PhistucK


Yoav Weiss (@Shopify)

unread,
Jan 2, 2025, 9:51:42 AMJan 2
to blink-dev, PhistucK, ev...@google.com, Yoav Weiss, blink-dev, dan...@microsoft.com, Chromestatus, Chris Harrelson
LGTM2

LGTM1

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Mike Taylor

unread,
Jan 6, 2025, 4:02:10 PMJan 6
to Yoav Weiss (@Shopify), blink-dev, PhistucK, ev...@google.com, dan...@microsoft.com, Chromestatus, Chris Harrelson

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/3d0bf2ec-d1f4-40e1-9ca6-4e8b03e2bbd4n%40chromium.org.
Reply all
Reply to author
Forward
0 new messages