Intent to Ship: Playback Statistics API for WebAudio

[Chromestatus]

Contact emails

hong...@google.com, fhern...@google.com, ol...@google.com

Explainer

https://github.com/WICG/web_audio_playout

Specification

https://webaudio.github.io/web-audio-api/#AudioPlaybackStats

Summary

This feature adds an AudioContext.playbackStats attribute which returns an AudioPlaybackStats object. This object provides audio playback statistics such as average latency, minimum/maximum latency, underrun duration, and underrun count. This API allows web applications to monitor audio playout quality and detect glitches. Note: This feature was previously tracked as AudioContext.playoutStats. It has been renamed to AudioContext.playbackStats to align with the final Web Audio API specification. The old name is supported as a deprecated alias for backward compatibility.

Blink component

Blink>WebAudio

Web Feature ID

web-audio

Motivation

There is currently no way to detect whether WebAudio playout has glitches (gaps in the played audio, which typically happens due to underperformance in the audio pipeline). There is an existing way to measure the instantaneous playout latency using AudioContext.outputLatency, but no simple way to measure average/minimum/maximum latency over time. With this API, we want to propose a way to be able to measure the delay of that audio and the glitchiness of the audio.

Initial public proposal

https://github.com/WICG/proposals/issues/142

TAG review

Early TAG review request: https://github.com/w3ctag/design-reviews/issues/939

TAG review status

Issues addressed

Origin Trial Name

Playout Statistics API for WebAudio

Chromium Trial Name

AudioContextPlayoutStats

Origin Trial documentation link

https://github.com/WICG/web_audio_playout

WebFeature UseCounter name

kAudioContextPlayoutStats

Risks

Interoperability and Compatibility

No information provided

Gecko: Positive (https://github.com/WebAudio/web-audio-api/pull/2645) The spec PR was reviewed and approved by a Mozilla engineer.

WebKit: No signal

Web developers: Positive (https://github.com/WICG/proposals/issues/142#issuecomment-1981012486)

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None.

Debuggability

Can be tested by creating an AudioContext and evaluating context.playoutStats in the console.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes
https://wpt.fyi/results/webaudio/the-audio-api/the-audiocontext-interface/audiocontext-playoutstats.html?label=experimental&label=master&aligned These will be updated if the API shape changes.

Flag name on about://flags

No information provided

Finch feature name

AudioContextPlayoutStats

Rollout plan

Will ship enabled for all users

Requires code in //chrome?

False

Tracking bug

https://g-issues.chromium.org/issues/475838360

Launch bug

https://launch.corp.google.com/launch/4308993

Availability expectation

Feature is available only in Chromium browsers for the foreseeable future.

Adoption expectation

Feature is used by specific partner(s) to provide functionality within 12 months of launch in Chrome.

Adoption plan

Nothing specific planned.

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function?

None.

Estimated milestones

Shipping on desktop	146
Origin trial desktop first	131
Origin trial desktop last	136
Origin trial extension 1 end milestone	145
Origin trial extension 2 end milestone	142
Origin trial extension 3 end milestone	139
DevTrial on desktop	129
Shipping on Android	146
Shipping on WebView	146

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5172818344148992?gate=5078819495215104

Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACazmWW4MYVa_iGjN%3DK4O9B1DE3rt4_2Vkqnq6sKswHFjn6BzQ%40mail.gmail.com
Intent to Experiment: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACazmWWV6S6Ba%3Dd%3DgvjhERm1OnPyBMRJx5fbkP%3Df9zb3k%3DrNDA%40mail.gmail.com
Intent to Extend Experiment 1: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/691455eb.2b0a0220.e30ce.7e1f.GAE%40google.com
Intent to Extend Experiment 2: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/686d135c.2b0a0220.3a1521.0081.GAE%40google.com
Intent to Extend Experiment 3: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACazmWVkWb8DJM_aCRGOFpskBs-7h%3DO_yggKc3YBUkiieEO-UA%40mail.gmail.com

This intent message was generated by Chrome Platform Status.

[Alex Russell]

Hey Hongchan,

This looks extremely useful!

I'm a little surprised to see that the TAG didn't ask about alignment with other APIs in the space, e.g.:

    https://developer.mozilla.org/en-US/docs/Web/API/RTCAudioSourceStats

Will we get unified behaviour here? Will it be possible to get similar stats from an <audio> element or WebRTC contexts, e.g.?

Best,

Alex

[Hongchan Choi]

Thanks for reviewing the intent, Alex.

Regarding unified behavior with RTCAudioSourceStats, I don't believe this API serves the same goal. WebAudio’s stats are centered specifically around audio stream quality, so I am unsure about achieving unified behavior across these different APIs.

As for getting similar stats from the <audio> element or WebRTC contexts, both HTMLAudioElement and WebRTC's audio systems are "push-based." By design, they should not experience audio sample dropouts or glitches in the same way. The WebAudio (pull-based) playback stats are specifically designed to catch dropouts so developers can estimate the user's audio quality and experience.

Hopefully this answers your question!

Best,

Hongchan

[Yoav Weiss (@Shopify)]

Can you elaborate a bit more on any risks, if any?

 

Gecko: Positive (https://github.com/WebAudio/web-audio-api/pull/2645) The spec PR was reviewed and approved by a Mozilla engineer.

WebKit: No signal

Can you file for signals for both Gecko and WebKit? https://www.chromium.org/blink/launching-features/wide-review/#standards-positions

 

Shipping on desktop146 Origin trial desktop first131 Origin trial desktop last136 Origin trial extension 1 end milestone145 Origin trial extension 2 end milestone142 Origin trial extension 3 end milestone139 DevTrial on desktop129 Shipping on Android146 Shipping on WebView146

Any feedback from the OT?

[Hongchan Choi]

Thanks for your response, Yoav.

> Can you elaborate a bit more on any risks, if any?

- fhernqvist@ published this detailed self-review in 2024.

- There was also a discussion about the covert channeling, but the mitigation was designed/implemented after the privacy/security team's approval.

> Can you file for signals for both Gecko and WebKit? https://www.chromium.org/blink/launching-features/wide-review/#standards-positions

Done:
Gekco: https://github.com/mozilla/standards-positions/issues/1351
WebKit: https://github.com/WebKit/standards-positions/issues/610 

> Any feedback from the OT?

The current partner mentioned that "The metrics collected from are continuously used for the experimentation, we would not be able to verify the quality without them."

Hopefully this answers your question!

Best,

Hongchan

[Alex Russell]

I'm a little confused about why we think playback contexts like <audio> don't have similar issues when streaming. E.g., I understand that <audio> can point at a source which isn't fully loaded, which will lead to underruns and skipping during playback. Shouldn't we cover that case with a uniform API?

Best,

Alex

[Hongchan Choi]

Hello Alex,

Perhaps my explanation on different paradigms between <audio> (HTMLMediaElement) and Web Audio was not effective:

1. Pull Model: WebAudio uses a pull model. The system's audio callback demands a buffer of data at a precise interval. If the application fails to provide it in time, you get an underrun (a literal gap in the output stream). This is a real-time performance failure.
2. Push Model: The <audio> element follows a push model (streaming). When data is missing, the element enters a waiting state, pauses playback, and fires a waiting event. This is considered 'buffering,' which is an expected behavior of network-based streaming, not necessarily a failure of the audio engine itself.
3. Existing events: We already have ways to monitor <audio> stalls via the waiting, stalled, and playing events to signal the streaming/networking problems.

The another important distinction is the developer's ability to respond to it:

1. For WebAudio, an underrun is typically a CPU/scheduling issue where the audio thread cannot keep up with the graph load/complexity. For the <audio> element, 'stalled/watiing' status are mostly network-related.

2. This API is very useful for WebAudio apps because it provides the signal needed for load balancing. If a developer sees underruns increasing, they can dynamically simplify their DSP graph, reduce voices, or bypass expensive effects to restore real-time stability.

3. In contrast, for the <audio> element, the browser automatically handles the 'lack of audio frames' by pausing and entering a waiting state. There is no 'internal logic' for the developer to adjust other than waiting for the network data to arrive.

Hopefully this answer your question!

Best,

Hongchan

[Alex Russell]

Hey Hongchan,

This is helpful and explains why it should be different. LGTM1, and thanks for your patience.

Best,

Alex

[Hongchan Choi]

Thank you for the review and approval, Alex!

Best,

Hongchan