Intent to Experiment: Playout Statistics API for WebAudio

[Fredrik Hernqvist]

Contact emails

fhern...@google.com, ol...@google.com, hong...@google.com, agp...@google.com, gui...@google.com

Explainer

https://github.com/WICG/web_audio_playout

Specification

https://wicg.github.io/web_audio_playout

Summary

The AudioContext.playoutStats API allows an application to measure the quality and latency of audio playout using WebAudio.

Blink component

Blink>WebAudio

TAG review

Early TAG review request: https://github.com/w3ctag/design-reviews/issues/939

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

None

Gecko: No signal

WebKit: No signal

Web developers: Positive (https://github.com/WICG/proposals/issues/142)

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None

Goals for experimentation

This experiment will allow us to evaluate:

* How the API can be used by developers to improve Web Applications

* How the API can be used by developers to evaluate the user audio experience in their Web Applications.

Ongoing technical constraints

None.

Debuggability

Can be tested by creating an AudioContext and evaluating context.playoutStats in the console.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

No

We have not added these because the API shape might yet change. We will add these before launch.

Flag name on chrome://flags

None

Finch feature name

AudioContextPlayoutStats

Requires code in //chrome?

False

Estimated milestones

Origin trial desktop first	131
Origin trial desktop last	136
DevTrial on desktop	129

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5172818344148992?gate=4747770865123328

Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACazmWW4MYVa_iGjN%3DK4O9B1DE3rt4_2Vkqnq6sKswHFjn6BzQ%40mail.gmail.com

This intent message was generated by Chrome Platform Status.

[Fredrik Hernqvist]

We are also going to send this API to the PING for review.

[Alex Russell]

The explainer doesn't really show why the code developers have to use today is problematic (or what's impossible), and it's a bad smell for Explainers to lead with IDL. Is it possible to refactor that doc to explain *why* developers want to calculate these statistics? And is there correlation with dropped frame stats here and in other media areas, e.g. video? Should we be building a uniform set of properties across those interfaces?

Best,

Alex

[Fredrik Hernqvist]

Thanks for your feedback! We updated the explainer to address your questions about why the current APIs are insufficient and what we need this API for.

As for correlation between audio and video glitches we can address it here first and add it to the explainer if necessary:
If audio glitches occur due to high CPU load, they may be correlated with video glitches/frame drops. The closest existing thing to a uniform set of properties for audio and video are the stats for audio and video glitches in MediaStreamTrack. However, these stats track glitches originating between the source and the point at which they're measured in the MediaStreamTrack. For audio, this means that they're useful for capture (they can measure glitches from the microphone to the track) but not for playout (they cannot measure glitches from the track to the speaker). Because MediaStreams do not know where the media will end up (which might even be several different places), it's hard to define a stats API that measures glitches from audio originating from the MediaStreamTrack.
Unlike audio MediaStreamTracks, WebAudio playout is completely separate from video and does not necessarily interface with MediaStreams, so it makes sense for it to have its own audio-specific API.

Best,

Fredrik

[Alex Russell]

Thanks for updating the explainer!

LGTM to run the experiment, but let's make sure we're open to design changes and perhaps expanding the scope to other playback types.

Best,

Alex