Intent to Ship: AudioContext.outputLatency

376 views
Skip to first unread message

Ajay Rahatekar

unread,
Nov 12, 2021, 11:33:59 AM11/12/21
to blink-dev, Hongchan Choi

Contact emails

hong...@chromium.org, ajayra...@google.com


Explainer

N/A - the specification (W3C Recommendation) is already published.


Specification

https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency


Summary

AudioContext.outputLatency property is the estimation in seconds of audio output latency.

Technically, this is the interval between the time the UA requests the host system to play a buffer and the time at which the first sample in the buffer is actually processed by the audio output device. For devices such as speakers or headphones that produce an acoustic signal, this latter time refers to the time when a sample’s sound is produced.



Blink component

Blink>WebAudio


TAG review

Completed: Web Audio API specification is W3C Recommendation.


TAG review status

Completed: Web Audio API specification is W3C Recommendation.


Risks

There is a risk of the feature being used for fingerprinting. However outputLatency is the buffer size of the platform-provided audio callback, so the value is inherently platform-specific. That said, the majority of the platform audio buffer size is widely known. (MacOS = 128 frames, Windows = 10ms, Android = 96 frames, etc)


This feature  does not expose more than what you can query/infer from the UA string.


Interoperability and Compatibility


Gecko: Shipped/Shipping


WebKit: No signal


Web developers: Positive (https://github.com/WebAudio/web-audio-api/issues/12)


Other signals: A WebCodecs demo application needs this feature.



Is this feature fully tested by web-platform-tests?

No - this property represents a platform-provided output latency value of a selected audio device. It does not work properly without an actual audio device.


Flag name

N/A


Requires code in //chrome?

False


Tracking bug

http://crbug.com/785409


Launch bug

http://crbug.com/1258109


Estimated milestones

98



Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5682265146261504


This intent message was generated by Chrome Platform Status.



--
Ajay Rahatekar | Technical Program Manager | ajayra...@google.com | 650-797-1279

Yoav Weiss

unread,
Nov 15, 2021, 1:48:11 AM11/15/21
to Ajay Rahatekar, Mike Taylor, blink-dev, Hongchan Choi
On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev <blin...@chromium.org> wrote:

Contact emails

hong...@chromium.org, ajayra...@google.com


Explainer

N/A - the specification (W3C Recommendation) is already published.


Regardless, could you add a few sentences explaining what use cases the feature targets and how developers are expected to use it?
 

Specification

https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency


Summary

AudioContext.outputLatency property is the estimation in seconds of audio output latency.

Technically, this is the interval between the time the UA requests the host system to play a buffer and the time at which the first sample in the buffer is actually processed by the audio output device. For devices such as speakers or headphones that produce an acoustic signal, this latter time refers to the time when a sample’s sound is produced.



Blink component

Blink>WebAudio


TAG review

Completed: Web Audio API specification is W3C Recommendation.


Link to the TAG review?
 

TAG review status

Completed: Web Audio API specification is W3C Recommendation.


Risks

There is a risk of the feature being used for fingerprinting. However outputLatency is the buffer size of the platform-provided audio callback, so the value is inherently platform-specific. That said, the majority of the platform audio buffer size is widely known. (MacOS = 128 frames, Windows = 10ms, Android = 96 frames, etc)


This feature  does not expose more than what you can query/infer from the UA string.


Would that exposure map cleanly to UA-Platform, which is considered low-entropy and exposed by default? Or would it add more than that?

/cc +Mike Taylor   

Interoperability and Compatibility


Gecko: Shipped/Shipping


Link?
 

WebKit: No signal


Have we reached out? https://bit.ly/blink-signals
 

Web developers: Positive (https://github.com/WebAudio/web-audio-api/issues/12)


Other signals: A WebCodecs demo application needs this feature.



Is this feature fully tested by web-platform-tests?

No - this property represents a platform-provided output latency value of a selected audio device. It does not work properly without an actual audio device.


I'm guessing it is tested using browser tests or unit tests without an audio device, right?
Have y'all filed bugs against WPT to enable testing it similarly? 


Flag name

N/A


Requires code in //chrome?

False


Tracking bug

http://crbug.com/785409


Launch bug

http://crbug.com/1258109


Estimated milestones

98



Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5682265146261504


This intent message was generated by Chrome Platform Status.



--
Ajay Rahatekar | Technical Program Manager | ajayra...@google.com | 650-797-1279

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHB%2BDAgeEo86rN4TrL4n8DutRczgXAogTwYLTo%2BPZco84aMq%3Dg%40mail.gmail.com.

Mike Taylor

unread,
Nov 15, 2021, 9:49:30 AM11/15/21
to Yoav Weiss, Ajay Rahatekar, blink-dev, Hongchan Choi
On 11/15/21 1:47 AM, Yoav Weiss wrote:
On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev <blin...@chromium.org> wrote:

TAG review status

Completed: Web Audio API specification is W3C Recommendation.


Risks

There is a risk of the feature being used for fingerprinting. However outputLatency is the buffer size of the platform-provided audio callback, so the value is inherently platform-specific. That said, the majority of the platform audio buffer size is widely known. (MacOS = 128 frames, Windows = 10ms, Android = 96 frames, etc)


This feature  does not expose more than what you can query/infer from the UA string.


Would that exposure map cleanly to UA-Platform, which is considered low-entropy and exposed by default? Or would it add more than that?

/cc +Mike Taylor   

Looking at https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency, it states that it depends on the platform _and_ the hardware output device. If I use an app using outputLatency with speaker A, then switch to speaker B, will the outputLatency remain the same?

Hongchan Choi

unread,
Nov 15, 2021, 10:52:57 AM11/15/21
to Yoav Weiss, Ajay Rahatekar, Mike Taylor, blink-dev, Chris Cunningham
Thanks for the review, Yoav. My responses are inline:

On Sun, Nov 14, 2021 at 10:48 PM Yoav Weiss <yoav...@chromium.org> wrote:


On Fri, Nov 12, 2021 at 5:33 PM 'Ajay Rahatekar' via blink-dev <blin...@chromium.org> wrote:

Contact emails

hong...@chromium.org, ajayra...@google.com


Explainer

N/A - the specification (W3C Recommendation) is already published.


Regardless, could you add a few sentences explaining what use cases the feature targets and how developers are expected to use it?

This feature can be useful when you need to synchronize a video stream (or any kind of visual generated by the system) and an audio stream produced by Web Audio API. For example, WebCodecs can be used to decode a MediaStream into raw video and audio data, and then an app can apply processing only on audio data via Web Audio API. This will break the original synchronization, but this can be fixed by taking the hardware audio latency into account.
 
 

Specification

https://www.w3.org/TR/webaudio/#dom-audiocontext-outputlatency


Summary

AudioContext.outputLatency property is the estimation in seconds of audio output latency.

Technically, this is the interval between the time the UA requests the host system to play a buffer and the time at which the first sample in the buffer is actually processed by the audio output device. For devices such as speakers or headphones that produce an acoustic signal, this latter time refers to the time when a sample’s sound is produced.



Blink component

Blink>WebAudio


TAG review

Completed: Web Audio API specification is W3C Recommendation.


Link to the TAG review?


This was a blanket review for the entire API surface. No issues were raised against the `outputLatency` property.
 
 

TAG review status

Completed: Web Audio API specification is W3C Recommendation.


Risks

There is a risk of the feature being used for fingerprinting. However outputLatency is the buffer size of the platform-provided audio callback, so the value is inherently platform-specific. That said, the majority of the platform audio buffer size is widely known. (MacOS = 128 frames, Windows = 10ms, Android = 96 frames, etc)


This feature  does not expose more than what you can query/infer from the UA string.


Would that exposure map cleanly to UA-Platform, which is considered low-entropy and exposed by default? Or would it add more than that?

/cc +Mike Taylor   

I'll do some more investigation and follow up.
 

Interoperability and Compatibility


Gecko: Shipped/Shipping


Link?

 
 

WebKit: No signal


Have we reached out? https://bit.ly/blink-signals

We have not reached out, and did not see any comments in the spec discussion.
 
 

Web developers: Positive (https://github.com/WebAudio/web-audio-api/issues/12)


Other signals: A WebCodecs demo application needs this feature.



Is this feature fully tested by web-platform-tests?

No - this property represents a platform-provided output latency value of a selected audio device. It does not work properly without an actual audio device.


I'm guessing it is tested using browser tests or unit tests without an audio device, right?
Have y'all filed bugs against WPT to enable testing it similarly? 

Currently we only have an IDL test:

No. We have not filed a bug for the new type of testing. I am not aware if we have that kind of testing infrastructure on WPT. Please correct me if I am mistaken.

Hongchan Choi

unread,
Nov 15, 2021, 10:56:40 AM11/15/21
to Mike Taylor, Yoav Weiss, Ajay Rahatekar, blink-dev
The specification says: If the audio output device is changed the outputLatency attribute value will be updated accordingly.

So the answer is no. The value will change accordingly.

Mike Taylor

unread,
Nov 15, 2021, 11:05:37 AM11/15/21
to Hongchan Choi, Yoav Weiss, Ajay Rahatekar, blink-dev

If that's the case, then outputLatency can reveal more entropy than just platform alone, right? It would be useful to know what this looks like in practice, or what mitigations we might be able to apply depending on the size of these latency differences.

Ajay Rahatekar

unread,
Nov 15, 2021, 11:53:44 AM11/15/21
to Mike Taylor, Hongchan Choi, Yoav Weiss, blink-dev
The small number of platform + audio output device combination i.e. a large number of users with the same platform and output device  (users on macOS using airpods) would likely be a natural obfuscator. Not sure if we have data about the distribution.

Mike Taylor

unread,
Nov 16, 2021, 11:14:49 AM11/16/21
to Ajay Rahatekar, Hongchan Choi, Yoav Weiss, blink-dev
I suppose it would be nice to do some testing to verify the idea that there's a small number of platform + audio device pairs. I was hoping Mozilla's hardware survey might have something on that, but it doesn't track speakers.

Just from some simple testing with the following script (thanks dmcardle@ for the snippet) in Firefox on macOS by 3 people, we have very different values that change dramatically when you unplug a headset:

```
(function getNextSample() {
    console.log((new AudioContext).outputLatency);
    setTimeout(getNextSample, 10);
})();
```
Here's the values on my macbook, using Firefox Nightly:
bose headset plugged in: 0.005895833
take headset out: 0.013083333
plug bose headset back in: 0.005875 (different, but stable)

https://www.w3.org/TR/webaudio/#priv-sec mentions some mitigation strategies - do we have a plan to apply any of them?

Hongchan Choi

unread,
Nov 16, 2021, 12:16:14 PM11/16/21
to Mike Taylor, Ajay Rahatekar, Yoav Weiss, blink-dev, Chrome Cunningham
Thanks for the feedback, Mike. This is really helpful!

Mitigation strategies include adding jitter (dithering) and quantization so that the exact skew is incorrectly reported

This bit refers to the randomization mechanism deployed by other browsers, but this approach defeats the purpose of this specific API and it is also argued in the spec:

Note however that most audio systems aim for low latency, to synchronise the audio generated by WebAudio to other audio or video sources or to visual cues (for example in a game, or an audio recording or music making environment). Excessive latency decreases usability and may be an accessibility issue.
 
If the reported value is incorrect, the A/V synchronization won't be possible. (Perhaps chcunningham@ can say more about this use case.)

One mitigation idea: we can do some research to collect values reported by major platforms/devices and devise a reasonable "bucket system" for them.

Chris Harrelson

unread,
Nov 18, 2021, 3:22:31 PM11/18/21
to Hongchan Choi, Yoav Weiss, Ajay Rahatekar, Mike Taylor, blink-dev, Chris Cunningham
Ok. Please file a signals request on webkit-dev.
 

Yoav Weiss

unread,
Nov 18, 2021, 3:36:58 PM11/18/21
to blink-dev, hongchan, Ajay Rahatekar, Yoav Weiss, blink-dev, Chrome Cunningham, Mike Taylor
Hey folks!

It seems like there are some things to figure out here before shipping. Could y'all file a spec issue to discuss this with the broader community?

Hongchan Choi

unread,
Nov 18, 2021, 3:45:28 PM11/18/21
to Yoav Weiss, blink-dev, Ajay Rahatekar, Chrome Cunningham, Mike Taylor
Hi Yoav,

Could you clarify what needs to be discussed from the spec side? The feature (and its behavior) is properly specified, and its privacy aspect is already reviewed by W3C:
https://github.com/WebAudio/web-audio-api/issues/1495 (general security/privacy consideration)

I thought the request from this thread is to devise a Chrome-specific mitigation, not reopening the spec discussion. Please correct me if I'm mistaken.

Cheers,
Hongchan

Chris Cunningham

unread,
Nov 18, 2021, 10:57:38 PM11/18/21
to Hongchan Choi, Yoav Weiss, blink-dev, Ajay Rahatekar, Mike Taylor
> If the reported value is incorrect, the A/V synchronization won't be possible. (Perhaps chcunningham@ can say more about this use case.)

The typical video player design drives the clock with audio. In short, you send buffers of audio to the OS, use the latency value to know when those buffers actually reach the user's ears, and choose a video frame accordingly. Apps using WebCodecs will design apps this way (example here).

As Mike notes, the latency value can vary by quite a lot depending on the device (huge differences for bluetooth vs wired), so it's critical to have the platform tell you when this changes. In terms of precision, we can _probably_ do some rounding like to the nearest millisecond. Higher values may lead to issues; user studies have shown sensitivity to fairly low values of drift. 

Aside: we may eventually desire a display latency value for similar reasons. 

Yoav Weiss

unread,
Nov 19, 2021, 1:35:12 AM11/19/21
to Chris Cunningham, Hongchan Choi, blink-dev, Ajay Rahatekar, Mike Taylor
Apologies, I misread the discussion. A spec issue is indeed not needed, as this is already covered.

From my perspective, since this exposes active entropy, the fact that it exposes more data than the (passively exposed) platform is not prohibitive on its own.
At the same time, it'd be good to work with privacy-focused folks to make sure we expose as little information as needed to maintain the API's functionality, but not more.

Hongchan Choi

unread,
Nov 19, 2021, 11:45:28 AM11/19/21
to Yoav Weiss, Chris Cunningham, blink-dev, Ajay Rahatekar, Mike Taylor
Thanks for the clarification, Yoav!

I hope the mitigation approach in the previous comment makes sense from the privacy perspective, but it will hurt the usability/value of this API significantly.
For the privacy-focused discussion, should I use other venues rather than this I2S thread?

Cheers,
Hongchan

Chris Cunningham

unread,
Nov 22, 2021, 6:00:43 PM11/22/21
to blink-dev, hongchan, chcunn...@chromium.org, blink-dev, Ajay Rahatekar, mike...@chromium.org, yoav...@chromium.org
Re: privacy discussion, Hongchan and I scheduled to meet with Paul Jensen next week.

Also, I recently determined that this new WebAudio API (outputLatency) is largely redundant with an existing API (getOutputTimestamp()). 

Chrome has already shipped the latter API, so I believe this new API doesn't add potential for additional fingerprinting. Shipping both APIs is still a good idea for the sake of inter-op.

Hongchan Choi

unread,
Nov 29, 2021, 6:54:17 PM11/29/21
to Chris Cunningham, blink-dev, chcunn...@chromium.org, Ajay Rahatekar, mike...@chromium.org, yoav...@chromium.org
I agree that outputLatency is better in terms of both usability and interoperability.

FYI, the counter shows getOutputTimestamp is about 0.001% and there are no partners who are using this API:

Yoav Weiss

unread,
Dec 1, 2021, 6:18:18 AM12/1/21
to blink-dev, hongchan, blink-dev, Chrome Cunningham, Ajay Rahatekar, Mike Taylor, Yoav Weiss, Chris Cunningham
So are y'all planning to deprecate and remove `getOutputTimestamp` once this ships? If so, that sounds reasonable.

I note Chris asked y'all to file for a webkit signal upthread. Did that happen?

Hongchan Choi

unread,
Dec 1, 2021, 11:27:22 AM12/1/21
to Yoav Weiss, blink-dev, Chrome Cunningham, Ajay Rahatekar, Mike Taylor, Chris Cunningham
So far we have not discussed the deprecation plan, but I believe that's reasonable as well.

I registered myself to the webkit-dev list to post a question, but somehow I am getting 403 forbidden. The email is sent to the list, I am not seeing the message is getting posted on the list. Not sure what to do there.

Mike Taylor

unread,
Dec 6, 2021, 9:22:26 AM12/6/21
to Hongchan Choi, Yoav Weiss, blink-dev, Chrome Cunningham, Ajay Rahatekar, Chris Cunningham
To circle back - last week we (myself + some other privacy/anti-covert tracking reviewers) met with some folks working on this feature. We ended up with an AI for Hongchan to land metrics on the values that getOutputTimestamp would produce (if called, when a new AudioContext is created, IIRC) as a way to get some data on the utility for using either of these APIs for device fingerprinting.

Yoav Weiss

unread,
Dec 15, 2021, 11:47:01 AM12/15/21
to blink-dev, Mike Taylor, blink-dev, Chrome Cunningham, ajayra...@google.com, Chris Cunningham, hongchan, Yoav Weiss
LGTM1 conditional on those metrics landing (but no need to wait for data from the metrics to come in).

Mike Taylor

unread,
Dec 15, 2021, 11:47:56 AM12/15/21
to Yoav Weiss, blink-dev, Chrome Cunningham, ajayra...@google.com, Chris Cunningham, hongchan
LGTM2 w/ same condition.

Chris Harrelson

unread,
Dec 15, 2021, 11:58:46 AM12/15/21
to Mike Taylor, Yoav Weiss, blink-dev, Chrome Cunningham, ajayra...@google.com, Chris Cunningham, hongchan
LGTM3.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Hongchan Choi

unread,
Dec 15, 2021, 12:02:09 PM12/15/21
to Chris Harrelson, Mike Taylor, Yoav Weiss, blink-dev, Chrome Cunningham, ajayra...@google.com, Chris Cunningham
Thank you all!

1. I'll land the metrics CL and follow up here.
2. The discussion in the WebKit mailing list is ongoing, so I'll report back when it's done.

Cheers,
Hongchan

Joe Medley

unread,
Dec 21, 2021, 2:54:08 PM12/21/21
to blink-dev, hongchan, mike...@chromium.org, yoav...@chromium.org, blink-dev, Chrome Cunningham, Ajay Rahatekar, Chris Cunningham, Chris Harrelson
Is this shipping in 98?

Ajay Rahatekar

unread,
Dec 21, 2021, 3:32:51 PM12/21/21
to Joe Medley, blink-dev, hongchan, mike...@chromium.org, yoav...@chromium.org, Chrome Cunningham, Chris Cunningham, Chris Harrelson
The metrics are landing in Jan. So likely shipping in 99.

-Ajay

Joe Medley

unread,
Dec 21, 2021, 4:32:15 PM12/21/21
to Ajay Rahatekar, blink-dev, hongchan, mike...@chromium.org, yoav...@chromium.org, Chrome Cunningham, Chris Cunningham, Chris Harrelson
Please update the Chrome Status entry as soon as you know. (That will inform me as well.)
Joe Medley | Technical Writer, Chrome DevRel | jme...@google.com | 816-678-7195
If an API's not documented it doesn't exist.

Ajay Rahatekar

unread,
Dec 21, 2021, 5:11:25 PM12/21/21
to Joe Medley, blink-dev, hongchan, mike...@chromium.org, yoav...@chromium.org, Chrome Cunningham, Chris Cunningham, Chris Harrelson
Will do. Ty Joe.

Hongchan Choi

unread,
Feb 16, 2022, 4:58:17 PMFeb 16
to blink-dev, Ajay Rahatekar, blink-dev, hongchan, mike...@chromium.org, yoav...@chromium.org, Chrome Cunningham, Chris Cunningham, Chris Harrelson, Joe Medley
Hello all,

We landed the metric collection CL (https://crrev.com/c/3413794) and have gathered some data:

# Notable latency range
- Android Chrome: Mean 40.96, Peak 3~5ms
- Chrome OS: Mean 46.37, Peak at 15ms
- Mac OS X: Mean 935.36, Peak at 17ms
- Linux (dev): Mean 45.32, Peak at 33ms
- Windows: Mean 31.77, Peak at 31ms

More details can be found here: go/webaudio-outputlatency-metric-collection (sorry, googler-only)

# Thoughts
- Notable peaks roughly match the platform.
- Mac OS X has a diverse distribution and there is a small peak at 200ms on Mac OSX: it may indicate that the client is using the “AirPods”. (~1%)

We believe the value exposed by this property does not pose a significant issue on the user privacy, but any feedback would be appreciated!

Best,
Hongchan

Reply all
Reply to author
Forward
0 new messages