Intent to Deprecate: WebAudio doppler effect

[Raymond Toy]

Primary eng (and PM) emails

rt...@chromium.org, hong...@chromium.org, cwi...@google.com

Summary

Remove the doppler effect

Motivation

The doppler effect in WebAudio is an interesting effect implemented in the PannerNode but was never implemented in a general way that allows for smooth changes of position and velocity.

1. The effect is only available with AudioBufferSource nodes; oscillators and media elements do not implement doppler.
2. The effect was implemented by using the velocity of the source and the destination to adjust the playback attribute of the AudioBufferSource. This has strange effects if the source is connected to more than one panner.
3. Motion of the source or listener could not be automated in a smooth way and this causes audible glitching.

Based on these issues, the W3C Audio group has decided to remove the doppler effect completely from the existing PannerNode. (See https://github.com/WebAudio/web-audio-api/issues/372 for the main issue.)

Compatibility Risk

Compatibility risk is large. Chrome, Firefox, and Safari have implemented the doppler effect from the very beginning of the WebAudio implementation. It is unknown how many websites actually use the doppler effect, but suspect it is relatively small. However, some of the WebAudio demos (http://googlechrome.github.io/web-audio-samples/samples/audio/doppler.html) use it.

Alternative implementation suggestion for web developers

There is no built-in alternative. Developers will need to implement the doppler effect tehmselves, perhaps using a DelayNode to model the delay caused by the relative motion between the source and destination.

Usage information from UseCounter

There are no metrics on the usage, but we will add counters for this.

Entry on chromestatus.com, crbug.com, or MDN

There is no entry currently for chromestatus.com, but one will be added shortly. crbug.com/439644 tracks the status of the deprecation and removal fo the doppler effect

Requesting approval to remove too?

No, but we plan on removing it in about 6 mo (M45). The deprecation messages will say removal in M45. This, of course, depends on the actual use metrics.

[Philip Rogers]

LGTM to deprecate along with adding usecounters.

[Philip Jägenstedt]

LGTM

Setting a time for removal (M45) is a bit atypical, as you say it also
must depend on actual usage. Unless the 6 month timeframe is something
already agreed to by Mozilla and/or Apple, I'd suggest not mentioning
it in the deprecation message, to potentially allow earlier removal.
If it turns out that usage is currently very low, then waiting risks
usage rising to a point where removal is more difficult. (A bit of
code being copied into a more popular Web site is all it takes.)

> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

[Raymond Toy]

On Tue, Dec 16, 2014 at 2:59 PM, Philip Jägenstedt <phi...@opera.com> wrote:

LGTM

Setting a time for removal (M45) is a bit atypical, as you say it also
must depend on actual usage. Unless the 6 month timeframe is something
already agreed to by Mozilla and/or Apple, I'd suggest not mentioning
it in the deprecation message, to potentially allow earlier removal.
If it turns out that usage is currently very low, then waiting risks
usage rising to a point where removal is more difficult. (A bit of
code being copied into a more popular Web site is all it takes.)

It's been my experience that if you don't give a deadline, everyone ignores you. The JS console will have a deprecation message (currently including the deadline). We will be gathering statistics on usage so we'll know.

[Philip Jägenstedt]

On Wed, Dec 17, 2014 at 5:08 PM, Raymond Toy <rt...@google.com> wrote:
>
>
> On Tue, Dec 16, 2014 at 2:59 PM, Philip Jägenstedt <phi...@opera.com>
> wrote:
>>
>> LGTM
>>
>> Setting a time for removal (M45) is a bit atypical, as you say it also
>> must depend on actual usage. Unless the 6 month timeframe is something
>> already agreed to by Mozilla and/or Apple, I'd suggest not mentioning
>> it in the deprecation message, to potentially allow earlier removal.
>> If it turns out that usage is currently very low, then waiting risks
>> usage rising to a point where removal is more difficult. (A bit of
>> code being copied into a more popular Web site is all it takes.)
>
>
> It's been my experience that if you don't give a deadline, everyone ignores
> you. The JS console will have a deprecation message (currently including the
> deadline). We will be gathering statistics on usage so we'll know.

Very well, I think this is the first time the deprecation message has
included the milestone of removal, but let's give it a try.

Philip

[PhistucK]

Would you, please, edit the demos and remove the usage?

☆PhistucK

[mar...@gmail.com]

I really love the doppler effect implementation and am using it heavily in my web-racing-game. as no alternative is being offered, I find it very disappointing it is being removed. http://data.cyberlympics.com/html/start.html

[Philip Jägenstedt]

Raymond, do you know of any fleshed-out code example replacing the
doppler effect?

Not to down-play the trouble caused to developers, but the use counter
has now reached stable and usage is very, very low:
https://www.chromestatus.com/metrics/feature/timeline/popularity/620

Philip

[mar...@gmail.com]

I've been thinking about the removal quite a lot. I must admit the current implementation is not extreemly intuitive, it took me some time to get it to work with multiple moving objects and multiple views. the thing that worries me most about the removal, is that I will likely have to do calculations in javascript, that so far were hardware accelerated. It's the one feature that really put a smile on my face, after switching over from soundmanage2/flash. If there is a fast javascript alternative, I guess I can live with that too.

[Raymond Toy]

On Thu, Mar 19, 2015 at 7:28 AM, Philip Jägenstedt <phi...@opera.com> wrote:

Raymond, do you know of any fleshed-out code example replacing the
doppler effect?

Unfortunately no, I'm not aware of any.  However, Hongchan and I have been thinking of implementing Doppler using a delay node. Not sure if that will work or not, but it is certainly a cool effect to have. 

I think as VR becomes more popular and available, having a Doppler effect will help make the VR experience really immersive, so we should do something there.

The existing effect in the PannerNode will go away, as will the PannerNode itself, but we are replacing it with a new spatializer(?) node that can be automated.  If we can find a good Doppler effect that can be used with AudioBufferSourceNodes, OscillatorNodes and MediaStreamAudioSourceNodes, we'll try to get it added to the spec and implement it.

Not to down-play the trouble caused to developers, but the use counter
has now reached stable and usage is very, very low:
https://www.chromestatus.com/metrics/feature/timeline/popularity/620

That seems like the wrong metric. Maybe this is better: https://www.chromestatus.com/metrics/feature/timeline/popularity/622.

Although deprecated, we don't have to remove it until the new spatializer node is spec'ed and implemented.

Philip

[Chris Wilson]

Note that the way Doppler was defined in the Web Audio spec, it was simply not possible to implement generically; it was originally defined and implemented when the only source types available were BufferSourceNodes.  The Doppler effect that we had implemented worked purely by walking back up a chain to the sources and changing their playbackRates; once other source types were implemented (e.g. streaming audio from <audio> elements or user media input) this was no longer tenable, since you can't "change the playback rate" on live input.  In fact, we did not even implement it on OscillatorNodes, so it really does ONLY work on buffer sources.

If all you have are buffer sources (all that Doppler works on today) you can just do the math to determine how fast the object is moving toward/away from you, divide by the speed of sound, add 1 and set that factor as the playbackRate (if I've done my back-of-napkin-with-no-napkin-handy) calculations correct.  Additionally, the smoothing inherent by changing the playbackRate should make this sound pretty good - definitely tweakably better than the zippering caused by the Doppler effect today.

The delay-node-based way is much more physically accurate, but I realized as I was mentally calculating it would be harder in practice to get the delayTime to smooth well if your framerate was inconsistent.

[Raymond Toy]

On Thu, Mar 19, 2015 at 1:15 PM, Chris Wilson <cwi...@google.com> wrote:

Note that the way Doppler was defined in the Web Audio spec, it was simply not possible to implement generically; it was originally defined and implemented when the only source types available were BufferSourceNodes.  The Doppler effect that we had implemented worked purely by walking back up a chain to the sources and changing their playbackRates; once other source types were implemented (e.g. streaming audio from <audio> elements or user media input) this was no longer tenable, since you can't "change the playback rate" on live input.  In fact, we did not even implement it on OscillatorNodes, so it really does ONLY work on buffer sources.

If all you have are buffer sources (all that Doppler works on today) you can just do the math to determine how fast the object is moving toward/away from you, divide by the speed of sound, add 1 and set that factor as the playbackRate (if I've done my back-of-napkin-with-no-napkin-handy) calculations correct.  Additionally, the smoothing inherent by changing the playbackRate should make this sound pretty good - definitely tweakably better than the zippering caused by the Doppler effect today.

The delay-node-based way is much more physically accurate, but I realized as I was mentally calculating it would be harder in practice to get the delayTime to smooth well if your framerate was inconsistent.

Hongchan did a few quick tests using a DelayNode to simulate the Doppler effect. It kind of works but there are some artifacts and a few cases where the results were totally strange.  I think these point to issues in Chrome's implementation of the DelayNode.  We'll try to fix these and try again later.

One issue with using a delay is that there is now a delay.  :-) This might be unexpected if you were used to the existing scheme where there was no delay. 

[mar...@gmail.com]

I just spent some time implementing the doppler effect myself. It was more difficult than it sounded in chris's post, but with a bit of research and guesstimation I did get a working solution. I'm not entirely sure I implemented it correctly, but it sounds good. Thanks for the push in the right direction, I would not have known I only needed to change the playback rate.

[Philip Jägenstedt]

This was deprecated as planned, with deprecation messages like "dopplerFactor is deprecated and will be removed in M45 when all doppler effects are removed." If for some reason it's still not possible to remove these, the deprecation message needs to be updated or removed.

[Philip Jägenstedt]

After a CL to remove dead code related to Web Audio doppler effects, it became clear that in addition to the bits which are already deprecated on AudioListener, there's also PannerNode.prototype.setVelocity. It was actually left in the spec too, but that's being fixed by Raymond.

I propose that we update the deprecation message to target M55 removal, including PannerNode's setVelocity. That isn't even measured currently, but is bounded by AudioContextCreatePanner and will thus not be too spammy.

The reason I suggest such a long (additional) deprecation period is that these were once in the spec and are now no-ops and thus no burden at all. Once they are removed any code that isn't updated may start throwing exceptions, where the developer had no reason to anticipate future trouble.

Does that sound reasonable?

[Dimitri Glazkov]

LGTM.

[Philip Jägenstedt]

Making it so: https://codereview.chromium.org/1745103002/