Intent to Implement and Ship: WebAudio SpatialPannerNode

[Raymond Toy]

Intent to Implement and Ship: WebAudio SpatialPannerNode

Contact emails

rt...@chromium.org, hong...@chromium.org

Spec

http://webaudio.github.io/web-audio-api/

http://webaudio.github.io/web-audio-api/#the-spatialpannernode-interface

Summary

The SpatialPannerNode allows for spatialized audio in WebAudio. It is the replacement for the deprecated PannerNode.

Motivation

In the deprecated PannerNode, the position and orientation of the source and listener is done using Javascript. Because of the timing between the Javascript thread and the audio thread, this cannot be done in a sample accurate way and can cause glitching of the audio if the changes are not done smoothly.

With the SpatialPannerNode, the x,y,z coordinates as well as the orientation vector can be automated on a sample-accurate basis using available AudioParam automation methods. This precision control reduces the possibility of glitching and makes for smooth transitions as the source moves.

Interoperability and Compatibility Risk

The SpatialPannerNode is a new item in the WebAudio spec, so there will be some interoperability issues until all browsers implement it. The spec discussion is at https://github.com/WebAudio/web-audio-api/issues/372

There are a few details that need to be resolved:  elevation/aziumth instead of x/y/z, default SpatialPanner values, promise for loadHRTFDatabase (which is somewhat slow), SpatialListener attribute on AudioContext is missing.

We will, of course, wait for these to be resolved before shipping the SpatialPanner.

Ongoing technical constraints

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

OWP launch tracking bug

crbug.com/590000

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5767190162702336

Requesting approval to ship?

Yes.

[Philip Jägenstedt]

LGTM1, but it would be nice to hear back on this thread before shipping with an FYI about those issues were resolved.

SpatialPannerNode and PannerNode appear to be identical except for how the position and orientation are set. Will it be possible to define PannerNode in terms of SpatialPannerNode internally, by simply mapping setPosition and setOrientation to three setValueAtTime calls each? If it is that simple, do you still think it's worth trying to deprecate and remove PannerNode later?

I ask because if it seems undesirable to have both SpatialPannerNode and PannerNode in the long term, then simply extending PannerNode to engulf SpatialPannerNode seems like a very real option. I don't have a strong sense of what would be best, though, but curious to hear what has been discussed.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Chris Harrelson]

LGTM2

[Raymond Toy]

On Tue, Mar 1, 2016 at 5:44 AM, Philip Jägenstedt <phi...@opera.com> wrote:

LGTM1, but it would be nice to hear back on this thread before shipping with an FYI about those issues were resolved.

Will do. 

SpatialPannerNode and PannerNode appear to be identical except for how the position and orientation are set. Will it be possible to define PannerNode in terms of SpatialPannerNode internally, by simply mapping setPosition and setOrientation to three setValueAtTime calls each? If it is that simple, do you still think it's worth trying to deprecate and remove PannerNode later?

One of the issues raised was replacing the Cartesiaon x,y,z orientation (and up) vectors with elevation/azimuth (polar coordinates) because it makes automation of rotating sources much easier. But as a drop-in for PannerNode, it would probably be best to keep the Cartesian coordinates.  This makes it a little bit more difficult to automate rotations smoothly.

I ask because if it seems undesirable to have both SpatialPannerNode and PannerNode in the long term, then simply extending PannerNode to engulf SpatialPannerNode seems like a very real option. I don't have a strong sense of what would be best, though, but curious to hear what has been discussed.

I think the intent is to get rid of PannerNode completely, but I suspect that will take a while before usage drops off enough to remove it completely.

I think the main issue was that you couldn't automate the PannerNode positions and had to be done from the main thread.  This can cause glitches in the audio.  SpatialPanner fixes that particular problem.

[Philip Jägenstedt]

On Wed, Mar 2, 2016 at 12:19 AM, Raymond Toy <rt...@google.com> wrote:

On Tue, Mar 1, 2016 at 5:44 AM, Philip Jägenstedt <phi...@opera.com> wrote:

LGTM1, but it would be nice to hear back on this thread before shipping with an FYI about those issues were resolved.

Will do. 

SpatialPannerNode and PannerNode appear to be identical except for how the position and orientation are set. Will it be possible to define PannerNode in terms of SpatialPannerNode internally, by simply mapping setPosition and setOrientation to three setValueAtTime calls each? If it is that simple, do you still think it's worth trying to deprecate and remove PannerNode later?

One of the issues raised was replacing the Cartesiaon x,y,z orientation (and up) vectors with elevation/azimuth (polar coordinates) because it makes automation of rotating sources much easier. But as a drop-in for PannerNode, it would probably be best to keep the Cartesian coordinates.  This makes it a little bit more difficult to automate rotations smoothly.

I ask because if it seems undesirable to have both SpatialPannerNode and PannerNode in the long term, then simply extending PannerNode to engulf SpatialPannerNode seems like a very real option. I don't have a strong sense of what would be best, though, but curious to hear what has been discussed.

I think the intent is to get rid of PannerNode completely, but I suspect that will take a while before usage drops off enough to remove it completely.

I think the main issue was that you couldn't automate the PannerNode positions and had to be done from the main thread.  This can cause glitches in the audio.  SpatialPanner fixes that particular problem.

I took a quick look in Edge, Firefox Nightly and Safari, and they all support PannerNode, although Safari's implementation is still prefixed. Given this, the non-trivial usage, and the apparent ease with which it could continue to be supported, I would be very hesitant to remove it. This is an issue for the Audio WG, so I filed the issue Merge SpatialPannerNode back into PannerNode? Can you add this to the list of issues to be resolved before shipping?

Philip

[Rick Byers]

How are the outstanding spec issues coming here?  This all seems reasonable to me, but I'd personally prefer to wait until you feel the ship-blocking outstanding design issues are resolved before an intent decision is made (although I see no reason to expect any problems here).  In general, intent-to-ship is a signal that an API design is complete.

Rick

[Raymond Toy]

It's a bit stalled right now.  But we have a teleconf this week, and I'll bring the issue up to see if we can't resolve it.  I think they're not really convtroversial; they just need some kind of agreement.

[Rick Byers]

Sounds good, thanks!  Please ping this thread when the API design has settled.

[Raymond Toy]

Will do.  And for the record, I really like Philip's proposal in https://github.com/WebAudio/web-audio-api/issues/749. If this is agreed upon, I'll either update this intent or write a new one. (Not sure what would be best right now.)

[Raymond Toy]

We discussed this on our teleconf yesterday.  There is general agreement that updating the existing PannerNode with the automations is acceptable. I have an action item to submit a pull request to the spec to show what this would look like.

In addition, we will probably deprecate the setVelocity/setOrientation methods since you can do the same thing using the new AudioParams.  If so, we'll have to decide when to remove (if ever) these methods.  As part of the CL, we'll add counters for these methods.

I'll also update the intent soon.