Intet to Ship: RTCRtpReceiver playoutDelayHint property

[Ruslan Burakov]

Contact emails kud...@webrtc.org, hb...@chromium.org, min...@chromium.org, jak...@google.com Explainer https://github.com/henbos/webrtc-timing/pull/4 

Spec https://henbos.github.io/webrtc-timing/#dom-rtcrtpreceiver-playoutdelayhint

Summary

Native WebRTC provides methods to increase latency on the remote sources (audio or video). Increased latency opens rooms for additional optimizations and better audio, video quality. We are adding ability to control it from javascript by exposing new property "playoutDelayHint" on RTCRtpReceiver object which adds delay to the audio/video source playout in seconds. By default it is null which means no user preference (up to the browser to decide - current default behavior in browsers supporting WebRTC). Note: that in the origin trial experiment the property was called differently - "jitterBufferDelayHint".

Link to “Intent to Experiment” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/ujRD28oEej4/nVNMHnp-AgAJ

Interoperability and compatibility risk

The property "playoutDelayHint" is added to RTCRtpReceiver. If it is not supported by the browser then the new field "playoutDelayHint" will be attached to RTCRtpReceiver object. The risk to break pre-existing javascript code is low.

• Firefox: No public signals, however we have discussed it with them earlier (start of the discussion). Feature itself was deemed as potentially useful but not crucial that is why discussion was put on hold until WebRTC 1.0 spec is fully ready.

• Edge: No public signals

• Safari:No public signals

• Web Developers:No signals

Is this feature tested?

Yes; it was tested during public origin trial run: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

Tracking bug

webrtc:10287 

Link to entry on the Chrome Platform Status dashboard

https://www.chromestatus.com/feature/5685279400656896

[Philip Jägenstedt]

I think this is fairly low risk because of the style of the API, where latency varies over time already with network/device conditions, and the effect is on the user experience and not exposed to scripts. That being said, the more detail there is in the spec on what the effect at the network layer should be, the more consistent user experience will be, and the less reason anyone will have to use different hints for different browsers.

That is to say this looks good, but two questions:

Have you solicited feedback from Safari? +Youenn Fablet is probably the right person.

Do you intend to move https://henbos.github.io/webrtc-timing into the WICG? A mundane but real benefit of having it in the WICG (or a working group) is that it's easier for spec scraping tooling (like Reffy) to discover the spec's existence.

P.S. I've manually added this to https://bit.ly/blinkintents, it was missed because of a typo in the subject.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/32984c04-9685-407b-859f-3c0f195cdec2%40chromium.org.

[Henrik Boström]

On Friday, October 4, 2019 at 1:57:07 PM UTC+2, Philip Jägenstedt wrote:

I think this is fairly low risk because of the style of the API, where latency varies over time already with network/device conditions, and the effect is on the user experience and not exposed to scripts. That being said, the more detail there is in the spec on what the effect at the network layer should be, the more consistent user experience will be, and the less reason anyone will have to use different hints for different browsers.

That is to say this looks good, but two questions:

Have you solicited feedback from Safari? +Youenn Fablet is probably the right person.

This issue was brought up for the working group (start of discussions here). I believe Youenn was present, but I don't remember? There was no interest by the working group at the time so we moved it to an extension spec.

My current feeling is that extensions will get little interest until after 1.0 is complete.

Do you intend to move https://henbos.github.io/webrtc-timing into the WICG? A mundane but real benefit of having it in the WICG (or a working group) is that it's easier for spec scraping tooling (like Reffy) to discover the spec's existence.

The plan is to move various extensions to a single "webrtc-extension" spec in order to make features more visible (for example I also intend to put adaptivePtime there and will encourage other browser implementors to use the same document). After WebRTC 1.0 has reached PR status we'll likely revisit extension APIs and see if any of them make sense to merge into the main spec.

I have not thought out where this spec will live in the long term. I should talk with you offline to see about what to do about moving this spec to somewhere more visible than my github account.

P.S. I've manually added this to https://bit.ly/blinkintents, it was missed because of a typo in the subject.

On Fri, Oct 4, 2019 at 11:37 AM 'Ruslan Burakov' via blink-dev <blin...@chromium.org> wrote:

Contact emails kud...@webrtc.org, hb...@chromium.org, min...@chromium.org, jak...@google.com Explainer https://github.com/henbos/webrtc-timing/pull/4 

Spec https://henbos.github.io/webrtc-timing/#dom-rtcrtpreceiver-playoutdelayhint

Summary

Native WebRTC provides methods to increase latency on the remote sources (audio or video). Increased latency opens rooms for additional optimizations and better audio, video quality. We are adding ability to control it from javascript by exposing new property "playoutDelayHint" on RTCRtpReceiver object which adds delay to the audio/video source playout in seconds. By default it is null which means no user preference (up to the browser to decide - current default behavior in browsers supporting WebRTC). Note: that in the origin trial experiment the property was called differently - "jitterBufferDelayHint".

Link to “Intent to Experiment” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/ujRD28oEej4/nVNMHnp-AgAJ

Interoperability and compatibility risk

The property "playoutDelayHint" is added to RTCRtpReceiver. If it is not supported by the browser then the new field "playoutDelayHint" will be attached to RTCRtpReceiver object. The risk to break pre-existing javascript code is low.

• Firefox: No public signals, however we have discussed it with them earlier (start of the discussion). Feature itself was deemed as potentially useful but not crucial that is why discussion was put on hold until WebRTC 1.0 spec is fully ready.

• Edge: No public signals

• Safari:No public signals

• Web Developers:No signals

Is this feature tested?

Yes; it was tested during public origin trial run: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

Tracking bug

webrtc:10287 

Link to entry on the Chrome Platform Status dashboard

https://www.chromestatus.com/feature/5685279400656896

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Youenn Fablet]

I do not think this is high in the priority list of WebRTC features from a WebKit point of view.

The use case seems fine. The fact that the API is at RTCRtpReceiver seems fine as well.

There is an inherent weakness in defining it as a hint, it might trigger different behaviours according browsers and/or devices.

[Henrik Boström]

On Friday, October 4, 2019 at 6:04:02 PM UTC+2, Youenn Fablet wrote:

I do not think this is high in the priority list of WebRTC features from a WebKit point of view.

The use case seems fine. The fact that the API is at RTCRtpReceiver seems fine as well.

There is an inherent weakness in defining it as a hint, it might trigger different behaviours according browsers and/or devices.

You're right. This is a compromise for the fact that the API is attempting to control mechanisms (such as jitter buffer) that are user agent implementation-specific land.

It's hard to have a large degree of specificity on such a high-level API.

I think when we have WebRTC "Next Version" APIs, like have the application control encoding and transports, we'll no longer need APIs for that are defined as "hints". But these APIs are far in the future.

[Philip Jägenstedt]

Alright, thanks all. I think the only remaining question is where the
spec will end up, but let's leave it to hbos@ to work that out with
the WebRTC WG.

LGTM1

>>>> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

>>>> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/32984c04-9685-407b-859f-3c0f195cdec2%40chromium.org.
>>
>>

> --
> You received this message because you are subscribed to the Google Groups "blink-dev" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/0999850f-cbf4-487b-a858-a54ad6e08274%40chromium.org.

[Yoav Weiss]

On Fri, Oct 4, 2019 at 12:37 PM 'Ruslan Burakov' via blink-dev <blin...@chromium.org> wrote:

Contact emails kud...@webrtc.org, hb...@chromium.org, min...@chromium.org, jak...@google.com Explainer https://github.com/henbos/webrtc-timing/pull/4 

That's not an explainer.

 

Spec https://henbos.github.io/webrtc-timing/#dom-rtcrtpreceiver-playoutdelayhint

The spec seems to be having some linking issues. Also, as Philip mentioned, in order to enable industry members to collaborate on this incubation, the spec can't be on a personal repo for the long term.

WICG would be a natural temporary venue, unless there's a better CG for WebRTC related incubations.

I'm missing a TAG review for this or a reason why one is not needed. That seems particularly important given that the WebRTC WG did not review that work.

 

Summary

Native WebRTC provides methods to increase latency on the remote sources (audio or video). Increased latency opens rooms for additional optimizations and better audio, video quality. We are adding ability to control it from javascript by exposing new property "playoutDelayHint" on RTCRtpReceiver object which adds delay to the audio/video source playout in seconds. By default it is null which means no user preference (up to the browser to decide - current default behavior in browsers supporting WebRTC). Note: that in the origin trial experiment the property was called differently - "jitterBufferDelayHint".

The use case for this feature is not entirely clear to me. What are scenarios where developers would pass along a higher delay to avoid buffering later on? Streaming content scenarios (compared to conversational ones)? Something else?

Link to “Intent to Experiment” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/ujRD28oEej4/nVNMHnp-AgAJ

Interoperability and compatibility risk

The property "playoutDelayHint" is added to RTCRtpReceiver. If it is not supported by the browser then the new field "playoutDelayHint" will be attached to RTCRtpReceiver object. The risk to break pre-existing javascript code is low.

• Firefox: No public signals, however we have discussed it with them earlier (start of the discussion). Feature itself was deemed as potentially useful but not crucial that is why discussion was put on hold until WebRTC 1.0 spec is fully ready.

• Edge: No public signals

• Safari:No public signals

• Web Developers:No signals

Why are pushing for this then? What makes this important? Do we have partners that would like to adopt this? (If so, I'd state their support as a web developer signal)

 

Is this feature tested?

Yes; it was tested during public origin trial run: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

What were the results of the origin trial? 

Tracking bug

webrtc:10287 

Link to entry on the Chrome Platform Status dashboard

https://www.chromestatus.com/feature/5685279400656896

--

[Henrik Boström]

This is a minor addition to webrtc-pc (https://w3c.github.io/webrtc-pc/), which has gone through TAG review.

I don't believe we should request a new TAG review every time a minor addition like an attribute is added to an interface in WebRTC?

As for explainer, while a document wasn't provided, I think the contents of this intent thread covers explaining it.

The reason why this was added in a separate repository than webrtc-pc is because that spec is close to Proposed Recommendation and new features are not added there.

How to handle minor additions is discussed here: https://github.com/w3c/webrtc-pc/issues/2323

I intend to follow up with foolip@ on this issue.

> Why are pushing for this then? What makes this important? Do we have partners that would like to adopt this? (If so, I'd state their support as a web developer signal)

We have strong interest from WebRTC products. This has been discussed internally. We want to push it to improve quality for our users.

We should make this "Web developer signal: Positive".

[Ruslan Burakov]

Hi,
The feedback from our first experiment can be found here:
https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/ujRD28oEej4/nVNMHnp-AgAJ
```
We see improvements in our technical audio metrics if we add small additional delay to both audio/video streams, without negative feedback from users about additional delay.
```

And we are receiving similar positive results for our second experiment.

To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

[Ruslan Burakov]

More about experiments results + overall feature can be found here (Sorry, google only):

go/hotlaneaudiojitterbuffermindelay

Also TAG review seems as overkill for this feature.

[Yoav Weiss]

(re-reading my previous comment, apologies if my tone was over-critical. It wasn't intended as such)

On Mon, Oct 14, 2019 at 6:54 PM Henrik Boström <hb...@chromium.org> wrote:

This is a minor addition to webrtc-pc (https://w3c.github.io/webrtc-pc/), which has gone through TAG review.

I don't believe we should request a new TAG review every time a minor addition like an attribute is added to an interface in WebRTC?

Could be, but if you believe this attribute doesn't require such a review, you should state that explicitly.

At the same time, I'd be concerned if this specific attribute wasn't reviewed by anyone outside the team.

Did the WG discuss the design at all, or just decided that it's not something that should go into the current version?

One point regarding the design: Is there a way to detect support for the feature? Should/can developers work-around the lack of the feature when not supported?

As for explainer, while a document wasn't provided, I think the contents of this intent thread covers explaining it.

I'm sorry but they don't really cover the use cases and how developers are expected to use this feature.

An external document may not be required here, but covering the above inline would have helped me review this, and would have helped folks watching this thread in understanding what the feature is supposed to accomplish.

 

The reason why this was added in a separate repository than webrtc-pc is because that spec is close to Proposed Recommendation and new features are not added there.

How to handle minor additions is discussed here: https://github.com/w3c/webrtc-pc/issues/2323

FWIW, in other WGs this is handled by creating different branches for the different spec versions, and only including new features in the L+1 or "experimental" branches. I don't see why a separate repo is needed.

But if it is, it should be somewhere that enables wide collaboration.

  

I intend to follow up with foolip@ on this issue.

> Why are pushing for this then? What makes this important? Do we have partners that would like to adopt this? (If so, I'd state their support as a web developer signal)

We have strong interest from WebRTC products. This has been discussed internally. We want to push it to improve quality for our users.

We should make this "Web developer signal: Positive".

OK. Not necessarily blocking, but would be good to try and collect similar opinions from more WebRTC products.

 

On Friday, October 4, 2019 at 11:37:34 AM UTC+2, Ruslan Burakov wrote:

Contact emails kud...@webrtc.org, hb...@chromium.org, min...@chromium.org, jak...@google.com Explainer https://github.com/henbos/webrtc-timing/pull/4 

Spec https://henbos.github.io/webrtc-timing/#dom-rtcrtpreceiver-playoutdelayhint

Summary

Native WebRTC provides methods to increase latency on the remote sources (audio or video). Increased latency opens rooms for additional optimizations and better audio, video quality. We are adding ability to control it from javascript by exposing new property "playoutDelayHint" on RTCRtpReceiver object which adds delay to the audio/video source playout in seconds. By default it is null which means no user preference (up to the browser to decide - current default behavior in browsers supporting WebRTC). Note: that in the origin trial experiment the property was called differently - "jitterBufferDelayHint".

Link to “Intent to Experiment” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/ujRD28oEej4/nVNMHnp-AgAJ

Interoperability and compatibility risk

The property "playoutDelayHint" is added to RTCRtpReceiver. If it is not supported by the browser then the new field "playoutDelayHint" will be attached to RTCRtpReceiver object. The risk to break pre-existing javascript code is low.

• Firefox: No public signals, however we have discussed it with them earlier (start of the discussion). Feature itself was deemed as potentially useful but not crucial that is why discussion was put on hold until WebRTC 1.0 spec is fully ready.

• Edge: No public signals

• Safari:No public signals

• Web Developers:No signals

Is this feature tested?

Yes; it was tested during public origin trial run: https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/Tgm4qiNepJc/NUEciMYzBwAJ

Tracking bug

webrtc:10287 

Link to entry on the Chrome Platform Status dashboard

https://www.chromestatus.com/feature/5685279400656896

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/38361776-bf47-4d7c-8fe0-c7ef142d7a07%40chromium.org.

[Philipp Hancke]

Am Mo., 14. Okt. 2019 um 17:54 Uhr schrieb Henrik Boström <hb...@chromium.org>:

This is a minor addition to webrtc-pc (https://w3c.github.io/webrtc-pc/), which has gone through TAG review.

I don't believe we should request a new TAG review every time a minor addition like an attribute is added to an interface in WebRTC?

As for explainer, while a document wasn't provided, I think the contents of this intent thread covers explaining it.

The reason why this was added in a separate repository than webrtc-pc is because that spec is close to Proposed Recommendation and new features are not added there.

How to handle minor additions is discussed here: https://github.com/w3c/webrtc-pc/issues/2323

I intend to follow up with foolip@ on this issue.

> Why are pushing for this then? What makes this important? Do we have partners that would like to adopt this? (If so, I'd state their support as a web developer signal)

We have strong interest from WebRTC products. This has been discussed internally. We want to push it to improve quality for our users.

when you say "webrtc products" how many of them are google ones?

[Ruslan Burakov]

Sorry again, full results breakdowns are Google only.

There has been some confusion. We have successfully utilized this API in the two different experiments which correspond to two distinct use cases.

First is a min delay experiment. There we were adding small and unnoticeable amount of delay for both audio and video in order to improve audio quality. By default this value in WebRTC is 0 or "play audio/video as fast as possible". It did show improvements in audio:
go/hotlaneaudiojitterbuffermindelay

This API is needed as not every application on the web would benefit from additional delay. We can't bake it as a new default because it would potentially hurt user experience in highly interactive applications, for e.g. Google Stadia.

The other experiment/use case is when we increase delay by high value for muted users. They don't notice additional delay as they don't have means to interact quickly with the call because they are muted. However, as we buffer more audio/video in case of network issues, the audio experience improves. But this experiment is more complex and it is still ongoing. Initial results can be seen here:
go/passive-listener-initial-breakdown

[Henrik Boström]

>  Could be, but if you believe this attribute doesn't require such a review, you should state that explicitly.

OK will keep that in mind in the future.

> At the same time, I'd be concerned if this specific attribute wasn't reviewed by anyone outside the team.

> Did the WG discuss the design at all, or just decided that it's not something that should go into the current version?

I brought this up to the WebRTC Working Group (editor's meeting) more than half a year ago and the WG did not want to spend time on this new feature as it was taking away time from "finishing 1.0".

Hints need to be described delicately, when the WG last saw the proposal they said we need to spend more time clarifying it and that this should happen outside of the WG, which is what we did. And here we are.

Developer's needs move faster than the working group sometimes. We need to respond to product needs.

> FWIW, in other WGs this is handled by creating different branches for the different spec versions, and only including new features in the L+1 or "experimental" branches. I don't see why a separate repo is needed.

> But if it is, it should be somewhere that enables wide collaboration.

The attempt to bring extensions together (https://github.com/w3c/webrtc-pc/issues/2323) is an attempt to give editors and implementers another chance to revisit this when there is time, and to gain more visibility. I agree that we should bring this to WICG for visibility if this does not get adopted by the working group.

> I'm sorry but they don't really cover the use cases and how developers are expected to use this feature.

> An external document may not be required here, but covering the above inline would have helped me review this, and would have helped folks watching this thread in understanding what the feature is supposed to accomplish.

By default, WebRTC attempts to do a good tradeoff between quality and interactiveness that works for "most applications". How the browser does this is implementer territory, the spec does not say what size of the jitter buffer to use for example. This API gives the ability to specify "more" or "less" interactive applications by hinting what delays are acceptable for the application's use case. Like Ruslan describes above, you might have a highly interactive user experience where low latency is everything, or you might have something passive - like listening to a presentation - where it doesn't matter if you are 10 seconds behind the stream.

> One point regarding the design: Is there a way to detect support for the feature?

This can easily be feature detected: "Does the prototype contain this attribute?"

> Should/can developers work-around the lack of the feature when not supported?

The developer neither should or can. This is a hint to the browser. If you can't provide a hint then quality may not be optimal in some use cases, but the browser will still try to do its best job to optimize for user experience.

[win...@gmail.com]

My main concern with this proposal remains: It's essentially an API for JS to make browsers stutter.

How web compatible is it to outsource control of our jitter buffer length? As a non-dominant vendor, we sometimes see web sites not testing their site with every browser or platform, so giving this tuning control to someone who may never test its effect on our browser, isn't terribly appealing. When things stutter people blame the browser.

I go on to say: I'm not immune to its benefits, just making a point that it might take a while to work out the details, which might be better done in an extension spec?

My second concern is we don't appear to have worked out those details. I've objected to "hints" before. They lack normative steps one can write tests from. In its current form this is hardly saying more than: we're implementing this surface somehow. To be fair, I'm sympathetic to the difficulty of writing such tests (degradationPreference had the same problem, and a similar fate), but that doesn't outweigh my concerns.

Our response for how to best interpret this (let's be honest: Chrome-tuned) value from JavaScript, may sadly be to ignore it, or risk stutter.

Question for the Chrome team: how are you planning to remain compatible with yourself after a couple of years from e.g. games tuning your jitter buffer to the min for max realtime response? I worry we're painting ourselves in a corner.

If these are just hints, have you considered more declarative coarse-grained controls like e.g.: latency = "participant" | "audience" | "game" | "default" ? That might be a more web compatible approach.

[Henrik Boström]

Thanks for joining in the discussion, Jan-Ivar.

Like I said earlier in the thread, hints are a a compromise due to the fact that we need to have a way of influencing something which is inherently controlled by the UA.

> I think when we have WebRTC "Next Version" APIs, like have the application control encoding and transports, we'll no longer need APIs for that are defined as "hints". But these APIs are far in the future.

Let's not make "perfect" be the enemy of "progress", eh.

On Tuesday, October 15, 2019 at 5:10:41 PM UTC+2, win...@gmail.com wrote:

My main concern with this proposal remains: It's essentially an API for JS to make browsers stutter.

How web compatible is it to outsource control of our jitter buffer length? As a non-dominant vendor, we sometimes see web sites not testing their site with every browser or platform, so giving this tuning control to someone who may never test its effect on our browser, isn't terribly appealing. When things stutter people blame the browser.

I go on to say: I'm not immune to its benefits, just making a point that it might take a while to work out the details, which might be better done in an extension spec?

My second concern is we don't appear to have worked out those details. I've objected to "hints" before. They lack normative steps one can write tests from. In its current form this is hardly saying more than: we're implementing this surface somehow. To be fair, I'm sympathetic to the difficulty of writing such tests (degradationPreference had the same problem, and a similar fate), but that doesn't outweigh my concerns.

First of all, while attempting to honor the hint could yield more stutter, this is not a "jitter buffer length setter"; the browser has the final say in the jitter buffer. If the browser cannot provide a relatively stutter-free user experience with the target hint, it should use a higher target.

A value of 0 does not mean "delete the jitter buffer", it means "give me the bare minimum delay that the UA can provide".

Today, the application has no way to influence how the UA behaves, all of this is implementation-specific. That means that one browser could optimize its jitter buffer for gaming and another browser could optimize its jitter buffer for presentations. The browser has no way to know what the application use case is, so it just does a wild guess. How web compatible is that? Well, luckily all the browsers have had the use case of "participant" in mind when optimizing their implementations, so web compat ends up working despite the lack of spec guidance here. But...

- If the application's use case requires low latency, the UA is not optimized for their use case.

- If the application's use case is passive listening, the UA is not optimized for their use case.

If the application's use case is different than the browser's default use case then surely it will blame the browser for not working optimally for their use case. So let's allow the application to specify its use case, or else we're all under fire :)

Our response for how to best interpret this (let's be honest: Chrome-tuned) value from JavaScript, may sadly be to ignore it, or risk stutter.

If Firefox's jitter buffer length is as small as it can be without getting stutter already, then setting this attribute to 0 should have no effect. But if Firefox has some amount of wiggle-room that was added due to optimizing for "participant" use case, then the right thing may be to lower that additional wiggle-room delay.

For other use cases, increasing the value by adding additional delays would not risk suttering, in fact it would decrease the risk of stutter. Would Firefox ignore the value 5 or 10 as well?

 

Question for the Chrome team: how are you planning to remain compatible with yourself after a couple of years from e.g. games tuning your jitter buffer to the min for max realtime response? I worry we're painting ourselves in a corner.

The way I see this API is it is another input the the jitter buffer algorithm. The algorithm will be tuned and improve over the years whether or not the UA has any idea what the application's use case is. But I'm worried that we can't optimize for different use cases if the application can't say what the use case is that we need to optimize for.

If these are just hints, have you considered more declarative coarse-grained controls like e.g.: latency = "participant" | "audience" | "game" | "default" ? That might be a more web compatible approach.

We discussed an enum approach, I think we even went back and forth a bit, but after discussing that option we found that it was even less specific. How much latency is "game"? Well that depends on the game doesn't it? A game can be very interactive like a first-person shooter or it can be a turn-based card game. If we didn't specify what range of seconds each enum value means then each browser could end up with different targets, so if we're going to define things in terms of seconds anyway, it's a lot more appealing to let the application tell us the desired seconds than for us to guess between a range.

With seconds, what the application asked for has higher specificity than an enum, even if how the browser attempts to honor that request is implementation-specific.

With an enum, both what the application asked for and how to honor it becomes implementation-specific. I think seconds is the more web compatible approach. It is also closer to what we want.

[Yoav Weiss]

When making a decision here, I think there are multiple points to take into consideration.

On the plus side, the use case this tackles seems valuable and shipping a solution for it will improve video call user experience, and we have the origin trial results to prove that.

At the same time, the WG decision to not discuss this means that the solution for the use-case wasn't widely reviewed. On this thread we've heard multiple opinions regarding the design and speculations regarding its compat implications. It's not obvious to me which speculation is correct, but it seems like something that would benefit from a wider discussion, which makes me regret the lack of a TAG review for this even more.

On top of that, lack of external partners makes this harder than it should be, as I don't think that we could move the current spec from it's current home in Henrik's personal repo to a venue where more folks could contribute to it, like the WICG. (although maybe Youenn's slightly-positive opinion can help there)

I almost wish you would have reached out to external partners at the Origin Trial phase, as having broad industry support would have made things easier.

With all that said, the user benefits seem to outweigh the compat risk this poses.

LGTM2 under the following conditions:

• You would submit the wider question of hints and how they should be implemented (i.e. enums vs. numbers) for the TAG to review (maybe on their design-principles repo). I don't think we need to block on an answer here, but it would be good to have a principled approach to this repeating question.
• You would kick off a WICG discourse thread explaining the use-cases this tackles, pointing to the current spec and ask for its adoption. I'll let the other chairs there determine if this can be adopted by the CG.
• We would be willing to break away from the current design if once other vendors find the time to review this, the WG would reach a conclusion that this needs to be changed. If and when that happens, we'd need to figure out a path for deprecation & removal/modification that would be web compatible.

[alex...@gmail.com]

Hi, I am developer from Yandex. We would be very interested in this API but I unfortunately I can't share our usecases due to NDA

[Philipp Hancke]

out of curiosity, does this feature depend on the unstandardized playout-delay header extension that is somewhat documented in https://github.com/webrtc/webrtc-org/pull/216?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEhePGdQMA88hbYjAcEGH1hfEX8HCHKtNnvaM5DZF7Fixg%40mail.gmail.com.

[Henrik Boström]

No this does not depend on any header extension, this is a setter on the local receiver.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Mike West]

LGTM3, with much the same reservations and rationale as Yoav. Two additions:

There seems to be clear agreement that this proposal addresses a problem that seems to be worth solving. The experimental data shows real-world improvements for real users on real applications, and there's apparently interest in this mechanism from outside Google as well. I think it's reasonable to ship the mechanism to gain more experience about how it's used in the wild. If the compatibility concerns raised earlier in the thread turn out to be valid, I'd expect us to be willing to revisit the shape of the API (perhaps along the lines of AudioContextOptions.latencyHint, which accepts either a specific double, or one of the AudioContextLatencyCategory enum values).

-mike

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEhePGdQMA88hbYjAcEGH1hfEX8HCHKtNnvaM5DZF7Fixg%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6664e711-35cd-43bf-9a9f-0233101db8d5%40chromium.org.

[Philipp Hancke]

ah, so the playout delay hint (which remotely controls this property) turned out to be not the right solution? Maybe you can update the webrtc.org page?

I've also filed https://github.com/henbos/webrtc-extensions/issues/8 since documentation on how to use it (trivial) and evaluate the impact (hoping for something more useful than 'less people complain about robotic voice) will greatly help with adoption.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEhePGdQMA88hbYjAcEGH1hfEX8HCHKtNnvaM5DZF7Fixg%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6664e711-35cd-43bf-9a9f-0233101db8d5%40chromium.org.

[jbru...@mozilla.com]

On Thursday, October 24, 2019 at 3:03:03 PM UTC-4, Philipp Hancke wrote:

ah, so the playout delay hint (which remotely controls this property) turned out to be not the right solution? Maybe you can update the webrtc.org page?

I assume you meant the header extension here?

[Ruslan Burakov]

Started WICG discourse thread here.

[Ruslan Burakov]

Started TAG review here.