Intent to Ship: adaptivePtime property for RTCRtpEncodingParameters

[Jakob Ivarsson]

Contact emails

min...@chromium.org, jak...@google.com, elad...@chromium.org, hb...@chromium.org

Explainer

https://w3c.github.io/webrtc-extensions/explainer#a-new-flag-in-rtcrtpencodingparameters-for-adaptive-packet-rate

Specification

https://w3c.github.io/webrtc-extensions/#dom-rtcrtpencodingparameters-adaptiveptime

Design docs

https://w3c.github.io/webrtc-extensions/explainer#a-new-flag-in-rtcrtpencodingparameters-for-adaptive-packet-rate

Summary

Adds the adaptivePtime property to RTCRtpEncodingParameters (https://w3c.github.io/webrtc-pc/#rtcrtpencodingparameters). This new flag sets a sender in a real-time communication (RTC) system to enable/disable adaptive packet rate.

The feature has been tested under an origin trial to verify that enabling the adaptive packet rate by using this API can benefit some RTC systems to achieve higher audio quality in low bandwidth networks. From testing of this API in Google Meet, we learned that such a goal can be achieved. 

The trade-off is higher latency which is a reason to add this API since some applications might have strict latency requirements.

Blink component

Blink>WebRTC

Search tags

congestion control, real-time communication, packet rate, bitrate

TAG review

https://github.com/w3ctag/design-reviews/issues/517

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

Preventing potential interoperability issues with its underlying feature of adaptive packet time was a main reason for introducing this flag. Some implementations MAY have taken a fixed packet rate as an assumption, and thus may fail or perform sub-optimally with an adaptive packet rate.

As far as we know, all major browsers use the same audio receiver (NetEq) that handles variable packet rate well. The Opus spec also requires the decoder to handle this. This was validated during the origin trial where various browsers received the variable packet rate and no interoperability issues were found.

Regarding the implementation of the flag itself, Mozilla has been involved in the discussions of technical details of adaptive packet rate, and has said that the feature seemed fine, although they have not expressed a signal to implement it. Other vendors have not been contacted.

There is no compatibility risk. Setting the flag does NOT mean that the sender must change its frame rate dynamically, since the sender may decide to stay on the same packet rate, due to, for example, a stable network. Thus there is no behavioral difference between the case that the feature is not implemented, and the case that the feature is implemented but packet rate adaptation logic actively decides to keep the packet rate.

Gecko: Neutral (https://github.com/w3c/webrtc-pc/pull/2309#issuecomment-538463368) Mozilla has said that the feature seemed fine, although have not expressed a signal to implement it.

WebKit: No signal

Web developers: Positive

Is this feature fully tested by web-platform-tests?

No

Tracking bug

https://crbug.com/1086942

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5752004691361792

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/lOi5XonBstA/m/sabhJM2jAwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Mt9GmmF-Vwc

Intent to Extend Origin Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/T1ad0wJQ9qQ 

This intent message was generated by Chrome Platform Status.

[Philipp Hancke]

In the field trial flags I see

 {"WebRTC-Audio-AdaptivePtime" => "min_encoder_bitrate:16"}

how would that be handled without the field trial? If this is going to be a hardcoded value, can you please make it 16000 for consistency with maxBitrate et al?

Looking forward to the feature!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuCnQsGvyXarxBKMZrr2ASyT4t0Zm02-fcYb8iJYtGkj0Q%40mail.gmail.com.

[Daniel Bratell]

jakobi, this seems like a straight forward change, but can you comment on Philipp's question/suggestion in case there is something that needs to change before shipping?

/Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKi%2Bw4Vsgcrd0HZYqMx2BdRDQzAKrJE9gk3edHkum78HPxg%40mail.gmail.com.

[Jakob Ivarsson]

We made a small configuration change during the origin trial which I forgot to hard code. This is fixed in: https://webrtc-review.googlesource.com/c/src/+/212501

[Yoav Weiss]

The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct? 

As far as we know, all major browsers use the same audio receiver (NetEq) that handles variable packet rate well. The Opus spec also requires the decoder to handle this. This was validated during the origin trial where various browsers received the variable packet rate and no interoperability issues were found.

Regarding the implementation of the flag itself, Mozilla has been involved in the discussions of technical details of adaptive packet rate, and has said that the feature seemed fine, although they have not expressed a signal to implement it. Other vendors have not been contacted.

There is no compatibility risk. Setting the flag does NOT mean that the sender must change its frame rate dynamically, since the sender may decide to stay on the same packet rate, due to, for example, a stable network. Thus there is no behavioral difference between the case that the feature is not implemented, and the case that the feature is implemented but packet rate adaptation logic actively decides to keep the packet rate.

Gecko: Neutral (https://github.com/w3c/webrtc-pc/pull/2309#issuecomment-538463368) Mozilla has said that the feature seemed fine, although have not expressed a signal to implement it.

WebKit: No signal

Can you please file for signals according to bit.ly/blink-signals? 

Web developers: Positive

Any links? 

Is this feature fully tested by web-platform-tests?

No

Tracking bug

https://crbug.com/1086942

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5752004691361792

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/lOi5XonBstA/m/sabhJM2jAwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Mt9GmmF-Vwc

Intent to Extend Origin Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/T1ad0wJQ9qQ 

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuCnQsGvyXarxBKMZrr2ASyT4t0Zm02-fcYb8iJYtGkj0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Jakob Ivarsson]

The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct?

Correct, the spec states that the flag is default false.

Can you please file for signals according to bit.ly/blink-signals?

Done: https://lists.webkit.org/pipermail/webkit-dev/2021-March/031757.html

Any links?

No good links unfortunately. I can only speak on behalf of Google Meet. We also have anecdotal reports of other developers trying out the API (e.g. https://crbug.com/webrtc/12242). Another data point is Philipp's comment earlier in this thread.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuCnQsGvyXarxBKMZrr2ASyT4t0Zm02-fcYb8iJYtGkj0Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Chris Harrelson]

On Thu, Mar 25, 2021 at 5:42 AM 'Jakob Ivarsson' via blink-dev <blin...@chromium.org> wrote:

The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct?

Correct, the spec states that the flag is default false.

Can you please file for signals according to bit.ly/blink-signals?

Done: https://lists.webkit.org/pipermail/webkit-dev/2021-March/031757.html

Thanks. For Mozilla you'll also need to file an issue to request an official signal.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuDjFVjEwdJyKZis6K1-fkQ0MpEfGqzys0%3Dm8jn%3DtszGMQ%40mail.gmail.com.

[Jakob Ivarsson]

Thanks. For Mozilla you'll also need to file an issue to request an official signal.

Sorry, I didn't realize that. Done: https://github.com/mozilla/standards-positions/issues/508

[Philip Jägenstedt]

Thanks for emailing webkit-dev and posting on mozilla/standards-positions!

A lot of WebRTC stuff discussed in the working group ends up being seen by @Youenn Fablet, but in this case going through https://github.com/w3c/webrtc-pc/issues/2300 and https://github.com/w3c/webrtc-pc/pull/2309 I can't spot anything. @Jan-Ivar Bruaroey was pinged there, though.

bit.ly/blink-signals says of these signals that "In all cases, a reasonable amount of time should be allowed for responses to this process." How much time to wait is the main question for me on this intent. Although this proposal has probably crossed the radar of some of the people involved, given it at least a week seems reasonable.

I presume the desire is for this to be enabled in M91, which branches on Thu, Apr 8? If we get feedback on the day before the branch point, is there a backup plan, such as continuing the original trial, while you work through the feedback?

Best regards,

Philip

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuDWB%2B844-BpH43AoBhuKP%3DjX2w4_ydnaKPs6AJyAHByAA%40mail.gmail.com.

[Jakob Ivarsson]

Update from WebKit (https://lists.webkit.org/pipermail/webkit-dev/2021-March/031763.html): "This looks ok to me. I think it was discussed within WebRTC WG and reached consensus there as well."

Regarding the version to ship in: The origin trial ends in M89 so, even though we were very late with creating this intent, M90 would be the ideal version to ship in. M91 should be fine to ship in as well but in that case we would want another extension of the origin trial to also cover M90.

[Yoav Weiss]

LGTM1

On Tue, Mar 30, 2021 at 2:02 PM Jakob Ivarsson <jak...@google.com> wrote:

Update from WebKit (https://lists.webkit.org/pipermail/webkit-dev/2021-March/031763.html): "This looks ok to me. I think it was discussed within WebRTC WG and reached consensus there as well."

Regarding the version to ship in: The origin trial ends in M89 so, even though we were very late with creating this intent, M90 would be the ideal version to ship in. M91 should be fine to ship in as well but in that case we would want another extension of the origin trial to also cover M90.

Are you hoping to merge back to M90? Or are you planning to enable it using server side config?

[Chris Harrelson]

LGTM2

[Jakob Ivarsson]

Are you hoping to merge back to M90? Or are you planning to enable it using server side config?

 We got approval to merge back to M90 if this gets LGTM before Monday.

[Philip Jägenstedt]

Hi Jakob,

The remaining blocker here per our own process is https://github.com/mozilla/standards-positions/issues/508, where "a reasonable amount of time should be allowed for responses."

Nils is going to be back on Monday, so maybe his feedback will just in time, depending on timezones and the interpretation of "before Monday."

I realize that the overwhelming likelihood in this case is that Mozilla will say "seems fine" again, but if we don't consistently give the process "a reasonable amount of time" it won't seem meaningful for other vendors to provide feedback when we're close to launching. 

So I'm pretty sure this is going to be fine, but would like to wait until Nils has replied on #508, with a timeout on Wednesday or so.

If the M90 merge ends up not possible, we should definitely extend the original trial to M90 and I'd be happy to LGTM that.

As a meta point, I assume this is surprising and frustrating, so is there a place in the blink launch process documentation where it should be more clear that TAG review and vendor "Official Standards Process" requests should be filed as early as possible?

Best regards,

Philip

[Philip Jägenstedt]

It's been a week now. Yoav just pinged https://github.com/mozilla/standards-positions/issues/508, but a total of 10 days seems like a reasonable amount of time to wait in this case.

LGTM3 to ship in M91 and extend the origin trial to M90.

If there should be any more feedback from any vendor, positive or otherwise, please do circle back to this thread.