Intent to Ship: adaptivePtime property for RTCRtpEncodingParameters

192 views
Skip to first unread message

Jakob Ivarsson

unread,
Mar 18, 2021, 10:00:46 AMMar 18
to blin...@chromium.org

Contact emails

min...@chromium.org, jak...@google.com, elad...@chromium.org, hb...@chromium.org


Explainer

https://w3c.github.io/webrtc-extensions/explainer#a-new-flag-in-rtcrtpencodingparameters-for-adaptive-packet-rate


Specification

https://w3c.github.io/webrtc-extensions/#dom-rtcrtpencodingparameters-adaptiveptime


Design docs

https://w3c.github.io/webrtc-extensions/explainer#a-new-flag-in-rtcrtpencodingparameters-for-adaptive-packet-rate


Summary

Adds the adaptivePtime property to RTCRtpEncodingParameters (https://w3c.github.io/webrtc-pc/#rtcrtpencodingparameters). This new flag sets a sender in a real-time communication (RTC) system to enable/disable adaptive packet rate.


The feature has been tested under an origin trial to verify that enabling the adaptive packet rate by using this API can benefit some RTC systems to achieve higher audio quality in low bandwidth networks. From testing of this API in Google Meet, we learned that such a goal can be achieved. 


The trade-off is higher latency which is a reason to add this API since some applications might have strict latency requirements.


Blink component

Blink>WebRTC


Search tags

congestion control, real-time communication, packet rate, bitrate


TAG review

https://github.com/w3ctag/design-reviews/issues/517


TAG review status

Issues addressed


Risks



Interoperability and Compatibility

Preventing potential interoperability issues with its underlying feature of adaptive packet time was a main reason for introducing this flag. Some implementations MAY have taken a fixed packet rate as an assumption, and thus may fail or perform sub-optimally with an adaptive packet rate.


As far as we know, all major browsers use the same audio receiver (NetEq) that handles variable packet rate well. The Opus spec also requires the decoder to handle this. This was validated during the origin trial where various browsers received the variable packet rate and no interoperability issues were found.


Regarding the implementation of the flag itself, Mozilla has been involved in the discussions of technical details of adaptive packet rate, and has said that the feature seemed fine, although they have not expressed a signal to implement it. Other vendors have not been contacted.

There is no compatibility risk. Setting the flag does NOT mean that the sender must change its frame rate dynamically, since the sender may decide to stay on the same packet rate, due to, for example, a stable network. Thus there is no behavioral difference between the case that the feature is not implemented, and the case that the feature is implemented but packet rate adaptation logic actively decides to keep the packet rate.

Gecko: Neutral (https://github.com/w3c/webrtc-pc/pull/2309#issuecomment-538463368) Mozilla has said that the feature seemed fine, although have not expressed a signal to implement it.


WebKit: No signal


Web developers: Positive



Is this feature fully tested by web-platform-tests?

No


Tracking bug

https://crbug.com/1086942


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5752004691361792


Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/lOi5XonBstA/m/sabhJM2jAwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Mt9GmmF-Vwc

Intent to Extend Origin Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/T1ad0wJQ9qQ 


This intent message was generated by Chrome Platform Status.


Philipp Hancke

unread,
Mar 18, 2021, 10:14:01 AMMar 18
to Jakob Ivarsson, blink-dev
In the field trial flags I see
 {"WebRTC-Audio-AdaptivePtime" => "min_encoder_bitrate:16"}
how would that be handled without the field trial? If this is going to be a hardcoded value, can you please make it 16000 for consistency with maxBitrate et al?

Looking forward to the feature!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABi2VuCnQsGvyXarxBKMZrr2ASyT4t0Zm02-fcYb8iJYtGkj0Q%40mail.gmail.com.

Daniel Bratell

unread,
Mar 22, 2021, 9:59:28 AMMar 22
to Philipp Hancke, Jakob Ivarsson, blink-dev

jakobi, this seems like a straight forward change, but can you comment on Philipp's question/suggestion in case there is something that needs to change before shipping?

/Daniel

Jakob Ivarsson

unread,
Mar 22, 2021, 10:09:33 AMMar 22
to Daniel Bratell, Philipp Hancke, blink-dev
We made a small configuration change during the origin trial which I forgot to hard code. This is fixed in: https://webrtc-review.googlesource.com/c/src/+/212501

Yoav Weiss

unread,
Mar 25, 2021, 5:08:16 AMMar 25
to blink-dev, jak...@google.com, philipp...@googlemail.com, blink-dev, Daniel Bratell
The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct? 

As far as we know, all major browsers use the same audio receiver (NetEq) that handles variable packet rate well. The Opus spec also requires the decoder to handle this. This was validated during the origin trial where various browsers received the variable packet rate and no interoperability issues were found.


Regarding the implementation of the flag itself, Mozilla has been involved in the discussions of technical details of adaptive packet rate, and has said that the feature seemed fine, although they have not expressed a signal to implement it. Other vendors have not been contacted.

There is no compatibility risk. Setting the flag does NOT mean that the sender must change its frame rate dynamically, since the sender may decide to stay on the same packet rate, due to, for example, a stable network. Thus there is no behavioral difference between the case that the feature is not implemented, and the case that the feature is implemented but packet rate adaptation logic actively decides to keep the packet rate.

Gecko: Neutral (https://github.com/w3c/webrtc-pc/pull/2309#issuecomment-538463368) Mozilla has said that the feature seemed fine, although have not expressed a signal to implement it.


WebKit: No signal

Can you please file for signals according to bit.ly/blink-signals

Web developers: Positive

Any links? 


Is this feature fully tested by web-platform-tests?

No


Tracking bug

https://crbug.com/1086942


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5752004691361792


Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/lOi5XonBstA/m/sabhJM2jAwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Mt9GmmF-Vwc

Intent to Extend Origin Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/T1ad0wJQ9qQ 


This intent message was generated by Chrome Platform Status.


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Jakob Ivarsson

unread,
Mar 25, 2021, 8:43:02 AMMar 25
to Yoav Weiss, blink-dev, philipp...@googlemail.com, Daniel Bratell
The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct?
Correct, the spec states that the flag is default false.

Can you please file for signals according to bit.ly/blink-signals?

Any links?
No good links unfortunately. I can only speak on behalf of Google Meet. We also have anecdotal reports of other developers trying out the API (e.g. https://crbug.com/webrtc/12242). Another data point is Philipp's comment earlier in this thread.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Chris Harrelson

unread,
Mar 25, 2021, 7:33:48 PMMar 25
to Jakob Ivarsson, Yoav Weiss, blink-dev, philipp...@googlemail.com, Daniel Bratell
On Thu, Mar 25, 2021 at 5:42 AM 'Jakob Ivarsson' via blink-dev <blin...@chromium.org> wrote:
The explainer is not explicit on that front, but I'm assuming that the feature is off without an explicit opt-in. Is that correct?
Correct, the spec states that the flag is default false.

Can you please file for signals according to bit.ly/blink-signals?

Thanks. For Mozilla you'll also need to file an issue to request an official signal.
 

Jakob Ivarsson

unread,
Mar 29, 2021, 5:42:45 AMMar 29
to Chris Harrelson, Yoav Weiss, blink-dev, philipp...@googlemail.com, Daniel Bratell
Thanks. For Mozilla you'll also need to file an issue to request an official signal.

Philip Jägenstedt

unread,
Mar 30, 2021, 6:59:50 AMMar 30
to Jakob Ivarsson, Youenn Fablet, Jan-Ivar Bruaroey, Chris Harrelson, Yoav Weiss, blink-dev, philipp...@googlemail.com, Daniel Bratell
Thanks for emailing webkit-dev and posting on mozilla/standards-positions!

A lot of WebRTC stuff discussed in the working group ends up being seen by @Youenn Fablet, but in this case going through https://github.com/w3c/webrtc-pc/issues/2300 and https://github.com/w3c/webrtc-pc/pull/2309 I can't spot anything. @Jan-Ivar Bruaroey was pinged there, though.

bit.ly/blink-signals says of these signals that "In all cases, a reasonable amount of time should be allowed for responses to this process." How much time to wait is the main question for me on this intent. Although this proposal has probably crossed the radar of some of the people involved, given it at least a week seems reasonable.

I presume the desire is for this to be enabled in M91, which branches on Thu, Apr 8? If we get feedback on the day before the branch point, is there a backup plan, such as continuing the original trial, while you work through the feedback?

Best regards,
Philip

Jakob Ivarsson

unread,
Mar 30, 2021, 8:02:36 AMMar 30
to Philip Jägenstedt, Youenn Fablet, Jan-Ivar Bruaroey, Chris Harrelson, Yoav Weiss, blink-dev, philipp...@googlemail.com, Daniel Bratell
Update from WebKit (https://lists.webkit.org/pipermail/webkit-dev/2021-March/031763.html): "This looks ok to me. I think it was discussed within WebRTC WG and reached consensus there as well."

Regarding the version to ship in: The origin trial ends in M89 so, even though we were very late with creating this intent, M90 would be the ideal version to ship in. M91 should be fine to ship in as well but in that case we would want another extension of the origin trial to also cover M90.

Yoav Weiss

unread,
Mar 30, 2021, 9:09:13 AMMar 30
to Jakob Ivarsson, Philip Jägenstedt, Youenn Fablet, Jan-Ivar Bruaroey, Chris Harrelson, blink-dev, philipp...@googlemail.com, Daniel Bratell
LGTM1

On Tue, Mar 30, 2021 at 2:02 PM Jakob Ivarsson <jak...@google.com> wrote:
Update from WebKit (https://lists.webkit.org/pipermail/webkit-dev/2021-March/031763.html): "This looks ok to me. I think it was discussed within WebRTC WG and reached consensus there as well."

Regarding the version to ship in: The origin trial ends in M89 so, even though we were very late with creating this intent, M90 would be the ideal version to ship in. M91 should be fine to ship in as well but in that case we would want another extension of the origin trial to also cover M90.

Are you hoping to merge back to M90? Or are you planning to enable it using server side config?

Chris Harrelson

unread,
Mar 30, 2021, 1:26:19 PMMar 30
to Yoav Weiss, Jakob Ivarsson, Philip Jägenstedt, Youenn Fablet, Jan-Ivar Bruaroey, blink-dev, philipp...@googlemail.com, Daniel Bratell
LGTM2

Jakob Ivarsson

unread,
Apr 1, 2021, 3:17:36 AMApr 1
to Chris Harrelson, Yoav Weiss, Philip Jägenstedt, Youenn Fablet, Jan-Ivar Bruaroey, blink-dev, philipp...@googlemail.com, Daniel Bratell
Are you hoping to merge back to M90? Or are you planning to enable it using server side config?

 We got approval to merge back to M90 if this gets LGTM before Monday.

Philip Jägenstedt

unread,
Apr 1, 2021, 3:50:09 AMApr 1
to Jakob Ivarsson, Chris Harrelson, Yoav Weiss, Youenn Fablet, Jan-Ivar Bruaroey, blink-dev, philipp...@googlemail.com, Daniel Bratell
Hi Jakob,

The remaining blocker here per our own process is https://github.com/mozilla/standards-positions/issues/508, where "a reasonable amount of time should be allowed for responses."

Nils is going to be back on Monday, so maybe his feedback will just in time, depending on timezones and the interpretation of "before Monday."

I realize that the overwhelming likelihood in this case is that Mozilla will say "seems fine" again, but if we don't consistently give the process "a reasonable amount of time" it won't seem meaningful for other vendors to provide feedback when we're close to launching. 

So I'm pretty sure this is going to be fine, but would like to wait until Nils has replied on #508, with a timeout on Wednesday or so.

If the M90 merge ends up not possible, we should definitely extend the original trial to M90 and I'd be happy to LGTM that.

As a meta point, I assume this is surprising and frustrating, so is there a place in the blink launch process documentation where it should be more clear that TAG review and vendor "Official Standards Process" requests should be filed as early as possible?

Best regards,
Philip

Philip Jägenstedt

unread,
Apr 8, 2021, 4:27:47 AMApr 8
to Jakob Ivarsson, Chris Harrelson, Yoav Weiss, Youenn Fablet, Jan-Ivar Bruaroey, blink-dev, philipp...@googlemail.com, Daniel Bratell
It's been a week now. Yoav just pinged https://github.com/mozilla/standards-positions/issues/508, but a total of 10 days seems like a reasonable amount of time to wait in this case.

LGTM3 to ship in M91 and extend the origin trial to M90.

If there should be any more feedback from any vendor, positive or otherwise, please do circle back to this thread.
Reply all
Reply to author
Forward
0 new messages