Intent to Ship: WebRTC Scalable Video Coding extensions

456 views
Skip to first unread message

Harald Alvestrand

unread,
Oct 25, 2021, 6:04:03 AM10/25/21
to blink-dev

DRFT


Contact emails

h...@chromium.org, sergio.gar...@gmail.com


Explainer

https://w3c.github.io/webrtc-svc/


Specification

https://w3c.github.io/webrtc-svc/


Design docs


https://w3c.github.io/webrtc-svc/


Summary

This extension defines a standard method for picking between possible Scalable Video Coding (SVC)  configurations on an outgoing WebRTC video track.




Blink component

Blink>WebRTC>Video


Search tags

video, svc


TAG review

https://github.com/w3ctag/design-reviews/issues/396


TAG review status

Issues addressed


Risks

None identified

Interoperability and Compatibility

This is the first implementation of the spec.


Gecko: Positive - https://github.com/mozilla/standards-positions/issues/544 


WebKit: Positive, with change request  - https://lists.webkit.org/pipermail/webkit-dev/2021-October/032008.html

https://github.com/w3c/webrtc-svc/issues/49 



Web developers: Positive (sample)


Ergonomics

This is an extension to the WebRTC 1.0 API (webrtc-pc).
Support can be detected by checking the presence of scalabilityModes values in the RTCRtpCodecCapability dictionary.


Security

None identified above what's already part of video in WebRTC in general.



Debuggability

No special considerations.


Is this feature fully tested by web-platform-tests?

Yes (as far as the API surface goes; effect on bitstream content is not tested).


Flag name

RTCSvcScalabilityMode


Requires code in //chrome?

False


Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=986069


Estimated milestones

M97



Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5769626174619648


This intent message was generated by Chrome Platform Status.


Harald Alvestrand

unread,
Oct 25, 2021, 6:07:19 AM10/25/21
to blink-dev
Note - the change request from WebKit (https://github.com/w3c/webrtc-svc/issues/49) involves removing the list of scalability modes from the codec information in WebRTC and instead using the MediaCapabilities API with the "webrtc" usage to detect whether a given scalability mode is available or not.

This would require gating this API behind the updated MediaCapabilities API, and would not give exactly the same programming paradigm (see bug).

Fernando Serboncini

unread,
Oct 25, 2021, 3:32:30 PM10/25/21
to Harald Alvestrand, blink-dev
It seems they are asking for a delay on Chrome launching this until the WebRTC WG makes a decision on it.
It's not clear from the issue that there's a consensus on the right approach there.

Did you consider launching things separately and delaying the scalability modes? Or does the whole launch make no sense without it?

Harald Alvestrand

unread,
Oct 25, 2021, 5:00:46 PM10/25/21
to Fernando Serboncini, blink-dev
The scalability modes (being able to set them) are the point of the launch.
Figuring out which of the desired ones are available seems like it would be a requirement.

Yoav Weiss

unread,
Nov 10, 2021, 1:04:05 PM11/10/21
to blink-dev, Harald Alvestrand, blink-dev, Fernando Serboncini
How long of a delay are we talking about here? Weeks? Months? Years?

Mike West

unread,
Nov 24, 2021, 9:21:29 AM11/24/21
to Yoav Weiss, blink-dev, Harald Alvestrand, Fernando Serboncini
Friendly ping on Yoav's question about timelines.

-mike


--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1197505b-23e6-491a-8fc6-4b386cce0bcen%40chromium.org.

Harald Alvestrand

unread,
Dec 6, 2021, 9:27:02 AM12/6/21
to Mike West, Yoav Weiss, blink-dev, Fernando Serboncini
It seems like we don't have a strong push towards either the MediaCapabilities or the Codec Capabilities solution in the issue on the sender side (https://github.com/w3c/webrtc-svc/issues/49). This is bad for quick resolution.

In the interest of getting the uncontroversial parts shipped - what would people think of letting the Codec Capabilities list of modes remain behind the flag, and push the rest of the API to public?
Many usages of the function would work quite well with only probing for supported modes by trying to set the ones they want; we could then let the issue sort itself out in peace.

(On the receiver side, there seems to be consensus on abandoning the mode list for other reasons.)



Philip Jägenstedt

unread,
Dec 8, 2021, 11:52:37 AM12/8/21
to Harald Alvestrand, Mike West, Yoav Weiss, blink-dev, Fernando Serboncini
Hi Harald,

Can you spell out what the uncontroversial parts of this would be? Looking at the IDL in https://w3c.github.io/webrtc-svc/ it looks like it's all about modes.

My guess is that it's RTCRtpEncodingParameters's scalabilityMode, but is that right?

Best regards,
Philip

Philipp Hancke

unread,
Dec 8, 2021, 12:37:57 PM12/8/21
to Philip Jägenstedt, Harald Alvestrand, Mike West, Yoav Weiss, blink-dev, Fernando Serboncini
Am Mi., 8. Dez. 2021 um 17:52 Uhr schrieb Philip Jägenstedt <foo...@chromium.org>:
Hi Harald,

Can you spell out what the uncontroversial parts of this would be? Looking at the IDL in https://w3c.github.io/webrtc-svc/ it looks like it's all about modes.

My guess is that it's RTCRtpEncodingParameters's scalabilityMode, but is that right?

Bernard Aboba

unread,
Dec 8, 2021, 9:21:37 PM12/8/21
to blink-dev, Philipp Hancke, Harald Alvestrand, mk...@chromium.org, yoav...@chromium.org, blink-dev, fs...@chromium.org, Philip Jägenstedt, Chris Cunningham
Harald said: 

"It seems like we don't have a strong push towards either the MediaCapabilities or the Codec Capabilities solution in the issue on the sender side (https://github.com/w3c/webrtc-svc/issues/49). This is bad for quick resolution."

[BA] On the receiver/decoder side (for WebRTC-SVC, Media Capabilities and WebCodecs), we have a path forward  which involves using a referenceScaling boolean and removing scalabiltyMode advertisement and configuration.  The consensus is  reflected in the current editor's draft of WebRTC-SVC (see:  https://w3c.github.io/webrtc-svc/ ) and compatible PRs are under development for MediaCapabilities and WebCodecs. 

On the sender/encoder side, we have added the "L1T1" scalability mode and specified its use in both advertisement and encoder configuration. 

Chris can provide more details with respect to the moving parts in Media Capabilities and WebCodecs. 

Here are links to the (now resolved) WebRTC-SVC issues:

Here are links to related WebCodecs issues: 

Here are links to the related Media Capabilities issues:  

Philip Jägenstedt

unread,
Dec 15, 2021, 12:01:32 PM12/15/21
to Bernard Aboba, blink-dev, Philipp Hancke, Harald Alvestrand, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Chris Cunningham
Hi Bernard,

Can you clarify what the consensus is on RTCRtpEncodingParameters's scalabilityMode member? That remains in https://w3c.github.io/webrtc-svc/, but will it be removed? Meanwhile, referenceScaling is only partly spec'd, there's no IDL for it but a link to https://github.com/w3c/media-capabilities/issues/182.

Harald, if you could confirm the precise API surface that you'd like to ship, that would be great.

Best regards,
Philip

Bernard Aboba

unread,
Dec 15, 2021, 1:23:18 PM12/15/21
to blink-dev, Philip Jägenstedt, blink-dev, Philipp Hancke, Harald Alvestrand, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Chris Cunningham, Bernard Aboba
There seems to be agreement to add support for referenceScaling in Media Capabilities (https://github.com/w3c/media-capabilities/issues/182) so I'm assuming that a PR will follow.  

Chris - please jump in if I'm misunderstanding the consensus on Issue 182. 

RTCRtpEncodingParameters.scalabilityMode (Section 5.1) is  required in order to set the scalability mode in RTCRtpSender.setParameters() as well as addTransceiver().   There has been no discussion of removing it. 

Harald Alvestrand

unread,
Dec 15, 2021, 6:14:42 PM12/15/21
to Philip Jägenstedt, Bernard Aboba, blink-dev, Philipp Hancke, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Chris Cunningham
At the moment, I think we can safely ship:

- RTCRtpEncodingParameters extension scalabilityMode (https://w3c.github.io/webrtc-svc/#dom-rtcrtpencodingparameters-scalabilitymode)

We have an open discussion on whether or not to ship this part on senders (we've decided not to ship it on receivers):

- RTCRtpCodecCapability extension scalabilityModes (https://w3c.github.io/webrtc-svc/#dom-rtcrtpcodeccapability-scalabilitymodes)

There are no mandatory-to-implement scalability modes except for L1T1 (which we need to add support for).

I think that as currently specified, feature detection can be done in the absence of the RTCRtpCodecCapability extension by setting the mode to L1T1, reading back the encoding parameters, and seeing if the mode is set.



Chris Cunningham

unread,
Dec 20, 2021, 3:07:14 PM12/20/21
to blink-dev, Harald Alvestrand, Bernard Aboba, blink-dev, philipp...@googlemail.com, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Chris Cunningham, Philip Jägenstedt, Johannes Kron
Sorry I'm late. Lots of family stuff this month. I'm about to be OOO for the holidays. 


> There seems to be agreement to add support for referenceScaling in Media Capabilities (https://github.com/w3c/media-capabilities/issues/182) so I'm assuming that a PR will follow.  

I can confirm this agreement for MediaCapabilities. I expect +Johannes Kron will send a PR to amend the MC spec. 

Chris Harrelson

unread,
Jan 5, 2022, 11:51:44 AMJan 5
to Chris Cunningham, blink-dev, Harald Alvestrand, Bernard Aboba, philipp...@googlemail.com, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Philip Jägenstedt, Johannes Kron
LGTM1 to ship scalabilityMode, but not the pluralized name or referenceScaling.

Please open a new intent if you wish to ship one of the others (otherwise this intent-to-ship would be too confusing).

Thanks, and happy new year.

Daniel Bratell

unread,
Jan 5, 2022, 11:52:55 AMJan 5
to Chris Harrelson, Chris Cunningham, blink-dev, Harald Alvestrand, Bernard Aboba, philipp...@googlemail.com, mk...@chromium.org, yoav...@chromium.org, fs...@chromium.org, Philip Jägenstedt, Johannes Kron

Yoav Weiss

unread,
Jan 5, 2022, 12:16:28 PMJan 5
to Daniel Bratell, Chris Harrelson, Chris Cunningham, blink-dev, Harald Alvestrand, Bernard Aboba, philipp...@googlemail.com, mk...@chromium.org, fs...@chromium.org, Philip Jägenstedt, Johannes Kron
LGTM3

Joe Medley

unread,
Jan 5, 2022, 2:57:48 PMJan 5
to blink-dev, yoav...@chromium.org, Chris Harrelson, Chris Cunningham, blink-dev, Harald Alvestrand, Bernard Aboba, philipp...@googlemail.com, mk...@chromium.org, fs...@chromium.org, Philip Jägenstedt, Johannes Kron, Daniel Bratell
When are you hoping to ship?
Reply all
Reply to author
Forward
0 new messages