Intent to Ship: WebRTC Encoded Tansform (V2)

[Chromestatus]

Contact emails

gui...@chromium.org, h...@chromium.org, top...@chromium.org, alons...@google.com, gui...@google.com

Explainer

None

Specification

https://github.com/w3c/webrtc-encoded-transform

Summary

This API allows processing of encoded media flowing through an RTCPeerConnection. Chromium shipped an early version of this API in 2020. Since then, the spec has changed and other browsers have shipped the updated version of the spec (Safari in 2022 and Firefox in 2023). This launch refers to the latest spec version and is part of Interop 2025. This launch does not cover the generateKeyFrame method, which is still under discussion https://github.com/w3c/webrtc-encoded-transform/issues/143 https://github.com/w3c/webrtc-encoded-transform/issues/271 https://github.com/w3c/webrtc-encoded-transform/pull/269

Blink component

Blink>WebRTC>PeerConnection

TAG review

None

TAG review status

Issues addressed

Risks

Interoperability and Compatibility

Interoperability: This change improves interoperability since it implements an API already implemented by other browsers. Interoperability risks are relatively low and derive from slight differences in implementation. We have seen that WPTs don't have the same pass rate for all implementations, but we have observed that in most cases this is due to the WPTs testing unspecified behavior. Compatibility: Existing applications using the old version of the API will continue to work without changes. Existing applications using the new version of the API should start working with Chromium.

Gecko: Shipped/Shipping

WebKit: Shipped/Shipping

Web developers: Positive

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

N/A

Debuggability

None

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes

https://wpt.fyi/results/webrtc-encoded-transform?label=experimental&label=master&aligned

Flag name on about://flags

RTCRtpScriptTransform

Finch feature name

RTCRtpScriptTransform

Rollout plan

Will ship enabled for all users

Requires code in //chrome?

False

Tracking bug

https://issues.chromium.org/issues/354881878

Availability expectation

Feature available on all major browsers.

Adoption expectation

Feature is used by specific partner(s) to provide functionality within 12 months of launch in Chrome

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function?

None

Sample links

https://webrtc.github.io/samples/src/content/insertable-streams/endtoend-encryption

Estimated milestones

Shipping on desktop	141
Shipping on Android	141
Shipping on WebView	141

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5175278159265792?gate=5141653464285184

This intent message was generated by Chrome Platform Status.

[Philip Jägenstedt]

Very happy to see us approaching interoperability on this feature!

Looking at https://wpt.fyi/results/webrtc-encoded-transform?label=experimental&label=master&aligned there are quite a lot of failures in all browsers. Does wpt.fyi reflect the test results we'll have after enabling this, and are there a problem with the test suite that needs to be addressed here?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/689cc98b.050a0220.b43f3.1226.GAE%40google.com.

[Guido Urdaneta]

The multi-browser failures in wpt.fyi are mainly due to their including features that are still under discussion (SFrame and generateKeyFrame). SFrame has not been shipped by any browser yet (it is not covered in the Interop 2025 effort). Safari and Firefox do have generateKeyFrame, but they are not completely compatible and there are open issues that are being discussed in the W3C WebRTC WG. We are not including these features in this launch.

The generateKeyFrame issues are expected to be resolved after the WG discussion, and we plan to ship it shortly after.

The results of Chrome and Edge will improve with this launch, in particular script-metadata-transform.https.html, script-transform-sendKeyFrameRequest.https.html and RTCRtpScriptTransform-encoded-transform.https.html will pass fully, which will put them on par with Firefox and Safari.

[Yoav Weiss (@Shopify)]

Can the failing tests be renamed to `.tentative` until those discussions settle?

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxY0oMTxiSQMjZifiusK%3DL7yFd6BNggn6emi1Pxha%3DRjEQ%40mail.gmail.com.

[Guido Urdaneta]

Since those tests are included in Interop 2025 I believe we cannot unilaterally make them tentative. However, it is likely that some parts of the generateKeyFrame WPTs will change after the upcoming discussion.

Since we are leaving generateKeyFrame out of this launch I think the tentative status of those tests should not affect this I2S.

Note also that Chromium already supports the functionality of generating key frames via the setParameters API (see https://w3c.github.io/webrtc-extensions/#dom-rtcencodingoptions, I2S: https://groups.google.com/a/chromium.org/g/blink-dev/c/pd3Hksi3jq0). This is a better API for this functionality that has the advantage of being decoupled from Encoded Transform.

[Daniel Bratell]

LGTM1

Regardless of how you handle those tests for now, I assume this is just the first step and once the spec is complete, it will all be good.

/Daniel

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxYeUckh5fv8S3DckUW6%2BRGiL38M9ZArp17frvYMJbLtaQ%40mail.gmail.com.

[Chris Harrelson]

LGTM2

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d1dbe684-2f57-4d5c-8dc6-a7a70bdbcdf7%40gmail.com.

[Yoav Weiss (@Shopify)]

LGTM3