Intent to Extend Experiment: WebRTC encoded transform - Constructor with custom Metadata (originally Modify Metadata functions)

[Guido Urdaneta]

Contact emails

h...@chromium.org, gui...@chromium.org, agp...@chromium.org

Explainer

https://github.com/guidou/webrtc-extensions/blob/main/constructor-explainer.md

Specification

https://w3c.github.io/webrtc-encoded-transform/#dom-rtcencodedvideoframe-constructor

Summary

Allow WebRTC Encoded Transform API to manipulate audio and video frame metadata. Some WebRTC Encoded Transform use cases involve manipulation of not only the payload of encoded video / audio frames but also its metadata. Some examples: * Altering the timestamp of a frame to introduce a delay, * Changing the mime type of the frame if the transform changes the type of the payload. * Forwarding of media to a new peer connection set up to use different metadata values Use cases: https://w3c.github.io/webrtc-nv-use-cases/#live-encoded-media https://w3c.github.io/webrtc-nv-use-cases/#stored-encoded-media https://w3c.github.io/webrtc-nv-use-cases/#auction Issue link: https://github.com/w3c/webrtc-nv-use-cases/issues/77

Blink component

Blink>WebRTC

TAG review

TAG review request for this specific change: https://github.com/w3ctag/design-reviews/issues/942

The original full spec was reviewed by TAG here: https://github.com/w3ctag/design-reviews/issues/531

TAG review status

Pending

Origin Trial Name

RTCEncodedFrameSetMetadata

Chromium Trial Name

RTCEncodedFrameSetMetadata

Origin Trial documentation link

https://github.com/palak8669/webrtc-encoded-transform/blob/create-encoded-explainer/create-encoded-explainer.md

WebFeature UseCounter name

RTCEncodedFrameSetMetadata

Risks

Interoperability and Compatibility

Interoperability risk: There is always the risk that other browsers will not implement this feature. This risk is mitigated by alignment across browser vendors in the W3C WebRTC Working Group around the spec. Compatibility risk: This is a new feature intended to support new use cases. It introduces no breaking changes, so we do not expect any compatibility issues.

Gecko: Positive (https://github.com/mozilla/standards-positions/issues/1009) During WebRTC WG meetings, Mozilla has shown positive signals and agreed with merging the PR in the main spec. See the exchange in https://github.com/w3c/webrtc-encoded-transform/pull/223

WebKit: Closed Without a Position (https://github.com/WebKit/standards-positions/issues/334) Marked as "Invalid" in the position request because this is a small change and the position was addressed in the PR review. The comments from WebKit in the PR review and during WebRTC WG meetings are positive and they have agreed with merging the PR in the main spec. See the exchange in https://github.com/w3c/webrtc-encoded-transform/pull/223

Web developers: Positive

Other signals:

Ergonomics

This feature is an extension to WebRTC Encoded Transform, which itself is an extension to WebRTC/RTCPeerConnection.

Activation

No significant risks identified.

Security

No new security risks identified.

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

No

Goals for experimentation

Reason this experiment is being extended

This feature shipped in M130, but OT partners indicate that a nontrivial number of their users are still on versions earlier than M130 and would like to give them more time to upgrade to M130+. We are requesting to extend the trial until M132.

Ongoing technical constraints

None

Debuggability

N/A

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes

https://wpt.fyi/results/webrtc-encoded-transform/tentative/RTCEncodedAudioFrame-metadata.https.html?label=master&label=experimental&aligned https://wpt.fyi/results/webrtc-encoded-transform/tentative/RTCEncodedVideoFrame-metadata.https.html?label=master&label=experimental&aligned

Flag name on about://flags

Finch feature name

RTCEncodedFrameSetMetadata

Non-finch justification

Guarded by a Blink RuntimeEnabledFeature.

Requires code in //chrome?

False

Tracking bug

https://issues.chromium.org/issues/40248396

Estimated milestones

Shipping on desktop	130
Origin trial desktop first	118
Origin trial desktop last	129
Origin trial extension 1 end milestone	129
Origin trial extension 2 end milestone	132
Origin trial extension 3 end milestone	127
Shipping on Android	130
Origin trial Android first	118
Origin trial Android last	129
Shipping on WebView	130
Origin trial WebView first	118
Origin trial WebView last	129

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5116073827893248?gate=5167176443756544

Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/x2ZACgXrqp0
Intent to Experiment: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxazRts59rCgrOHm2yDKwpGkXqsd-_5Wkurxid34FknDiQ%40mail.gmail.com
Intent to Extend Experiment 1: https://groups.google.com/a/chromium.org/g/blink-dev/c/dA4TndGG4VQ
Intent to Extend Experiment 2: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOmohSLP2iiU323j259Jr2wiwXRvxXd1UrUKA8g4bse4hF3zuA%40mail.gmail.com
Intent to Ship: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c04d3cba-f2de-420a-9938-6848adba9568n%40chromium.org

This intent message was generated by Chrome Platform Status.

[Mike Taylor]

Can you explain more? Part of the OT contract is that a site should be prepared to handle cases where the feature is not available (i.e., "I understand that I may need to apply feature detection / graceful degradation to handle the case where the experimental feature is unavailable.").

Is this a single partner? Multiple? Does introducing support this feature somehow make not supporting it more difficult than usual?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BBuZxYqJr6PnbbrFiTBFK9D4Ujy8ePY3nWaNDEnxP0wcxoQKw%40mail.gmail.com.

[Guido Urdaneta]

I can't share partner details, but at least one is in the middle of a tricky update process, complicated by the holiday period.

They can handle the case when the API is not available by disabling a product feature that depends on this API, but they would prefer to have that feature working during this period.

Also, this API was originally supposed to ship around M128, but due to an error, it shipped on M130, which made the update process a bit harder. 

[Mike Taylor]

I see - thanks for the extra context. I think under "normal" circumstances I would have stronger feelings - but given the mistake which prevented shipping in 128 and challenges around holiday freezes, LGTM to extend to M132 inclusive. 

I would suggest to your partners that further extensions won't be approved, and they should plan accordingly.