Intent to Ship: Expose rtpTimestamp from WebRTC video frames via VideoFrame.metadata()

[Anantanarayanan Iyengar]

Contact emails

dalec...@chromium.org, aiye...@nvidia.com

Specification

https://www.w3.org/TR/webcodecs-video-frame-metadata-registry/#dom-videoframemetadata

Summary

Adds a VideoFrame.metadata() method that returns a dictionary containing the rtpTimestamp field, if the underlying VideoFrame has this field in its native metadata. An empty dictionary is returned otherwise. Only video frames originating from WebRTC sources will have the rtpTimestamp metadata attached. Additional metadata fields are already present in the native implementation and may be exposed to JavaScript over time, as outlined in the proposed spec: https://www.w3.org/TR/webcodecs-video-frame-metadata-registry/#dom-videoframemetadata

Blink component

Blink>Media>WebCodecs

Web Feature ID

webcodecs

Motivation

This feature exposes the rtpTimestamp field on the JavaScript facing VideoFrame.metadata dictionary when it is present in the underlying native media::VideoFrameMetadata. It allows applications using MediaStreamTrackProcessor (e.g., to render decoded WebRTC frames to a canvas) or WebCodecs (e.g., for custom decoding pipelines) to correlate each exposed frame with its original RTP transport timestamp. This is useful for: Media synchronization across tracks Jitter or latency diagnostics Aligning decoded video with captured or received audio Spec link: https://www.w3.org/TR/webcodecs-video-frame-metadata-registry/#dom-videoframemetadata-rtptimestamp Chromium implementation CL: https://chromium-review.googlesource.com/c/chromium/src/+/6499588 Chromium bug: https://crbug.com/414545889

Initial public proposal

No information provided

TAG review

No information provided

TAG review status

Not applicable

Risks

Interoperability and Compatibility

The feature is additive and backward compatible. Existing WebCodecs and WebRTC APIs remain unchanged.

        No known interop issues. WPT coverage validates expected behavior. Firefox and WebKit positions are         pending.

Gecko: No signal (https://github.com/mozilla/standards-positions/issues/1233) Position request filed on May 19, 2025. Awaiting response.

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/497) Position request filed on May 19, 2025. Awaiting a response from WebKit.

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None. This feature only exposes additional read only metadata (rtpTimestamp) on VideoFrame objects that are already surfaced through WebCodecs and WebRTC pipelines. No changes to existing Android WebView behavior or APIs. Safe for WebView.

Debuggability

No impact to DevTools workflows. Feature testing is via JS API inspection and WPT automation.

VideoFrame metadata inspection (including rtpTimestamp) can be done via Javascript using MediaStreamTrackProcessor and WebCodecs/WebRTC or automated via WebDriver BiDi tests.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes
https://wpt.fyi/results/webcodecs/videoFrame-metadata-rtpTimestamp.https.html?label=experimental&label=master&aligned

Flag name on about://flags

--enable-blink-features=VideoFrameMetadataRtpTimestamp

Finch feature name

None. Rollout is via milestone 146 stable. No finch experiment planned.

Non-finch justification

The feature is additive and backward compatible. Existing WebCodecs and WebRTC APIs remain unchanged.

Rollout plan

Will ship enabled for all users

Requires code in //chrome?

False

Availability expectation

Available by default in Chrome 146 for all desktop platforms. Mobile availability (Android) expected to follow in the same milestone.

Adoption expectation

Expected to be adopted by WebRTC and streaming applications that correlate decoded VideoFrame metadata with RTP timestamps for synchronization and telemetry. This includes browser based streaming clients that use MediaStreamTrackProcessor.

Adoption plan

Ship enabled by default for all users in Chrome 146. Coordinate with developers via WPT test results and the published ChromeStatus entry. No developer facing migration required.

Non-OSS dependencies

Does the feature depend on any code or APIs outside the Chromium open source repository and its open-source dependencies to function?

None. The feature is implemented entirely within Chromium’s open source stack (WebRTC, Blink, and WebCodecs)

Estimated milestones

Shipping on desktop	146
Shipping on Android	146
Shipping on WebView	146

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None expected. This feature implements the rtpTimestamp entry in the WebCodecs VideoFrame Metadata Registry and matches the current registry text (name, type, and semantics). The registry status is “W3C Draft Registry,” but any further edits we expect to be editorial or additive. If a normative change were proposed (e.g., semantics/units' clarification or constraints), we would align Chromium accordingly and update tests.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5186046555586560?gate=5179324311011328

This intent message was generated by Chrome Platform Status.

[Philipp Hancke]

This makes a lot of sense but why did this end up as `.metadata()` when

  https://w3c.github.io/webrtc-encoded-transform/#dom-rtcencodedvideoframe-getmetadata

which it is going to be used in concert with calls it `.getMetadata()`? Guido or Harald might know...

Has this shipped in other browsers or could this still be fixed?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR12MB87961707EEA95FF947A2DBC9A6F3A%40PH7PR12MB8796.namprd12.prod.outlook.com.

[Dale Curtis]

I can't speak to RTCEncodedVideoFrame, but on VideoFrame metadata() matches the style of our other read-only accessors.

- dale

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKiJDF4pNvQ_dvZ2FdC89PyETNhQPXWdeB_JCysiPeNLLOQ%40mail.gmail.com.

[Dale Curtis]

Also, to clarify your "in concert" remark for other readers, the complimentary WebCodecs' class to RTCEncodedVideoFrame is EncodedVideoChunk, not VideoFrame.

- dale

[Chris Harrelson]

LGTM1

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwf81y1vEohWApCuVQRM4Mc9CrH7_MuMdEGWcr8Va0k-jQ%40mail.gmail.com.

[Vladimir Levin]

Hey, I'm curious about the choice of either having rtpTimestamp if it's present in the underlying metadata and not having the field at all if it's not there, as opposed to having the field with some undefined value. Can you comment on this?

I'm not sure if that's a common pattern in this space

Thanks,

Vlad

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR12MB87961707EEA95FF947A2DBC9A6F3A%40PH7PR12MB8796.namprd12.prod.outlook.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Dale Curtis]

I'm not sure how that would work for primitive types without an optional type or requiring a sentinel value specified. Specifying a sentinel value can be prohibitive since some fields may be carried over from legacy specs. Can you elaborate?

See https://source.chromium.org/chromium/chromium/src/+/main:media/base/video_frame_metadata.h for the internal version this is shaped after. Generally the optional type is also useful for constraining memory usage if the carried type is large.

- dale

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR12MB87961707EEA95FF947A2DBC9A6F3A%40PH7PR12MB8796.namprd12.prod.outlook.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Vladimir Levin]

That makes sense thanks.

LGTM2

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/PH7PR12MB87961707EEA95FF947A2DBC9A6F3A%40PH7PR12MB8796.namprd12.prod.outlook.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Alex Russell]

LGTM3