Intent to Experiment: WebGPU WebCodecs integration

[Corentin Wallez]

Hey all,

This is an I2E for the WebGPU / WebCodecs integration point that is not part of the WebGPU v1 shipment as the WebGPU W3C group decided to remove it from the V1 milestone. However it is a critical integration point for the video applications looking to use WebGPU as part of their video processing pipeline. So we would like to start a second original trial, specifically for this feature, that developers can use to keep prototyping WebGPU video processing after WebGPU v1 is shipped. Because of the weird nature of this trial, I didn't know how to fill all the fields in ChromeStatus, so let me know if more details are needed!

Contact emails

cwa...@google.com, kai...@google.com, baj...@google.com

Explainer

https://gpuweb.github.io/gpuweb/explainer/#image-input
https://github.com/gpuweb/gpuweb/issues/1380

Specification

https://gpuweb.github.io/gpuweb/#gpuexternaltexture

Design docs

https://github.com/gpuweb/gpuweb/issues/1380

Summary

WebGPU exposes an API to create opaque "external texture" objects from HTMLVideoElement. These object can be used to sample the video frames efficiently, potentially in a 0-copy way directly from the source YUV data. However the WebGPU specification for the first version of WebGPU does not allow creating GPUExternalTextures from WebCodecs VideoFrame objects. This capability is important for advanced video processing applications that are already using WebCodecs and would like to integrate WebGPU in the video processing pipeline. This features adds support for using a VideoFrame as the source for a GPUExternalTexture.

Blink component

Blink>WebGPU

TAG review

TAG review status

Pending

Risks

Interoperability and Compatibility

Gecko: No signal WebCodecs is listed as "worth prototyping" which likely means this intergration is the same.

WebKit: No signal WebCodecs is prototyped in Safari TP, so this integration is likely interesting.

Web developers: Positive

Other signals:

Ergonomics

No ergonomic risk. This API would be used at the intersection of WebGPU and WebCodec. It is designed to keep performance as high as possible by allowing o-copy sampling of YUV frame data.

Security

The lifetime management of VideoFrame was taken into account of this feature. No other security considerations.

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Goals for experimentation

Reason this experiment is being extended

Ongoing technical constraints

Debuggability

No support.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes

DevTrial instructions

https://github.com/gpuweb/gpuweb/issues/1380

Flag name

Requires code in //chrome?

False

Estimated milestones

No milestones specified

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5078348864159744

[Ken Russell]

To clarify the signals from Gecko and WebKit, "No signal" should be considered as removed from both explanations.

Hoping the Blink API owners approve this experiment - it's crucial in order to achieve key customer use cases.

Thanks,

-Ken

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGdfWNPH6Jk-g%2B3Nkx0F_rmn05kWcWU%3Dd3cOJNpjfZ3nAWjXcQ%40mail.gmail.com.

[Yoav Weiss]

What milestones are you planning to experiment in?

 

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5078348864159744

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGdfWNPH6Jk-g%2B3Nkx0F_rmn05kWcWU%3Dd3cOJNpjfZ3nAWjXcQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMYvS2dh42%2B0dmweBBnR8txaP-Agua4XgR6%3Dmk-1R1dDnBsB8Q%40mail.gmail.com.

[Corentin Wallez]

Whoops, forgot to fill this. It would be M114-M119 (6 milestones). I'm reasonably hopeful that we can agree in the WebGPU group about this simple integration in that time frame.

[Yoav Weiss]

LGTM to experiment M114-M119 (inclusive)

[Corentin Wallez]

Thank you!

[Corentin Wallez]

Sorry for the back and forth, I got confused with milestones, instead we'd like to experiment between M113 and M118 (inclusive). Does this still look good to you?

[Yoav Weiss]

Yup, still LGTM