Intent to Prototype: ImageDecoder API extension for WebCodecs

[Dale Curtis]

Contact emails dalec...@chromium.org Explainer https://github.com/dalecurtis/image-decoder-api/blob/master/explainer.md Design docs/spec TBD TAG review https://github.com/w3ctag/design-reviews/issues/433#issuecomment-617557134 https://github.com/WICG/web-codecs/issues/50 https://github.com/w3ctag/tracking-issues/issues/12   Summary The ImageDecoder API intends to provide web authors random access to decoded images (as ImageBitmap) for each frame in a given image. The returned ImageBitmap objects can be used for drawing to canvas or WebGL (as well as any other future ImageBitmap use cases). Since the API is not bound to the DOM it may also be used in workers. Motivation Today <img> elements don’t provide access to any frames beyond the first. They also provide no control over which frame is displayed in an animation. Without using a WASM/JS based decoders or message passing from a DOM accessible area, workers have no ability to decode images. As we look to provide audio and video codecs through WebCodecs we intend to provide interfaces for images as well. Risks

Interoperability and Compatibility The supported image codecs between browsers is not standardized and this API makes no attempt at doing so. Given how prevalent GIF, JPG, and PNG are we expect overlap in some formats between all browsers though. Firefox: No public signals Through WebCodecs, expressed unofficial, but positive sentiment. Edge: No public signals Safari: No public signals Web developers: Positive (https://discourse.wicg.io/t/proposal-imagedecoder-api-extension-for-webcodecs/4418) Ergonomics Relies on ImageBitmap and ImageBitmapOptions support. Activation WASM polyfills could be provided. Security No new security risks. Intends to reuse the same image decoding infrastructure as used by <img> without any additional new surface area.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)? Yes. Reuses existing internal image decoders already available.

Is this feature fully tested by web-platform-tests? No, TBD Link to entry on the Chrome Platform Status https://www.chromestatus.com/feature/4561928577875968

This intent message was generated by Chrome Platform Status.

[Stephen Mcgruer]

Hey Dale,

> Is this feature fully tested by web-platform-tests? > No, TBD

When you say 'TBD', are there any testing needs this I2P has that WPT doesn't provide today? (Just checking in case there's support you need!)

I skimmed the explainer and it seemed like it could be tested by writing to a canvas and then either doing a ref-test against an expected image or just reading the bits from the canvas and checking they are as expected, but perhaps I have overlooked something :).

Thanks,

Stephen

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAPUDrwdEr0FX9MhBYu2Tjixh-aC1G_swLZh4coJWoA3N%3D2vjLw%40mail.gmail.com.

[Dale Curtis]

On Thu, Apr 23, 2020 at 12:13 PM Stephen Mcgruer <smcg...@chromium.org> wrote:

Hey Dale,

> Is this feature fully tested by web-platform-tests? > No, TBD

When you say 'TBD', are there any testing needs this I2P has that WPT doesn't provide today? (Just checking in case there's support you need!)

Nope, just that I haven't written any yet! Thanks for checking.

 

I skimmed the explainer and it seemed like it could be tested by writing to a canvas and then either doing a ref-test against an expected image or just reading the bits from the canvas and checking they are as expected, but perhaps I have overlooked something :).

Yes, that's what I was thinking. Probably using a couple of encodes from different types of images. For animated ones the frame color would change each time so it could be sample tested in canvas.

 

[Dale Curtis]

This has landed in 84.0.4130.0 on the canary channel behind --enable-blink-features=WebCodecs for those interested in testing it out.

- dale

[Thomas Steiner]

I've been having a lot of fun with this: https://twitter.com/ChromiumDev/status/1255870582442405889.