Intent to Ship: javascript image decode() api

[Vladimir Levin]

Contact emails

vmp...@chromium.org, dom...@chromium.org

Spec

Spec: https://html.spec.whatwg.org/multipage/embedded-content.html#dom-img-decode

Tag review: https://github.com/w3ctag/design-reviews/issues/182

Summary

This change allows web developers to request to decode an img element. The call to a new HTML <img> element’s decode() function returns a promise, which, when fulfilled, ensures that the image can be appended to the DOM without causing a decoding delay on the next frame.

Sample code:

var img = new Image();

img.src = "nebula.jpg";

img.decode().then(function() {

document.getElementById("container").appendChild(img);

});

The call to decode informs the browser that this image needs to be prepared into a form that allows for quick rasterization. This means that the image will be fully loaded and decoded so that sampling out of the image for the purposes of rasterization does not require any further decoding.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/d/topic/blink-dev/LMV9dBG_1-I/discussion

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Demo link

You can see a demo of the change here:

https://drive.google.com/file/d/0B5cKk7MxQ2LsUUotR3RuWUlpSXc/view?usp=sharing

The first page, sync.html, uses an onload handler, which when fired will append the image to the DOM. This causes a noticeable delay in the requestAnimationFrame-driven animation of the spinning arrow.

The second page, async.html, uses decode() functionality (similar to the sample code above), which when resolved will append the image to the DOM. Note a significant reduction in the delay during the raf animation of the spinning arrow.

The code used in this demo is here:

https://drive.google.com/drive/folders/0B5cKk7MxQ2LsNVQzaVVoWHRnVHc

Interoperability and Compatibility Risk

This is a new stand-alone feature, in that it doesn’t have any interactions with existing features (aside from img elements). This means that there is minimal risk to the existing platform. This feature also has interest and support from WebKit, who initiated the discussion.

Risk of removing the feature in the future: If the web developers feature-detect this API, it poses no risk if it were to be removed. If the feature is used without detection then it poses a risk of breaking existing usages if removed. This can be mitigated by replacing the API with no-ops. Since this is primarily an optimization API, it doesn’t add any tools for creating effects or visual changes, so its removal would be straightforward.

Edge: No signals

Firefox: No signals

Safari: Public support (initial proposal: https://github.com/whatwg/html/issues/1920)

Web developers: Positive (AMP team indicated interest in using this)

Is this feature fully tested by web-platform-tests?

Yes, https://github.com/w3c/web-platform-tests/tree/master/image-decodes

(Bug for moving them to the correct directory: https://github.com/w3c/web-platform-tests/issues/6441)

OWP launch tracking bug

crbug.com/705669

Entry on the feature dashboard

https://www.chromestatus.com/feature/5637156160667648

[PhistucK]

Can you poke Edge and Firefox about it? Chrome will be alone in shipping this for the foreseeable future (even if WebKit implements it now, Safari 11.0 will probably not include it).

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADsXd2P9SAwdAr29zfa1hSZ6J%2BC-1oOVRbJtn3dUTUcH-rGCFQ%40mail.gmail.com.

[Boris Zbarsky]

On 7/19/17 6:02 PM, Vladimir Levin wrote:
> Spec:
> https://html.spec.whatwg.org/multipage/embedded-content.html#dom-img-decode

Is the API being added to SVGImageElement too, or just HTMLImageElement?

-Boris

[Vladimir Levin]

> Can you poke Edge and Firefox about it? Chrome will be alone in shipping this for the foreseeable future (even if WebKit implements it now, Safari 11.0 will probably not include it).

Will do.

 

> Is the API being added to SVGImageElement too, or just HTMLImageElement?

This is being added to the img tag. You could specify an svg image as the src for the img and then use the decode api. This doesn't affect the svg tag though.  

[Boris Zbarsky]

On 7/20/17 12:43 PM, Vladimir Levin wrote:
> This doesn't affect the svg tag though.

Should it? I thought there was general agreement on trying to keep the
API surfaces in sync here.

-Boris

[PhistucK]

I think Boris is referring to <svg><image src="will-this-image-element-support-decode()?"><svg>, not to <img src="image.svg">.

☆PhistucK

[mk...@chromium.org]

I agree with Boris' suggestion that we ought to keep `SVGImageElement` and `HTMLImageElement` aligned unless there's a good reason not to. Is there a good reason not to?

Skimming through the tests, it looks like y'all are already covering the "SVG image inside an HTML `<img>` tag" case. Will it be possible to extend the implementation to cover the "image inside an SVG" case as well?

-mike

[Vladimir Levin]

On Tue, Jul 25, 2017 at 4:19 AM, <mk...@chromium.org> wrote:

I agree with Boris' suggestion that we ought to keep `SVGImageElement` and `HTMLImageElement` aligned unless there's a good reason not to. Is there a good reason not to?

Skimming through the tests, it looks like y'all are already covering the "SVG image inside an HTML `<img>` tag" case. Will it be possible to extend the implementation to cover the "image inside an SVG" case as well?

I agree that it sounds like a good idea not to diverge here.  From the implementation perspective, it should be straight forward to implement this.

-mike

On Thursday, July 20, 2017 at 7:00:57 PM UTC+2, PhistucK wrote:

I think Boris is referring to <svg><image src="will-this-image-element-support-decode()?"><svg>, not to <img src="image.svg">.

☆PhistucK

On Thu, Jul 20, 2017 at 7:43 PM, Vladimir Levin <vmp...@chromium.org> wrote:

> Can you poke Edge and Firefox about it? Chrome will be alone in shipping this for the foreseeable future (even if WebKit implements it now, Safari 11.0 will probably not include it).

Will do.

 

> Is the API being added to SVGImageElement too, or just HTMLImageElement?

This is being added to the img tag. You could specify an svg image as the src for the img and then use the decode api. This doesn't affect the svg tag though.  

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f63c6d96-66c1-4805-8dc6-3cc7db9bbc4e%40chromium.org.

[PhistucK]

The specification details all of those cases -

https://html.spec.whatwg.org/multipage/embedded-content.html#dom-img-decode

☆PhistucK

On Fri, Aug 4, 2017 at 11:53 AM, Jacob G <kurte...@gmail.com> wrote:

What happens when calling .decode on a broken image?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1c70c2de-0b61-4e3f-a35d-bda90563eb13%40chromium.org.

[Philip Jägenstedt]

Looks like this has fallen through the cracks and stayed there for a while.

I've looked over the spec and the TAG feedback and think this makes sense. That it might not be needed in a perfectly-layered system isn't IMHO reason enough to hold back, and if the underlying layers are exposed it seems plausible that this will be straightforwardly explained by them.

I think what remains to ship this is:

• A plan for SVGImageElement that's solid enough that we can go ahead and just ship that together with HTMLImageElement.
• Impl bugs for Gecko, Edge and WebKit, if not already filed (couldn't find any)

It was delightful to see the tests in wpt already :)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1c70c2de-0b61-4e3f-a35d-bda90563eb13%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_Kz46n-PExSP9c39GQf3aRQOHc2UQCzjUXi%3DCZuFxGSLA%40mail.gmail.com.

[Vladimir Levin]

On Mon, Aug 14, 2017 at 4:56 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

Looks like this has fallen through the cracks and stayed there for a while.

I've looked over the spec and the TAG feedback and think this makes sense. That it might not be needed in a perfectly-layered system isn't IMHO reason enough to hold back, and if the underlying layers are exposed it seems plausible that this will be straightforwardly explained by them.

I think what remains to ship this is:

• A plan for SVGImageElement that's solid enough that we can go ahead and just ship that together with HTMLImageElement.
• Impl bugs for Gecko, Edge and WebKit, if not already filed (couldn't find any)

As an update for SVGImageElement, when implementing a support for this (behind the same flag as HTMLImageElement), we ran into a spec issue in that current wording conflicts with the intent of the feature. We're working on resolving this. I'll update this thread when we make progress on that front. 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1c70c2de-0b61-4e3f-a35d-bda90563eb13%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_Kz46n-PExSP9c39GQf3aRQOHc2UQCzjUXi%3DCZuFxGSLA%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYczYRbw1HpkGvemtJdcyfFG9aX4Y70HrjWLzazuG__Nzw%40mail.gmail.com.

[Vladimir Levin]

As an update for SVGImageElement, when implementing a support for this (behind the same flag as HTMLImageElement), we ran into a spec issue in that current wording conflicts with the intent of the feature. We're working on resolving this. I'll update this thread when we make progress on that front. 

Update:

- We've added the same functionality to SVGImageElement along with corresponding web platform tests.

- Note that WebKit implemented this feature in https://trac.webkit.org/changeset/221805/webkit.

[Chris Harrelson]

On Thu, Sep 28, 2017 at 2:57 PM, Vladimir Levin <vmp...@chromium.org> wrote:

As an update for SVGImageElement, when implementing a support for this (behind the same flag as HTMLImageElement), we ran into a spec issue in that current wording conflicts with the intent of the feature. We're working on resolving this. I'll update this thread when we make progress on that front. 

Update:

- We've added the same functionality to SVGImageElement along with corresponding web platform tests.

- Note that WebKit implemented this feature in https://trac.webkit.org/changeset/221805/webkit.

Another update: Safari also added the "async" attribute to images, which is currently un-speced. Discussion of that is here.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADsXd2OnaKaXzn3s80%2BUw89gxOqQ1N7r2M18gZR3vPik1YnhLg%40mail.gmail.com.

[Philip Jägenstedt]

On Fri, Sep 29, 2017 at 12:09 AM Chris Harrelson <chri...@chromium.org> wrote:

On Thu, Sep 28, 2017 at 2:57 PM, Vladimir Levin <vmp...@chromium.org> wrote:

As an update for SVGImageElement, when implementing a support for this (behind the same flag as HTMLImageElement), we ran into a spec issue in that current wording conflicts with the intent of the feature. We're working on resolving this. I'll update this thread when we make progress on that front. 

Update:

- We've added the same functionality to SVGImageElement along with corresponding web platform tests.

- Note that WebKit implemented this feature in https://trac.webkit.org/changeset/221805/webkit.

Another update: Safari also added the "async" attribute to images, which is currently un-speced. Discussion of that is here.

Shall we consider this intent revived, or are the more things to be done first? 

[Chris Harrelson]

No, I just mentioned it to give extra information about a related feature.

LGTM1 to ship the image decode API. 

[Philip Jägenstedt]

Well then, LGTM2! Always a please to see tests swish by.

http://wpt.fyi/html/semantics/embedded-content/the-img-element/decode should hopefully be more green in ~2 weeks.

[Mike West]

LGTM3.

-mike

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYftPMRXOiufPfQMpxdWdBhQCTmSCNGwPrCEjxpawo%2BxJA%40mail.gmail.com.