Intent to Implement and Ship: bilinear interpolation for upscaled images

[Philip Rogers]

Contact emails

vmp...@chromium.org, eri...@chromium.org, p...@chromium.org

Spec

This is left to the user agent in https://drafts.csswg.org/css-images-3/#the-image-rendering

Summary

We'd like to switch the default image upscaling algorithm from bicubic to bilinear.

Motivation

Switching to bilinear interpolation for upscales will be faster, use less memory, and match other browsers. The memory savings come from an implementation detail of how we cache upscaled images: the bilinear codepath caches only the original image whereas the bicubic codepath caches the upscaled image. In addition, by unifying our image pipeline to use bilinear everywhere, we will be able to remove the ImageQualityController which is responsible for the visual "pop" as an animating image changes quality (https://crbug.com/503040).

Here is an example of the difference on https://pr.gg/mandrill.jpg upscaled 2x:

Bicubic:  Bilinear: 
​

Here's a more extreme example at 7x:

Bicubic:  Bilinear: 

Interoperability and Compatibility Risk

I tested Safari/WebKit, Firefox/Gecko, and Edge, and found they all appear to use bilinear for upscales. For posterity, I've archived these tests at https://docs.google.com/document/d/1tYMrIcvNP1H0eecO-fc0-km9DxlVlIAvAA90_mXyiJ8/.

Ongoing technical constraints

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

This change will only affect desktop platforms. Android is already special-cased to use bilinear interpolation.

OWP launch tracking bug

https://crbug.com/735174

Link to entry on the feature dashboard

No entry, this is a relatively small change.

Requesting approval to ship?

Yes.

[Philip Jägenstedt]

LGTM1, and nice examples!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJgFLLvgoN-WvRwGsDkVF-FvEWLsW8JGwvXBA0a1gw58kcqXJg%40mail.gmail.com.

[Mike West]

LGTM2. It doesn't look like this has much web-visible impact at all (though I suppose it might change pixels rendered into a canvas, I can't imagine developers relying on that behavior). As long as y'all are happy with the user-visible results of the bilinear algorithm, the memory savings seem like a very clear reason to ship.

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdikfDdEULZSreW0nJ8Q4Y%3Dzgf1Cnc%3DghLY-fv5xGfuTg%40mail.gmail.com.

[Nico Weber]

This seems more like a product decision than a web platform thing. If you haven't, I suggest getting input from UI folks on this as well.

--

[Chris Harrelson]

LGTM3

Regarding feedback from experts: there was a side conversation leading up to this intent with a number of GPU and Skia experts. One reason for this being an intent rather than product decision is in order to provide an FYI and opportunity for even wider feedback.

Developer feedback over time has also been very mixed - some really don't like the current behavior, and some do. The balance is not obviously in favor of bicubic, perhaps the opposite. The reason for mixed feedback is that bicubic works better for some types of images, whereas bilinear works better for others. I also think matching behavior on on other browsers is an important determining factor, because it reduces developer and user frustration due to inexplicable differences between browsers.

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMGbLiGxxdinED-kg%3DRtk1J%2BAejevM7egaVuPjZ2boAA0C0o_w%40mail.gmail.com.

[Philip Rogers]

Thanks everyone! We will go ahead with this change today.

This is partly a product decision so I ran this by some of our UI/UX designers. They pointed out that favicon scaling could be affected, but Vlad and I confirmed favicons would not be affected by this change.

[Brett Wilson]

I assume downsampling is unaffected? What is used for downsampling these days?

Brett

--

[Vladimir Levin]

On Fri, Jul 7, 2017 at 1:40 PM, Brett Wilson <bre...@chromium.org> wrote:

I assume downsampling is unaffected? What is used for downsampling these days?

You're right that downsampling is unaffected in this case. For downsampling we use mipmaps to do the scaling. Specifically, in software rasterization, we get the mip level at or above desired scale and then use bilinear interpolation. For gpu rasterization, we generate the full mip chain and use trilinear interpolation. There are some more special cases when it comes to gpu rasterization on android, in which case we may use bilinear interpolation similar to software rasterization.

- Vlad

[Brett Wilson]

On Fri, Jul 7, 2017 at 2:25 PM, Vladimir Levin <vmp...@google.com> wrote:

On Fri, Jul 7, 2017 at 1:40 PM, Brett Wilson <bre...@chromium.org> wrote:

I assume downsampling is unaffected? What is used for downsampling these days?

You're right that downsampling is unaffected in this case. For downsampling we use mipmaps to do the scaling. Specifically, in software rasterization, we get the mip level at or above desired scale and then use bilinear interpolation. For gpu rasterization, we generate the full mip chain and use trilinear interpolation. There are some more special cases when it comes to gpu rasterization on android, in which case we may use bilinear interpolation similar to software rasterization.

Thanks!

Brett