Intent to implement: "image-rendering: pixelated" CSS property-value

[Daniel Holbert]

Summary: The CSS declaration "image-rendering: pixelated" allows authors
to request that we scale up images by effectively making the pixels
larger (using a "nearest-neighbor" algorithm). This is in contrast to
the default (non-pixelated) scaling behavior, which tends to blur the
edges between an image's pixels when upscaling. The default behavior is
appropriate for use-cases like photos, but authors may prefer a
"pixelated" look e.g. when scaling up pixel-art or favicons.

Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=856337

Link to standard:
http://dev.w3.org/csswg/css-images/#valuedef-pixelated

Platform coverage: All

Estimated or target release: Firefox 35 or 36

Preference behind which this will be implemented:
None. This is a small, targeted feature; if we need to disable it for
some reason, we can easily do so with a small change to the CSS parser
(just removing the new keyword from the keyword-table for this property).

NOTES:
- Blink has already implemented "pixelated"[1] and it'll be shipping[2]
in Chrome 38 [3]. So, if & when we ship it, there will be
interoperability between at least 2 engines here. (Other rendering
engines expose similar behavior, albeit under different non-standard
keyword-names.)
- Gecko already has "image-rendering: -moz-crisp-edges" as a way to
request this behavior; the difference is that "-moz-crisp-edges" uses
the same algorithm (nearest-neighbor) regardless of whether it's
upscaling or downscaling, whereas "pixelated" is supposed to *only* use
that algorithm for upscaling, and use the default ("auto") behavior when
downscaling.

~Daniel

[1] Blink intent-to-implement:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/Q8N6FoeoPXI
[2] Blink intent-to-ship:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/zSasd2LL8Mc
[3]
http://blog.chromium.org/2014/08/chrome-38-beta-new-primitives-for-next.html

[Ehsan Akhgari]

On 2014-09-23, 4:53 PM, Daniel Holbert wrote:
> NOTES:
> - Blink has already implemented "pixelated"[1] and it'll be shipping[2]
> in Chrome 38 [3]. So, if & when we ship it, there will be
> interoperability between at least 2 engines here. (Other rendering
> engines expose similar behavior, albeit under different non-standard
> keyword-names.)
> - Gecko already has "image-rendering: -moz-crisp-edges" as a way to
> request this behavior; the difference is that "-moz-crisp-edges" uses
> the same algorithm (nearest-neighbor) regardless of whether it's
> upscaling or downscaling, whereas "pixelated" is supposed to *only* use
> that algorithm for upscaling, and use the default ("auto") behavior when
> downscaling.

Why are upscaling and downscaling treated differently for pixelated?

[Daniel Holbert]

On 09/23/2014 02:16 PM, Ehsan Akhgari wrote:
> Why are upscaling and downscaling treated differently for pixelated?

I'm not entirely sure what the origin of that distinction is, but my
understanding (mostly from reading Tab's comments/responses on the Blink
intent-to-implement thread) is that Nearest-Neighbor Scaling really
doesn't look "pixelated" at all when scaling down, so authors asking for
a "pixelated" look really don't want nearest-neighbor downscaling. The
default scaling algorithm will do a better job of downscaling your image
in a non-datalossy-way than nearest-neighbor would.

~Daniel

[Daniel Holbert]

On 09/23/2014 02:39 PM, Daniel Holbert wrote:
> On 09/23/2014 02:16 PM, Ehsan Akhgari wrote:
>> Why are upscaling and downscaling treated differently for pixelated?
>
> I'm not entirely sure what the origin of that distinction is, but my
> understanding (mostly from reading Tab's comments/responses on the Blink
> intent-to-implement thread) is that Nearest-Neighbor Scaling really
> doesn't look "pixelated" at all when scaling down

FWIW, I also emailed www-style to sanity-check my understanding & to see
if there are any other reasons for this behavior-difference:
http://lists.w3.org/Archives/Public/www-style/2014Sep/0340.html

~Daniel

[Daniel Holbert]

On 09/23/2014 02:56 PM, Daniel Holbert wrote:
> FWIW, I also emailed www-style to sanity-check my understanding & to see
> if there are any other reasons for this behavior-difference:
> http://lists.w3.org/Archives/Public/www-style/2014Sep/0340.html

Turns out there wasn't a strong reason for the difference; Tab's now
updated the ED to remove the requirement that we match "auto" when
downscaling.

This makes the implementation considerably simpler, which is great. It
also means that "pixelated" will essentially just be a
more-interoperable version of "-moz-crisp-edges", for the time being.

(Down the line, we might want to change "crisp-edges" to use a different
scaling algorithm, and then they wouldn't be aliases anymore. The spec
allows us flexibility in choice of algorithm for "crisp-edges" [and
there are other edge-preserving algorithms like hqx that could be
better]. "pixelated" is required to stick with nearest-neighbor, though.)

[Jonas Sicking]

On Tue, Sep 23, 2014 at 4:07 PM, Daniel Holbert <dhol...@mozilla.com> wrote:
> On 09/23/2014 02:56 PM, Daniel Holbert wrote:
>> FWIW, I also emailed www-style to sanity-check my understanding & to see
>> if there are any other reasons for this behavior-difference:
>> http://lists.w3.org/Archives/Public/www-style/2014Sep/0340.html
>
> Turns out there wasn't a strong reason for the difference; Tab's now
> updated the ED to remove the requirement that we match "auto" when
> downscaling.
>
> This makes the implementation considerably simpler, which is great. It
> also means that "pixelated" will essentially just be a
> more-interoperable version of "-moz-crisp-edges", for the time being.

Would it make sense to have separate properties for "scale up" and
"scale down"? With image-rendering being a shorthand for setting both?

Separately, isn't "image-rendering" a bit too generic of a name for
setting scaling strategy?

/ Jonas

[Cameron McCormack]

On 24/09/14 09:24, Jonas Sicking wrote:
> Would it make sense to have separate properties for "scale up" and
> "scale down"? With image-rendering being a shorthand for setting both?
>
> Separately, isn't "image-rendering" a bit too generic of a name for
> setting scaling strategy?

I guess it was chosen because we already have an image-rendering
property from SVG to control how raster images are scaled/rendered
(although it was always more of a hint, rather than this new value which
would be required behaviour).

[Daniel Holbert]

On 09/23/2014 04:24 PM, Jonas Sicking wrote:
> Would it make sense to have separate properties for "scale up" and
> "scale down"? With image-rendering being a shorthand for setting both?

Firstly: per my replies on the subthread started by ehsan, the
distinction in "scale up" vs. "scale down" behavior has (just now!) been
removed from the spec. It turns out that it was a somewhat arbitrary choice.

I suspect that removes the motivation for your question, but just in
case it doesn't: I don't think see a strong use-case for an author to
request specific & different scale-up vs. scale-down behaviors --
particularly given that there are only a few "image-rendering" options
anyway, and only one of them ("pixelated") is specced with
actually-reliable results. (The others are basically
whatever-the-rendering-engine-wants-to-do, under these rough
guidelines.) If an author really wants different behavior for scaling
up vs. scaling down, they can already get their desired result via media
queries and/or JS.

> Separately, isn't "image-rendering" a bit too generic of a name for
> setting scaling strategy?

Perhaps, but that ship has sailed :) This property was originally part
of SVG, and there, I think the "scaling" aspect was just implied. :)
(since it's already in the SVG name)

If it didn't already exist in SVG, and the CSSWG were creating this
property from scratch, they probably would have picked a different name
that included some mention of "scaling"; but here we are.

[Daniel Holbert]

On 09/23/2014 04:38 PM, Daniel Holbert wrote:
> On 09/23/2014 04:24 PM, Jonas Sicking wrote:
>> Would it make sense to have separate properties for "scale up" and
>> "scale down"? With image-rendering being a shorthand for setting both?
>
> Firstly: per my replies on the subthread started by ehsan

(Oh, I guess you were *responding* to that subthread :) I didn't
initially see that, since I hadn't yet bumped this to my threaded
dev.platform folder. Anyway, I don't see a compelling motivation to go
for separate downscaling vs. upscaling prefs separately, at this point.
It's possible the property could be split in the future, though.)

[Justin Dolske]

On 9/23/14 4:24 PM, Jonas Sicking wrote:

>> This makes the implementation considerably simpler, which is great. It
>> also means that "pixelated" will essentially just be a
>> more-interoperable version of "-moz-crisp-edges", for the time being.
>
> Would it make sense to have separate properties for "scale up" and
> "scale down"? With image-rendering being a shorthand for setting both?

Maybe.

We use this property in Firefox's UI, to make favicons look better when
upscaling on hidpi displays... NN upscaling looks better for tiny icons,
because "smooth" algorithms just make them look blurry. I noted in bug
839923 comment 0 that it's not ideal to have the same behavior for
downscaling, and the issue tangentially came up again in bug 1041845.

So there's a plausible use case. OTOH, scaling tiny icons is just
generally not good with either algorithm, so having the capability isn't
really of much use. There are other scaling algorithms explicitly
designed to scale small/pixel-art images much better, but AFAIK no
browser actually implements such a thing.

Justin

[Martin Thomson]

On 2014-09-23, at 13:53, Daniel Holbert <dhol...@mozilla.com> wrote:

> Link to standard:
> http://dev.w3.org/csswg/css-images/#valuedef-pixelated

Reading the spec it doesn’t say anything about what to do when the image is scaled up on one axis and down on the other. It’s probably not a particularly valid use case, but I’d expect there to be at least something on the subject. The image sizing examples in the same document actually demonstrate this exact case, odd as it might seem.

[Daniel Holbert]

On 09/23/2014 10:08 PM, Martin Thomson wrote:
> On 2014-09-23, at 13:53, Daniel Holbert <dhol...@mozilla.com> wrote:
>
>> Link to standard:
>> http://dev.w3.org/csswg/css-images/#valuedef-pixelated
>

> Reading the spec it doesn�t say anything about what to do when the image is scaled up on one axis and down on the other. It�s probably not a particularly valid use case, but I�d expect there to be at least something on the subject. The image sizing examples in the same document actually demonstrate this exact case, odd as it might seem.
>

Yup, good question.

So, three late-breaking updates from ~today that address this:

1) Tab moved this property-value to the CSS Images Level 3 spec in the
last day or so, so I believe the canonical definition is now here:
http://dev.w3.org/csswg/css-images-3/#valdef-image-rendering-pixelated

(This is a *slightly* different URL than the one I gave in my initial
"intent to implement" post -- note the "-3" in "css-images-3". Sorry for
the confusion.)

2) I posted to www-style asking basically your exact question earlier
today (following up on a thread from Simon Sapin), and Tab accepted
Simon's proposal to relax the language & make it only care if "at least
one" axis is being upscaled, and otherwise do the "auto" behavior.

3) Later on today, in response to ehsan's question on this thread, I
asked if there was any strong reason to have the upscaling/downscaling
behavior-difference in the first place; there was not, so the
distinction was removed altogether.

So the current spec text is simply:
# pixelated
# The image must be scaled with the
# "nearest neighbor" or similar algorithm,
# to preserve a "pixelated" look as the image
# changes in size.
from here:
http://dev.w3.org/csswg/css-images-3/#valdef-image-rendering-pixelated

Thanks,
~Daniel

[Daniel Holbert]

On 09/23/2014 01:53 PM, Daniel Holbert wrote:
> Link to standard:
> http://dev.w3.org/csswg/css-images/#valuedef-pixelated

As noted elsethread (in my response to Martin), it looks like the
canonical definition of this property-value is actually in a different
ED -- the "level 3" ED. (whereas the link above is currently the "level
4" ED).

The corrected link is:
http://dev.w3.org/csswg/css-images-3/#valdef-image-rendering-pixelated

(Both versions have spec-text on this property-value, and until this
morning, I believe the spec-text was the same; but there have been a few
small changes in response to my questions on www-style today, and those
changes have happened in the corrected URL for the level-3 spec -- not
the URL I'd originally posted for the level-4 spec.)

Thanks, & sorry for any confusion caused by this.
~Daniel

[Daniel Holbert]

On 09/23/2014 10:30 PM, Daniel Holbert wrote:
> As noted elsethread (in my response to Martin), it looks like the
> canonical definition of this property-value is actually in a different
> ED -- the "level 3" ED. (whereas the link above is currently the "level
> 4" ED).

(This change -- moving the "image-rendering" property between
spec-levels -- only happened yesterday[1] which is why I hadn't noticed
it until after sending my intent-to-implement post.)

[1] http://lists.w3.org/Archives/Public/www-style/2014Sep/0301.html

[Ehsan Akhgari]

On 2014-09-23, 7:07 PM, Daniel Holbert wrote:
> On 09/23/2014 02:56 PM, Daniel Holbert wrote:
>> FWIW, I also emailed www-style to sanity-check my understanding & to see
>> if there are any other reasons for this behavior-difference:
>> http://lists.w3.org/Archives/Public/www-style/2014Sep/0340.html
>
> Turns out there wasn't a strong reason for the difference; Tab's now
> updated the ED to remove the requirement that we match "auto" when
> downscaling.

Great!

> This makes the implementation considerably simpler, which is great. It
> also means that "pixelated" will essentially just be a
> more-interoperable version of "-moz-crisp-edges", for the time being.

So, what are we planning to do with -moz-crisp-edges?

I think keeping it in its current form may be pointless (unless if we
know this is something that the Web depends on?). If I'm reading the
spec correctly, we can actually unprefix it and make it equivalent to
pixelated, but I'm not sure how valuable that is. I think that this is
allowed by the spec though, so perhaps it should be modified to say how
crisp-edges must be different than pixelated.

> (Down the line, we might want to change "crisp-edges" to use a different
> scaling algorithm, and then they wouldn't be aliases anymore. The spec
> allows us flexibility in choice of algorithm for "crisp-edges" [and
> there are other edge-preserving algorithms like hqx that could be
> better]. "pixelated" is required to stick with nearest-neighbor, though.)

But there is nothing in the current spec text requiring the two to be
different, right?

[L. David Baron]

Or, alternatively, it seems like the use case here would be
addressed by doing what the spec said before. Is it really that
much harder to do? Is it just that we'd need to add another value
to pass through various layers of code, and then support that value?
Or is supporting that value actually hard?

(Also, I think 'crisp-edges' in the spec is intended to be
algorithms better than nearest neighbor. I'm hesitant to unprefix
'-moz-crisp-edges' immediately, although we might consider just
removing it.)

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla https://www.mozilla.org/ 𝄂
Before I built a wall I'd ask to know
What I was walling in or walling out,
And to whom I was like to give offense.
- Robert Frost, Mending Wall (1914)

[Daniel Holbert]

On 09/24/2014 09:32 AM, L. David Baron wrote:
> Or, alternatively, it seems like the use case here would be
> addressed by doing what the spec said before. Is it really that
> much harder to do?

No, it's not much harder.

> Is it just that we'd need to add another value to pass through
> various layers of code, and then support that value?

Correct -- we'd need to add a new enum value to GraphicsFilter, here...
http://mxr.mozilla.org/mozilla-central/source/gfx/thebes/GraphicsFilter.h
...and then we'd need to adjust all the code that depends on
GraphicsFilter enum-values to either have an "upscaling or
downscaling?" check (or in some cases, if we can't tell about
upscaling vs. downscaling at that point in the code, we'd need to
ensure that all upstream or downstream code includes such a check).

So: not particularly hard. Just some extra logic / complexity in a
handful of places -- on the order of 10 places, perhaps, based on a
MXR search for GraphicsFilter::FILTER_NEAREST. (That turns up 6
comparisons with "==" and 3 "case" statements, where we'd likely need
to consider the new enum value as well, or convert out of it before we
hit that code.)

If "auto" downscaling actually makes a noticable difference for e.g.
favicons, then I'm fine with adding back the distinction. (I'd be
interested in seeing examples of its benefits, though; that would help
make things more concrete.)

> (Also, I think 'crisp-edges' in the spec is intended to be
> algorithms better than nearest neighbor. I'm hesitant to unprefix
> '-moz-crisp-edges' immediately, although we might consider just
> removing it.)

I agree that we shouldn't unprefix -moz-crisp-edges.

I also don't think we should drop it quite yet, since it's the only
way authors have had to request this behavior (and no browser
release-version uses "pixelated" yet, so authors don't necessarily
know to use that keyword yet either. Though I expect they soon will,
given that this will be the only way to get NN-scaling in Chrome, once
it's released there.)

Maybe after we've shipped "pixelated" for a release, we can make the
call to unprefix or drop/hide -moz-crisp-edges.

[Daniel Holbert]

On 09/24/2014 07:38 AM, Ehsan Akhgari wrote:
>> This makes the implementation considerably simpler, which is great. It
>> also means that "pixelated" will essentially just be a
>> more-interoperable version of "-moz-crisp-edges", for the time being.
>

> So, what are we planning to do with -moz-crisp-edges?

I'm not aware of any specific plans to change it at the moment.

> I think keeping it in its current form may be pointless (unless if we
> know this is something that the Web depends on?)

It's unclear to me whether the web depends on it. As a proxy, we use it
in 16 places within /browser.

Moreover, note that this behavior is *only* available using per-browser
prefixed keywords; so I doubt authors have unprefixed fallback at this
point. So, unprefixing or removing "-moz-crisp-edges" would likely
break content at this point. (I'll bet authors will start including
unprefixed "pixelated" soon, though, because that'll be the only way to
get this behavior in Chrome, once it ships there.)

> If I'm reading the
> spec correctly, we can actually unprefix it and make it equivalent to
> pixelated, but I'm not sure how valuable that is.

We probably do want to eventually unprefix -moz-crisp-edges (and as you
say, we *could* do so now), but if we're planning to tweak which
algorithm we use (unclear), it might be wise to wait until we've done
that before unprefixing.

> I think that this is
> allowed by the spec though, so perhaps it should be modified to say how
> crisp-edges must be different than pixelated.

They don't have to be different. As Tab said on the Blink
intent-to-implement thread:

Having "crisp-edges" act like "pixelated" is an
allowed implementation strategy. It's also allowed,
though, to be smarter when doing "crisp-edges", and
use an intelligent pixel-scaling algorithm, of
which there are many.

"pixelated" was added by request of multiple users, who sometimes
literally want the "big pixel" look of plain nearest-neighbor
interpolation.

https://groups.google.com/a/chromium.org/d/msg/blink-dev/Q8N6FoeoPXI/hoECmv_OUkYJ

[Till Schneidereit]

On Wed, Sep 24, 2014 at 7:21 PM, Daniel Holbert <dhol...@mozilla.com>
wrote:

FWIW, NN interpolation is used by a lot of Flash content for the same
reason, which is why implementing it was pretty high priority in Shumway.
Given that, it's pretty likely that there's quite a bit of interest among
web designers, too.

[Daniel Holbert]

On 09/24/2014 09:32 AM, L. David Baron wrote:
> Or, alternatively, it seems like the use case here would be
> addressed by doing what the spec said before.

Following up more on this: the CSSWG has now resolved to *allow* (but
not require) the formerly-required-by-spec prettier downscaling
behavior, per the first "RESOLVED" at the top of
http://lists.w3.org/Archives/Public/www-style/2014Sep/0384.html

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1072703 to cover
this.

Given that the main use-cases for "image-rendering:pixelated" are for
*upscaling*, I don't think the optional-better-downscaling-work should
block us from shipping a straightforward (& spec-compliant)
nearest-neighbor implementation for "pixelated".

We can add better-downscaling logic separately, in the followup bug --
though it may even arrive in the same release where we ship "pixelated"
(or if not that, soon after), since as noted in my other repsonse to
dbaron, it's not *too* much work.

~Daniel

[Ehsan Akhgari]

On 2014-09-24, 9:12 PM, Daniel Holbert wrote:
> On 09/24/2014 09:32 AM, L. David Baron wrote:
>> Or, alternatively, it seems like the use case here would be
>> addressed by doing what the spec said before.
>
> Following up more on this: the CSSWG has now resolved to *allow* (but
> not require) the formerly-required-by-spec prettier downscaling
> behavior, per the first "RESOLVED" at the top of
> http://lists.w3.org/Archives/Public/www-style/2014Sep/0384.html

Hmm, doesn't that basically allow non-interoperable implementations? :(
I think Jonas' idea on having separate properties for the upscale vs.
downscale cases is much better.

> I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1072703 to cover
> this.
>
> Given that the main use-cases for "image-rendering:pixelated" are for
> *upscaling*, I don't think the optional-better-downscaling-work should
> block us from shipping a straightforward (& spec-compliant)
> nearest-neighbor implementation for "pixelated".
>
> We can add better-downscaling logic separately, in the followup bug --
> though it may even arrive in the same release where we ship "pixelated"
> (or if not that, soon after), since as noted in my other repsonse to
> dbaron, it's not *too* much work.

So, what's Chrome's position with regards to this spec churn? Is what
they're going to ship in Chrome 38 going to be interoperable with our
implementation?

[Daniel Holbert]

On 09/24/2014 06:26 PM, Ehsan Akhgari wrote:
> Hmm, doesn't that basically allow non-interoperable implementations? :(
> I think Jonas' idea on having separate properties for the upscale vs.
> downscale cases is much better.

I'm unconvinced about the usefulness of exposing that much control. This
is something that could be added to CSS Images level 4, though, if you
really thing it merits doing. I suspect CSS Images level 3 is too late
in the game for that sort of change.

> So, what's Chrome's position with regards to this spec churn?

As of this morning, their "intent to ship" thread had a few posts
sounding like they'd take out the smart-downscaling eventually, to align
with the ED as-it-stood-this-morning.

But now that prettier downscaling is allowed, I assume they'll stick
with what they've already implemented (which includes the default
downscaling behavior).

> Is what
> they're going to ship in Chrome 38 going to be interoperable with our
> implementation?

It depends on what you mean by "interoperable". If you're asking if
they'll produce the exact same result, pixel-for-pixel, when downscaling
an image, then no. But that's likely already the case, with the default
scaling behavior; I'd be surprised if we matched them 100% on
image-downscaling.

Also, stepping back a bit w.r.t. the interoperability of the
"image-rendering" property -- this property is explicitly a _hint_, to
express author intent. Its description in the spec starts like this:

# "The image-rendering property provides a hint
# to the user-agent about what aspects of an
# image are most important to preserve when the
# image is scaled, to aid the user-agent in the
# choice of an appropriate scaling algorithm.
http://dev.w3.org/csswg/css-images-3/#propdef-image-rendering

So, it's not required to behave exactly the same everywhere; it simply
codifies an author's intent. (OK, I suppose it *is* required to behave
exactly the same everywhere in the case of "pixelated" & upscaling,
since that requires a particular algorithm, to achieve a particular
effect. But other than that, it's purely a hint.)

So, I don't think pixel-for-pixel-identical is a level of
interoperability that's required or expected for this property, in general.

Also, note that once followup-bug-1072703 is fixed, we'll be as
interoperable with Chrome as our default downscaling behavior is with
theirs. (which is probably pretty close, though I suspect not
pixel-for-pixel identical.)

~Daniel

[Daniel Holbert]

On 09/24/2014 09:23 PM, Daniel Holbert wrote:
> So, it's not required to behave exactly the same everywhere; it simply
> codifies an author's intent. (OK, I suppose it *is* required to behave
> exactly the same everywhere in the case of "pixelated" & upscaling,
> since that requires a particular algorithm, to achieve a particular
> effect. But other than that, it's purely a hint.)

(And actually, even in the case of pixelated & upscaling, the spec just
requires "nearest-neighbor _or similar_". So, [as it goes on to say],
the spec "does not dictate any particular scaling algorithm to be used.")

[Jet Villegas]

Would it be wise to allow for "image-rendering: pixelated" that applies to any scale operation, and give us an option to add other operations (eg. "image-rendering: smooth" or "image-rendering: bilinear") later?

--Jet

_______________________________________________
dev-platform mailing list
dev-pl...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

[Ehsan Akhgari]

On 2014-09-25, 12:23 AM, Daniel Holbert wrote:
>> Is what
>> they're going to ship in Chrome 38 going to be interoperable with our
>> implementation?
>
> It depends on what you mean by "interoperable". If you're asking if
> they'll produce the exact same result, pixel-for-pixel, when downscaling
> an image, then no. But that's likely already the case, with the default
> scaling behavior; I'd be surprised if we matched them 100% on
> image-downscaling.

No, sorry for not being clear, I didn't mean pixel for pixel identical
results. My question was: are we going to have the same behavior for
pixelated in the downscaling case, since now the spec allows two
different behaviors for that case.


Cheers,
Ehsan

[Daniel Holbert]

On 09/25/2014 09:16 AM, Ehsan Akhgari wrote:
> No, sorry for not being clear, I didn't mean pixel for pixel identical
> results. My question was: are we going to have the same behavior for
> pixelated in the downscaling case, since now the spec allows two
> different behaviors for that case.

Gotcha.

Once the followup bug 1072703 is fixed, "yes", we'll have effectively
the same downscaling behavior as Chrome, with
"image-rendering:pixelated". (we'll both match our respective default
downscaling behaviors)

Before that (i.e. if we just take bug 856337), we won't -- we'd do
nearest-neighbor for downscaling, and they'll do their default thing.

I don't see this temporary difference as particularly problematic,
particularly given that "pixelated" is primarily an upscaling feature,
and given that we'll match them before too long. But if others
disagree, I'm open to holding off on shipping "image-rendering:
pixelated" until bug 1072703 is fixed.

(I don't think that waiting on that bug is worthwhile... In the
meantime, authors who want a pixelated look (and want to support
Firefox) are going to have to use "-moz-crisp-edges" instead, which
means prefixed CSS will be propagating on the web, which is undesirable.)

(side note: I'm keen on supporting "pixelated" in the near term because
I need it in a pile of reftests that I'm writing for "object-fit" and
"object-position", which will live in our directory of
reftests-that-get-upstreamed-to-the-w3c. As it stands right now, I'll
have to use "-moz-crisp-edges" in those reftests in order to get
reftestable image-upscaling behavior; but I'd rather use "pixelated" as
a single standardized supported-in-multiple-engines keyword, since these
tests are destined for an upstream non-moz testsuite.)

~Daniel

[James Graham]

On 25/09/14 05:23, Daniel Holbert wrote:

> It depends on what you mean by "interoperable". If you're asking if
> they'll produce the exact same result, pixel-for-pixel, when downscaling
> an image, then no. But that's likely already the case, with the default
> scaling behavior; I'd be surprised if we matched them 100% on
> image-downscaling.
>

> Also, stepping back a bit w.r.t. the interoperability of the
> "image-rendering" property -- this property is explicitly a _hint_, to
> express author intent. Its description in the spec starts like this:

So, are we sure that this is what the spec *should* say? can we imagine
a scenario in which authors either use hacks to specify different
properties for different browsers, or sites look considerably worse in
Gecko than some other browser, so we end up reverse engineering their
behaviour, due to differences in the scaling behaviour? If either of
these seems likely then the spec likely shouldn't claim to be a "hint"
but should actually specify what's needed for interop.

[Ehsan Akhgari]

On 2014-09-25, 12:43 PM, Daniel Holbert wrote:
> On 09/25/2014 09:16 AM, Ehsan Akhgari wrote:
>> No, sorry for not being clear, I didn't mean pixel for pixel identical
>> results. My question was: are we going to have the same behavior for
>> pixelated in the downscaling case, since now the spec allows two
>> different behaviors for that case.
>
> Gotcha.
>
> Once the followup bug 1072703 is fixed, "yes", we'll have effectively
> the same downscaling behavior as Chrome, with
> "image-rendering:pixelated". (we'll both match our respective default
> downscaling behaviors)
>
> Before that (i.e. if we just take bug 856337), we won't -- we'd do
> nearest-neighbor for downscaling, and they'll do their default thing.
>
> I don't see this temporary difference as particularly problematic,
> particularly given that "pixelated" is primarily an upscaling feature,
> and given that we'll match them before too long. But if others
> disagree, I'm open to holding off on shipping "image-rendering:
> pixelated" until bug 1072703 is fixed.

I would really prefer if we ship something interoperable with Chrome, so
unless bug 1072703 is a very large project, I don't think we should ship
support for pixelated without it.

> (I don't think that waiting on that bug is worthwhile... In the
> meantime, authors who want a pixelated look (and want to support
> Firefox) are going to have to use "-moz-crisp-edges" instead, which
> means prefixed CSS will be propagating on the web, which is undesirable.)

Yeah, I see your point, and I agree that none of the alternatives are
clearly ideal, but I think we should put a bit more weight in ensuring
interoperability here. Note that if I'm getting the dates right, we'll
ship this after Chrome does, so early adopters will probably have to use
both pixelated and -moz-crisp-edges together at least for a short while.

Cheers,
Ehsan

[Daniel Holbert]

On 09/25/2014 08:24 AM, James Graham wrote:
> So, are we sure that this is what the spec *should* say? can we imagine
> a scenario in which authors either use hacks to specify different
> properties for different browsers

Bad news: we are already in that world. Right now, if authors want
pixelated upscaling instead of smeared upscaling, we make them do the
following horrible horrible thing:

image-rendering: -moz-crisp-edges; /* Firefox */
image-rendering: -webkit-optimize-contrast; /* Safari */
image-rendering: pixelated; /* Chrome (soon) */
-ms-interpolation-mode: nearest-neighbor; /* IE */

Yikes. Let's make that better.

> or sites look considerably worse in
> Gecko than some other browser, so we end up reverse engineering their
> behaviour, due to differences in the scaling behaviour?

We've had bugs filed like this in the past, and these bugs give us an
opportunity to think about whether we should switch to a different
scaling algorithm for that content/image/platform/etc. If we can,
perhaps we should. This is a good thing.

Note that if the spec *required* a specific algorithm for interop, then
we wouldn't be allowed to improve on a gross rendering in cases like
this (unless we want to violate the spec). I don't think we want to ask
that the spec tie our hands like that.

> If either of
> these seems likely then the spec likely shouldn't claim to be a "hint"
> but should actually specify what's needed for interop.

The CSS and HTML specs *already* don't require any particular algorithm
be used for image-scaling (AFAIK). And that's a good thing, because it
lets us make an educated decision given the capabilities of a device,
the type of image, etc., and make improvements when we realize that
we're making the wrong tradeoff.

I think the "image-rendering" spec strikes a reasonable balance between
allowing authors to specify specific intents, without tying browsers'
hands to lock them into a specific implementation.

(And in any case, this thread is about just adding a single value to our
already-existing "image-rendering" property.)

~Daniel

[Daniel Holbert]

On 09/25/2014 09:10 AM, Jet Villegas wrote:
> Would it be wise to allow for "image-rendering: pixelated"
> that applies to any scale operation, and give us an option
> to add other operations (eg. "image-rendering: smooth" or
> "image-rendering: bilinear") later?

Down the line, we can definitely add support for more "image-rendering"
values (like the ones you mentioned), via the CSSWG, if we discover that
they're needed & useful.

[Daniel Holbert]

On 09/25/2014 10:29 AM, Ehsan Akhgari wrote:
>> I don't see this temporary difference as particularly problematic,
>> particularly given that "pixelated" is primarily an upscaling feature,
>> and given that we'll match them before too long. But if others
>> disagree, I'm open to holding off on shipping "image-rendering:
>> pixelated" until bug 1072703 is fixed.
>
> I would really prefer if we ship something interoperable with Chrome, so
> unless bug 1072703 is a very large project, I don't think we should ship
> support for pixelated without it.

Just to follow up on this:

I'm leaning towards of conservatism here & following ehsan's advice to
not ship "pixelated" until we've got downscaling-detection implemented,
for prettier downscaling with "image-rendering: pixelated". (bug 1072703)

Unfortunately, it turned out that this is a less-trivial project than I
was hoping -- it requires changes to all of our drawing paths, in all of
our per-platform gfx/2d/DrawTarget{$WHATEVER} files. It probably won't
be a ton of code, but it requires a good deal of testing/tweaking, on
every platform, to find all the paths that need adjusting, and to find
the right rects/transforms to inspect in each chunk of drawing code.
(And it likely requires some refactoring in these files, to share these
checks among the various drawing paths.)

So, I'm de-prioritizing work on "pixelated" for now, and I'm focusing on
finishing "object-fit" & "object-position" instead. I hope to circle
back to finish of "pixelated" before too long, though.