New RGB->YUV420 'smart' conversion

[Pascal Massimino]

[forking the discussion]

Hi,

On Sun, Aug 17, 2014 at 3:31 AM, pdknsk <pdk...@gmail.com> wrote:

For this image, improvement is massive. It actually looks better than 2x1 JPEG IMO, and at smaller file size. I tried a few images, and results are similar. It's interesting though that PSNR barely registers this improvement. Someone needs to invent a new metric.

Actually, SSIM *also* goes down sometimes, even if the result is less blurry for instance.

Metric all have their limitation, for sure.

(note: SSIM/PSNR must be computed in RGB space, not YUV, of course! This means with a full end-to-end chain of compress-decode-measure working with RGB. The PSNR or SSIM numbers printed by cwebp is useless here)

 

It does increase file size by some percent, but it's a good trade. With one image though, file size increases ~40% (and more with decreasing quality setting). It still has 2x1 JPEG beat, but with varying margin. It's attached.

yes, this algorithm *adds* signal to the YUV source (compared to the regular converter).

This means that the YUV source passed to the core codec, once processed, is indeed harder to compress because we've added extra details.

 

In another image I noticed an anomaly at low resolution. It's the arm of the red pikmin on the right. And the lower half of the antenna of the lobster wobbles slightly. Also attached.

Thanks for testing! I'll have a look at these images...

 

PS. Google should trumpet this improvement on all channels when it's officially released.

Oh, that's a good suggestion indeed.

 

[Pascal Massimino]

So i had a look at the images.

d3.png:

This is a tricky one! The source has a heavy underlying dithering pattern (see d3_source_zoomed.webp, the cropped source i used for testing is d3_source.webp).

The 'regular' compression blurs these details (see d3_normal.webp). The 'smart' one preserves this dithering pattern much better, but conversely uses a lot

of bits doing so (see d3_smart.webp).

Note: funnily, this one is these cases where the SSIM and PSNR goes down, that i was alluding to before:

==== NORMAL ====

DSSIM: 3.515385

R, G & B SSIM indices: 0.890983, 0.884298, 0.715536

Average SSIM: 0.830272

Max Difference:59 PSNR:31.373dB

==== NEW SMART ALGO ====

DSSIM: 3.023406

R, G & B SSIM indices: 0.724523, 0.669247, 0.854130

Average SSIM: 0.749300

Max Difference:39 PSNR:31.074dB

p2.jpg:

 there's a little distortion indeed, but it's within the normal expectation of lossy compression i'd say (i tried at default quality).

The source is quite compressed, so some care need to be had when *decompressing* the source. For instance, not using

the fancy upsampling in libjpeg6b during decompression of the source JPG would lead to very visible jagged red edges even

before libwebp starts compressing.

[Pascal Massimino]

... and i forgot the attachments. Here they are.

[pdknsk]

On d3, I cannot quite decide if the average user would care enough for the fine dots to have 40% file size (or load time) increase. I'm not sure. Of course the problem is still noticeable elsewhere, like the red stripes running diagonally through the image.

On p2, I can agree on the antenna, where the difference isn't immediately noticeable. Same with the mouth of the blue pikmin, which shifts a pixel or two. JPEG does this too at 2x2. On the arm however, the encoder literally cuts through it. I've re-attached the image uncompressed, resized from a 1998^2 1x1 JPEG. The problem is a bit more pronounced at 345^2 (previously: 350).

[Pascal Massimino]

On Mon, Aug 18, 2014 at 11:58 AM, Pascal Massimino <pascal.m...@gmail.com> wrote:

On Mon, Aug 18, 2014 at 11:57 AM, Pascal Massimino <pascal.m...@gmail.com> wrote:

On Mon, Aug 18, 2014 at 11:30 AM, Pascal Massimino <pascal.m...@gmail.com> wrote:

[forking the discussion]

Hi,

On Sun, Aug 17, 2014 at 3:31 AM, pdknsk <pdk...@gmail.com> wrote:

For this image, improvement is massive. It actually looks better than 2x1 JPEG IMO, and at smaller file size. I tried a few images, and results are similar. It's interesting though that PSNR barely registers this improvement. Someone needs to invent a new metric.

Actually, SSIM *also* goes down sometimes, even if the result is less blurry for instance.

Metric all have their limitation, for sure.

(note: SSIM/PSNR must be computed in RGB space, not YUV, of course! This means with a full end-to-end chain of compress-decode-measure working with RGB. The PSNR or SSIM numbers printed by cwebp is useless here)

 

It does increase file size by some percent, but it's a good trade. With one image though, file size increases ~40% (and more with decreasing quality setting). It still has 2x1 JPEG beat, but with varying margin. It's attached.

yes, this algorithm *adds* signal to the YUV source (compared to the regular converter).

This means that the YUV source passed to the core codec, once processed, is indeed harder to compress because we've added extra details.

 

In another image I noticed an anomaly at low resolution. It's the arm of the red pikmin on the right. And the lower half of the antenna of the lobster wobbles slightly. Also attached.

Thanks for testing! I'll have a look at these images...

Incidentally, this new conversion algo fixed the old issue #98.

I attach the before/after webp files for reference (-q 60).

[pdknsk]

> Of course the problem is still noticeable elsewhere, like the red stripes running diagonally through the image.

By still I mean also.

[Pascal Massimino]

Hi,

On Mon, Aug 18, 2014 at 6:33 PM, pdknsk <pdk...@gmail.com> wrote:

On d3, I cannot quite decide if the average user would care enough for the fine dots to have 40% file size (or load time) increase. I'm not sure. Of course the problem is still noticeable elsewhere, like the red stripes running diagonally through the image.

well, globally the new conversion makes files larger since it generates extra better-preserved details to compress.

This example is really an extreme case, hard to decide from codec standpoint. I'd say the deliberate removal of the

dithering pattern should be left as pre-processing to the user or app.

Beside, the red diagonal streaks are indeed giving hard time to the compression. The artifacts get less visible above -q 80.

I think this needs a separate fix than better conversion. Not sure. The underlying dithering pattern must causing trouble,

even when not very visible.

On p2, I can agree on the antenna, where the difference isn't immediately noticeable. Same with the mouth of the blue pikmin, which shifts a pixel or two. JPEG does this too at 2x2. On the arm however, the encoder literally cuts through it. I've re-attached the image uncompressed, resized from a 1998^2 1x1 JPEG. The problem is a bit more pronounced at 345^2 (previously: 350).

So it turns out the arm is approximately 2 pixels wide, with each pixel sitting across different 2x2 block.

The U/V conversion then averages them separately, unfortunately. A larger averaging window might help, not sure.

The surprise comes when you crop out exactly 1 pixel on the top of the source image: then the arm's two pixels end up

on the same 2x2 block and averaging works better.

See p2-2.webp attached, which is a compressed version of p2.png with just the top pixel row removed. The arm is back!

See p2-shifted.webp and p2-not-shifted.webp, which are the Y/U/V output (using -pgm) for the cropped region above the arm.

Probably need a better gradient-aware U/V subsampling filter.

thanks for spotting these! They'll get in my TODO list.

/skal

[pdknsk]

If I had my way images would use 1x1 (or this new good approximation by WebP), but the average user may not agree. I guess it's nigh impossible to have a smart heuristic that decides this. Perhaps there should be a quality threshold. For quality 20 you may not want the extra bits, but then again at quality 20 it may not weigh in much, I haven't tried. I notice that JPEG XR as default uses 1x1 for quality > 80 and 2x2 below. Photoshop 1x1 for JPEG quality > 50, and 2x2 below. Imagemagick 1x1 for JPEG quality >= 90, and 2x2 below. The last two are not directly comparable though. Imagemagick reports Photoshop quality > 50 JPEGs as quality > 82. The problem with this, other than deciding what may be a good threshold, is a sudden non-linear jump in file size and quality at the threshold.

[Brendan Bolles]

On Aug 19, 2014, at 1:47 PM, Pascal Massimino wrote:

> thanks for spotting these! They'll get in my TODO list.

Any updates on this? Is '-pre 4' still required to use the smart conversion?


Brendan

[Pascal Massimino]

Hi Brendan,

If you are referring to the p2-2.webp problem (fine details being converted-out), then 

yes i had a look. It turned out the problem could be mitigated with techniques similar

to Font Hinting [1] or Thirion's demons algorithm [2]. But also something simpler as

just selective smoothing-filter + sharpening.

Turned out, these could somehow mitigate the problem on your sample, but were

overall making things worse when it was not mandated. So the threshold was not

easy to decide. That's pretty much were i left the experiment.

If you are referring to -pre 4 'smart' RGB->YUV conversion, i didn't manage to

make it fast enough to make it suitable for default, sorry. So it's still under -pre 4,

but you can safely use this option as we'll keep it for compatibility anyway, even if

-pre 4 gets being the default behaviour.

hope it helps,

/skal

[1] http://en.wikipedia.org/wiki/Font_hinting

[2] see for instance: http://people.rit.edu/ndcsma/pubs/MICCAI_Sept_2009.pdf

Brendan

--
You received this message because you are subscribed to the Google Groups "WebP Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webp-discuss...@webmproject.org.
To post to this group, send email to webp-d...@webmproject.org.
Visit this group at http://groups.google.com/a/webmproject.org/group/webp-discuss/.
For more options, visit https://groups.google.com/a/webmproject.org/d/optout.

[pdknsk]

If you are referring to -pre 4 'smart' RGB->YUV conversion, i didn't manage to

make it fast enough to make it suitable for default, sorry. So it's still under -pre 4,

but you can safely use this option as we'll keep it for compatibility anyway, even if

-pre 4 gets being the default behaviour.

I have not run any benchmarks, but I haven't noticed encoding to be particularly slow. (I guess you automatically think at Google scale, converting millions of images at once.)

[Pascal Massimino]

Hi,

the slowdown is only for the RGBA->YUVA conversion. Afterward, the encoding time properly is a little slower (because there's usually

more details to compress and the output is hence a little larger), but not by much.

Also: i forgot to mention that taking alpha into account correctly (similarly to patch https://gerrit.chromium.org/gerrit/#/c/71555/) is a little involved

and isn't finished either.

[pdknsk]

I've run some limited benchmarks and pre4 about doubles encoding time, which is not too bad IMO. I hope Google isn't discouraged by this. The new conversion could really propel WebP into higher (or let's be honest, any) adoption (outside of Google).

[pdk...@gmail.com]

Unfortunately this didn't make it into general use. I guess the edge cases were too troublesome, or Google is preparing to switch WebP to VP9.

Personally I use WebP with pre4 for >= 2x images, to mask the imperfections introduced by edge cases. It works very well when used with <picture> and I very much recommend it.

[Pascal Massimino]

Hi,

On Sun, Nov 29, 2015 at 9:57 AM, <pdk...@gmail.com> wrote:

Unfortunately this didn't make it into general use. I guess the edge cases were too troublesome, or Google is preparing to switch WebP to VP9.

It's still there, and there's no plan to remove it (far from it!). It's just too slow to be the default, even with the last optimisations.

Btw, it definitely should be better named than '-pre 4'. Any suggestion?

Personally I use WebP with pre4 for >= 2x images, to mask the imperfections introduced by edge cases. It works very well when used with <picture> and I very much recommend it.

Good to know! what do you mean, exactly, by ">= 2x images"? Reduced more than 2x?

 

skal/

(btw: VP9 or VP10 also uses YCbCr colorspace, so this algo still apply there. It could even be adapted to jpeg's YUVj, probably).

[pdk...@gmail.com]

Good to know! what do you mean, exactly, by ">= 2x images"? Reduced more than 2x?

I meant high-dpi images.

http://www.webkit.org/demos/srcset/

PS. It still applies to VP9+ but not really, because they have 4:4:4 support.

[Pascal Massimino]

Hi,

On Mon, Nov 30, 2015 at 12:49 AM, Pascal Massimino <pascal.m...@gmail.com> wrote:

Hi,

On Sun, Nov 29, 2015 at 9:57 AM, <pdk...@gmail.com> wrote:

Unfortunately this didn't make it into general use. I guess the edge cases were too troublesome, or Google is preparing to switch WebP to VP9.

It's still there, and there's no plan to remove it (far from it!). It's just too slow to be the default, even with the last optimisations.

Btw, it definitely should be better named than '-pre 4'. Any suggestion?

btw, in the incoming 0.6.0 release, the new option is 'cwebp -sharp_yuv ...' (and it's not 'on' by default, obviously).

There's a corresponding new field WebPConfig::use_sharp_yuv

(the -pre 4 / WebPConfig::preprocessing ones still work, for backward compatibility. Better not use them, though.)

skal/