Re: can we please get 10bpc support in vp9?

[John Koleszar]

+oliver, webm-discuss, since this came up on that list today too

Hi Geoffrey,

Thanks for such a detailed and well considered post. I think there's a
lot to consider with 10 bit support, and a number of people have asked
for it, so it's certainly under consideration. It's going to be too
late to make it into the baseline feature set / profile of VP9. There
is going to be an additional extension defined to support 4:4:4 and a
4th plane (for alpha, or depth, for example), so we're already going
to have a second profile. It's worth talking about whether 10-bit
should be part of that, a second extension, or whether it's complexity
outweighs its benefits. As you say, the proliferation of profiles is
best avoided, so it needs to be considered carefully.

Congrats on getting an your VP8 encoder working. Always interested to
hear about other implementations, whether they ever see production or
not. Did you have any other insights from that work?


On Mon, May 13, 2013 at 12:51 AM, <geoff...@gmail.com> wrote:
> Hi there,
>
> Here's a rant prompted by the vp9 finalization post in webm-discuss. I think
> it probably belongs in this forum though. It's somewhat opinionated, but I
> don't mean to be obnoxious about it. I mean, it's your codec.
>
> I just think it would be significantly improved if it had 10 bit per channel
> decode support, mandatory and by default (encode would probably use 8bpc
> unless overridden though - that is, 10bpc as an optional extra, but one that
> must be supported). And that getting that in before everything is frozen
> would be a really good idea. On the flip side, it's already rather far along
> and trying to shoehorn that in at the last minute might be rather ...
> ambitious.
>
> I've read a few things about vp9, but haven't seen mention of any bit-depths
> greater than 8 bits per channel.
> http://datatracker.ietf.org/doc/draft-grange-vp9-bitstream/?include_text=1
> RFC seems to be the most detailed documentation around about what's turning
> up in it, (apart from the code itself, which I haven't gone through much at
> this point). So I apologize if there's something there already I've missed.
>
> I've messed around a bit with vp8, and actually written a functioning
> encoder (though feature-incomplete and very slow) for it from scratch.
>
> Working at 8-bits per channel hurts as you approach higher quality (lower
> quantizers), as you suffer non-trivial accumulated rounding error on top of
> quantization error. VP8 starts to look quite decent around quantizer 20 or
> so. The DC/AC quantizer LUTs result in quant factors of 21 and 24. However,
> the internal coefficients are multiplied/divided by 8 in the transform
> operation, so the actual effective quantization steps in pixel intensity
> units are 21/8 and 24/8 : 2.6 and 3.
>
> Putting that another way, messing with pushing the transform coeffs up and
> down by the smallest notch possible should perturb resultant pixel values by
> basis functions of RMS magnitude 2.6 or 3 pixel intensity units. Even if the
> reconstruction is done with somewhat higher working precision and then
> rounded to final pixel values fairly well (as it is), the quantization noise
> due to 8-bit representation is uncomfortably close to the quantization noise
> introduced through coefficient division and compounds the inaccuracy of the
> result. This also feeds into the effectiveness of spatial prediction modes,
> loop filtering, motion interpolation and other stuff. The whole
> effectiveness of compression at the higher end of quality is degraded by
> this. I realise that the quantizer scale and transforms has been reworked in
> vp9, which is nice, but if you're still working in 8bpc the point about
> quantizer notches hitting a similar scale to pixel-intensity notches at a
> particular quality/distortion level, and this degrading quailty per bit
> remains : the principle is the same although the fine details differ.
>
> It also means encoding of smooth, shallow gradients like skies is not as
> good as it should be, because a large portion of pixels may be within this
> small kind of intensity range, and there's no real masking effects from
> texture to hide it. If you just draw a sky-like gradient in 8-bit color,
> individual color bands are generally visible. That is, humans can see a
> 1-in-256 pixel level difference in this context (low-contrast, smooth
> features). With dithering, they disappear. If you then compress them, you
> end up with the problem that typically your DC notches are now more unevenly
> spaced than they should be due to limited working precision, and this effect
> is visible because of the absence of masking. The dithering is quantized out
> if present (except at very high bitrates) so ends up being beside the point.
> Loop-filtering doesn't help that much because it takes you from blocky
> uneven bands to smudgy uneven bands. Of course, some of this is due to just
> the fact that, hey, you're quantizing the video. But some of it is visibly
> worse because of insufficient channel precision.
>
> The problem is that in order to stamp out these artifacts properly, you end
> up in a quantizer zone where you're fighting the quantization of your pixels
> as well as the distortion you're introducing on purpose to save space and it
> looks tangibly, visibly worse than it ought to on the same bitrate with the
> same quantizer. This remains a problem at surprisingly low rates as the loss
> of masking through texture and smudgification make individual
> pixel-intensity-level scale perturbations (across blocks) actually count -
> essentially, your video becomes more sky-like to the extent that you
> compress the crap out of it.
>
> Examples of 8-vs-10 bits in h264 are visible here (particularly page 9, 10,
> 11 illustrate what I mean):
> http://x264.nl/x264/10bit_03-422_10_bit_pristine_video_quality.pdf
>
> The differences between h264 and vp8/9 should not be significant here;
> comparable gains should be expected : something like 0.2 dB PSNR up to 0.5
> dB at the highest possible bitrates: that's objective distortion, but note
> that the above stuff about sky gradients etc. is compounded by psychovisual
> considerations: i.e. it affects the quality in places where it's
> particularly noticeable; this isn't captured entirely by looking at it in
> PSNR terms. To really see how effective it is the best thing would be to
> build a 10-bit version of vp9 and seeing how it compares, but clearly that's
> not trivial.
>
> While it's true that a 10bpc option will make it harder on the CPU, and may
> complicate hardware design, I reckon it's really worth having. You already
> have vp8 as an easy-on-the-CPU option. You can allow vp9 video to be encoded
> at 8bpc when cheap encode/decode and low-rate video (which doesn't care so
> much) make it important, and switch to 10bpc when dealing with higher
> quality stuff or sending it to a stronger CPU.
>
> In my view, the right thing is to a) have 10bpc in the spec as mandatory to
> support for decode and b) ensure all vp9 hardware will deal with 8/10 bit
> decode. Actually using it is then a speed/quality tradeoff the encoder can
> make. The point is to avoid 'profile madness' and to have one format that
> decodes anywhere, albeit perhaps slowly and only at lower resolutions if you
> hit modest hardware with a 10-bit stream.
>
> A fallback position is to make it an optional extra - add a flag for it or
> the like, which sends you back into 'profile madness' territory: then, you
> can perhaps defer implementing anything at this point. Then you have the
> problem that when you do add it, old decoders and hardware built won't cope.
> 'Profile madness' is a bigger problem if it hits silicon; software can
> potentially be updated. If you end up with 'profile madness', specialised
> applications that know that everyone involved is able to deal with it can
> get the quality gain, but for the most part, it won't be realised. (10-bit
> h264 is in this position today). At this point you've really ended up with a
> separate codec, call it vp9.10 or something, with all the adoption and
> compatibility problems that causes. Although I reckon that's better than not
> having that option, it's not nearly as good as just getting mandatory 10-bit
> decode support in.
>
> Thanks for your consideration; I look forward to the arrival of final vp9
> (however many bits it has).
> - Geoffrey Attwater
>
> --
> You received this message because you are subscribed to the Google Groups
> "Codec Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to codec-devel...@webmproject.org.
> To post to this group, send email to codec...@webmproject.org.
> Visit this group at
> http://groups.google.com/a/webmproject.org/group/codec-devel/?hl=en.
> For more options, visit
> https://groups.google.com/a/webmproject.org/groups/opt_out.
>
>

[Pieter Kapsenberg]

Here are some challenges with implementing 10-bit (or more) from a hardware (HW) designer perspective:

• Data is organized in memory subsystem granularity. A common granularity is 64 bits, or 128. Going with 64 for this example, every memory transaction is 8 pixels when bit depth is 8. It packs neatly. For 10-bit, the most straightforward approach is to just use 16-bits per sample to make all the addressing math easy; you can reuse all the 8-bit stuff and for the most part just multiply by 2. Each transaction is then 4 pixels instead of 8, which of course it is wasteful since 6 bits per pixel are wasted. Packing 10-bits would allow for 6 pixels in a 64-bit word (60 bits used), but that makes all the addressing complicated. And still 4 bits are wasted.
  This kind of waste results in higher memory bandwidth (transactions per second), which is the main driver of a hw codec's power consumption in a mobile (battery) product.
• Another big problem is the rest of the system. A 10-bit codec requires different organization of pixels in memory than 8-bit, and the rest of the system must be able to handle this too. This includes the GPU for 3D (imagine 10-bit video playing on an OpenGL surface), the display processor which must create RGB out of YUV and talk to the monitor/screen. So an upgrade to 10-bit typically means a whole slew of HW units need to adapt.
• The majority of the silicon area used by a HW codec is used for pixel processing. Things like loop filters, motion compensation filters are full of pixel rams and multipliers that take up a bunch of area. Going from 8-bit to 10-bit pretty much directly implies 20% bump in area of these kinds of processing units, and if there is one thing a HW designer tries to minimize it is silicon area, which drives cost. 

Of course none of these issues are insurmountable, but because of the effort involved in the upgrade to 10-bit, it does require a compelling reason to convince the industry to move forward with it.