Does any one know what is the quantize_b in aom av1 encoder?

[Junhui Liang]

Is there any document about the quantize_b? What does the “b” mean ?

[James Zern]

Hi,

On Thu, Oct 13, 2022 at 3:12 AM Junhui Liang <huste...@gmail.com> wrote:

Is there any document about the quantize_b? What does the “b” mean ?

This has been carried over from vp9 and vp8 before it. _b is block, _mb is macroblock, _fp is fast (quantization) process.

Many things in libaom can be traced back to libvpx.

 

--
You received this message because you are subscribed to the Google Groups "AV1 Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to av1-discuss...@aomedia.org.
To view this discussion on the web visit https://groups.google.com/a/aomedia.org/d/msgid/av1-discuss/f86df675-92ef-4917-aa6a-5b483aa10490n%40aomedia.org.

[Junhui Liang]

Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b. 

In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep. 

But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean？Is that better than quantize_fp?

[James Zern]

Hi,

On Fri, Oct 14, 2022 at 12:12 AM Junhui Liang <huste...@gmail.com> wrote:

Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b. 

In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep. 

But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean？Is that better than quantize_fp?

The current code is:

             y  = ((((x * quant_ptr[rc]) >> 16) + x)

                 * quant_shift_ptr[rc]) >> 16;       /* quantize (x) */

It was modified in:

  https://chromium.googlesource.com/webm/libvpx/+/eca59cad0b76d86f8ca8288afa074ee47046e7c7

The idea is to add precision to the calculation, it was originally modified in:
  https://chromium.googlesource.com/webm/libvpx/+/e04e293522a3cf3761eae3690b8efbc2aa69848b

[Junhui Liang]

Thank you so much for your reply!

[Junhui Liang]

It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

[James Zern]

On Wed, Oct 19, 2022 at 12:21 AM Junhui Liang <huste...@gmail.com> wrote:

It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

I don't believe there is for this particular piece of code. I would suggest running some sample coefficient values through the calculation using the VP8 quantization tables to compare the results.

[Junhui Liang]

Thanks a lot for your reply!