Does any one know what is the quantize_b in aom av1 encoder?

38 views
Skip to first unread message

Junhui Liang

unread,
Oct 13, 2022, 6:12:35 AM10/13/22
to AV1 Discussion
Is there any document about the quantize_b? What does the “b” mean ?

James Zern

unread,
Oct 13, 2022, 7:39:31 PM10/13/22
to av1-d...@aomedia.org
Hi,

On Thu, Oct 13, 2022 at 3:12 AM Junhui Liang <huste...@gmail.com> wrote:
Is there any document about the quantize_b? What does the “b” mean ?

This has been carried over from vp9 and vp8 before it. _b is block, _mb is macroblock, _fp is fast (quantization) process.
Many things in libaom can be traced back to libvpx.
 

--
You received this message because you are subscribed to the Google Groups "AV1 Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to av1-discuss...@aomedia.org.
To view this discussion on the web visit https://groups.google.com/a/aomedia.org/d/msgid/av1-discuss/f86df675-92ef-4917-aa6a-5b483aa10490n%40aomedia.org.

Junhui Liang

unread,
Oct 14, 2022, 3:12:15 AM10/14/22
to AV1 Discussion, jz...@google.com
Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b. 
In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep. 
But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean?Is that better than quantize_fp?

James Zern

unread,
Oct 14, 2022, 1:14:18 PM10/14/22
to AV1 Discussion
Hi,

On Fri, Oct 14, 2022 at 12:12 AM Junhui Liang <huste...@gmail.com> wrote:
Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b. 
In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep. 
But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean?Is that better than quantize_fp?

The current code is:
             y  = ((((x * quant_ptr[rc]) >> 16) + x)
                 * quant_shift_ptr[rc]) >> 16;       /* quantize (x) */
It was modified in:

The idea is to add precision to the calculation, it was originally modified in:
  https://chromium.googlesource.com/webm/libvpx/+/e04e293522a3cf3761eae3690b8efbc2aa69848b

Junhui Liang

unread,
Oct 16, 2022, 12:27:05 AM10/16/22
to AV1 Discussion, jz...@google.com
Thank you so much for your reply!

Junhui Liang

unread,
Oct 19, 2022, 3:21:51 AM10/19/22
to AV1 Discussion, Junhui Liang, jz...@google.com
It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

James Zern

unread,
Oct 21, 2022, 4:58:28 PM10/21/22
to AV1 Discussion
On Wed, Oct 19, 2022 at 12:21 AM Junhui Liang <huste...@gmail.com> wrote:
It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

I don't believe there is for this particular piece of code. I would suggest running some sample coefficient values through the calculation using the VP8 quantization tables to compare the results.

Junhui Liang

unread,
Oct 23, 2022, 10:13:02 PM10/23/22
to AV1 Discussion, jz...@google.com
Thanks a lot for your reply!
Reply all
Reply to author
Forward
0 new messages