# Does any one know what is the quantize_b in aom av1 encoder?

38 views

### Junhui Liang

Oct 13, 2022, 6:12:35 AM10/13/22
to AV1 Discussion
Is there any document about the quantize_b? What does the “b” mean ?

### James Zern

Oct 13, 2022, 7:39:31 PM10/13/22
to av1-d...@aomedia.org
Hi,

On Thu, Oct 13, 2022 at 3:12 AM Junhui Liang <huste...@gmail.com> wrote:
Is there any document about the quantize_b? What does the “b” mean ?

This has been carried over from vp9 and vp8 before it. _b is block, _mb is macroblock, _fp is fast (quantization) process.
Many things in libaom can be traced back to libvpx.

--
You received this message because you are subscribed to the Google Groups "AV1 Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to av1-discuss...@aomedia.org.
To view this discussion on the web visit https://groups.google.com/a/aomedia.org/d/msgid/av1-discuss/f86df675-92ef-4917-aa6a-5b483aa10490n%40aomedia.org.

### Junhui Liang

Oct 14, 2022, 3:12:15 AM10/14/22
Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b.
In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep.
But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean？Is that better than quantize_fp?

### James Zern

Oct 14, 2022, 1:14:18 PM10/14/22
to AV1 Discussion
Hi,

On Fri, Oct 14, 2022 at 12:12 AM Junhui Liang <huste...@gmail.com> wrote:
Thanks for you reply. Now I knew the quantize_b is carried over from vpx and the _b is block. I am still confused about the quantization process of the quantize_b.
In the quantize_fp of vp8, the quantization proess is simple to understand. The code is y = ((x + round_ptr[rc]) * quant_ptr[rc]) >> 16. I can understand that it means coefficient devided the Qstep.
But in the quantize_b of vp8, the code is x+=round_ptr[rc]; y = (((x * quant_ptr[rc]) >> 16) + x) >> quant_shift_ptr[rc]. What does it mean？Is that better than quantize_fp?

The current code is:
y  = ((((x * quant_ptr[rc]) >> 16) + x)
* quant_shift_ptr[rc]) >> 16;       /* quantize (x) */
It was modified in:

The idea is to add precision to the calculation, it was originally modified in:

### Junhui Liang

Oct 16, 2022, 12:27:05 AM10/16/22

### Junhui Liang

Oct 19, 2022, 3:21:51 AM10/19/22
to AV1 Discussion, Junhui Liang, jz...@google.com
It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

### James Zern

Oct 21, 2022, 4:58:28 PM10/21/22
to AV1 Discussion
On Wed, Oct 19, 2022 at 12:21 AM Junhui Liang <huste...@gmail.com> wrote:
It is difficult for me to understand why it is more exact than quant_fp. What does the "exact" mean? Is there any document to explain the idea?

I don't believe there is for this particular piece of code. I would suggest running some sample coefficient values through the calculation using the VP8 quantization tables to compare the results.

### Junhui Liang

Oct 23, 2022, 10:13:02 PM10/23/22