implementing a webm video decoder questions

[tanguy...@hotmail.com]

Hello,

I'm trying to implement a webm video decoder on an embedded device.

Because resources are scarce, I'm coding my own webm container demuxer, and then using the libvpx decoder to decode the frames.

I have a few questions about libvpx:

1) Can you force the decoder to always output the decoded frames in a certain format (eg: VPX_IMG_FMT_I420 or VPX_IMG_FMT_RGB24)

     -  If you can't force this setting, will it always be the same inside of one video/file?

2) The documentation isn't clear on whether there is only one encoded frame in a simpleblock structure, or if there can be multiple.

     -  If the answer is multiple, can they be interlaced?

3) If the frames are stored in Decoding Order, but the decoder outputs them in Displaying order; How do I know if I've given enough frames to the decoder? Or how does the decoder know that there isn't another frame that needs to be shown first, but that it has not received yet?

I'm sorry if some of these questions seem trivial, but this is my first time coding something related to video codecs.

Any help would be greatly appreciated!

[James Zern]

Hi,

On Fri, Sep 30, 2016 at 7:26 AM, <tanguy...@hotmail.com> wrote:
> Hello,
>
> I'm trying to implement a webm video decoder on an embedded device.
> Because resources are scarce, I'm coding my own webm container demuxer, and
> then using the libvpx decoder to decode the frames.
>
> I have a few questions about libvpx:
>
> 1) Can you force the decoder to always output the decoded frames in a
> certain format (eg: VPX_IMG_FMT_I420 or VPX_IMG_FMT_RGB24)
> - If you can't force this setting, will it always be the same inside
> of one video/file?
>

The decoder will output what's appropriate for the profile
(420/422/444/440). For 8-bit video 420 is the only output, you can't
request a conversion, the other formats are an anachronism.

> 2) The documentation isn't clear on whether there is only one encoded frame
> in a simpleblock structure, or if there can be multiple.
> - If the answer is multiple, can they be interlaced?
>

There's only one displayable frame for each simpleblock. The decoder
can handle multiple frames as packed in a vp9 superframe which is
usually 1 alternate reference frame (non-displayable) and 1 dependent
(displayable) frame.

> 3) If the frames are stored in Decoding Order, but the decoder outputs them
> in Displaying order; How do I know if I've given enough frames to the
> decoder? Or how does the decoder know that there isn't another frame that
> needs to be shown first, but that it has not received yet?
>

There's no frame reordering in vp8 and vp9. In vp8's case if you
provide a non-displayable frame (alternate reference) as input you
won't receive any output and should proceed to the next frame.

[0phoff]

Thanks for the reply!

On Thursday, October 6, 2016 at 1:29:32 AM UTC+2, James Zern wrote:

Hi,

> 1) Can you force the decoder to always output the decoded frames in a
> certain format (eg: VPX_IMG_FMT_I420 or VPX_IMG_FMT_RGB24)
>      -  If you can't force this setting, will it always be the same inside
> of one video/file?
>

The decoder will output what's appropriate for the profile
(420/422/444/440). For 8-bit video 420 is the only output, you can't
request a conversion, the other formats are an anachronism.

If I understand you correctly, not all the formats specified in the enum vpx_img_fmt_t (see vpx/vpx_image.h) are supported by the VP8/VP9 codec?

So the only formats that are supported would be planar YUV? (with the different types of subsampling ofc)

What about the packed formats that are defined in the enum?

[James Zern]

Just some of the planar ones. That's 420 for vp8 and 420/422/444/440 for vp9 depending on the profile, from vp9/common/vp9_enums.h:

// Bitstream profiles indicated by 2-3 bits in the uncompressed header.

// 00: Profile 0.  8-bit 4:2:0 only.

// 10: Profile 1.  8-bit 4:4:4, 4:2:2, and 4:4:0.

// 01: Profile 2.  10-bit and 12-bit color only, with 4:2:0 sampling.

// 110: Profile 3. 10-bit and 12-bit color only, with 4:2:2/4:4:4/4:4:0

//                 sampling.

If you need to convert from one format to another libyuv or libswscale from ffmpeg may be helpful if the platform you're working on doesn't offer an alternative.