Re: Question about vpx_codec_decode in libvpx

147 views

Skip to first unread message

jim

unread,

Jun 17, 2015, 12:00:11 PM6/17/15

to Jim Bankoski, apps-devel

Hi Jim,

I am to add VP9 support in my project with libvpx, can you help to answer the following questions:

- What example can I refer to for how to use the encoder and decoder?

- What is the difference between the vp9_lossless_encoder.c, vp9_spatial_svc_encoder.c and vex_temporal_svc_encoder.c under the examples directory? Does it means that there are three modes the encoder works: lossless, spatial sac and temporal svc, what is the use case for these modes respectively?

- Does it means that the client do not need to tell the decoder what mode the encoder work?

Your help will be really appreciated!

Regards,

Jim

Original Message

Sender: Jim Bankoski<jimba...@google.com>

Recipient: jim<j...@justek.us>

Cc: apps-devel<apps-...@webmproject.org>

Date: Friday, May 15, 2015 21:31

Subject: Re: Question about vpx_codec_decode in libvpx

I'm afraid not - if a frame is corrupt you'll have to do some work more manually. You want to set the reference frame the encoder uses to a frame that occurred before the corruption. If you don't have a frame before the corruption to use as a reference you have to request a keyframe.

in the udp sample I suggested, this rechnique is used and is called a recovery frame ( search for the term recovery)..

This array covers the 3 types of recovery you might need.

unsigned int recovery_flags[] = {0, // NORMAL,

VPX_EFLAG_FORCE_KF, // KEY,

VP8_EFLAG_FORCE_GF | VP8_EFLAG_NO_UPD_ARF | VP8_EFLAG_NO_REF_LAST

| VP8_EFLAG_NO_REF_ARF, // GOLD = 2,

VP8_EFLAG_FORCE_ARF | VP8_EFLAG_NO_UPD_GF | VP8_EFLAG_NO_REF_LAST

| VP8_EFLAG_NO_REF_GF // ALTREF = 3

};

The first requests a keyframe, the second forces use of the golden frame as a reference by dissallowing the alt ref and last frame as reference, the third does the same thing but with the alt ref frame.

I hope this helps.

Good Luck,

Jim

On Thu, May 14, 2015 at 8:47 PM, jim <j...@justek.us> wrote:

Jim, thanks for your kindly help.

While in poor networks condition, the sreen becomes green. To improve the user experience, I think that we can let the screen freeze by using last successfully decoded frame to display, when the result of vpx_codec_decode() indicate that the current frame is corrupt.

So now my understand is :
step 1) Use vpx_codec_decode(vpx_codec_ctx_t *ctx,

   const uint8_t *data,

   unsigned int data_sz,

   void *user_priv,

   long deadline);
to decode rtp package data, and if the result is OK, and then get into step 2)

step 2) Use vpx_codec_get_frame(vpx_codec_ctx_t *ctx,

   vpx_codec_iter_t *iter)
to get the decoded frame, here the application should maintain these frames（the return value for vpx_codec_get_frame() ） if we want to use them in the future, right ? Another question here, we call vpx_codec_decode() one time, does it means that we can get one image in step 2), or some time we get more than one image ?

And in the case of the current frame to be decoded is a corrupt frame, in step 1) we get a result of VPX_CODEC_CORRUPT_FRAME. We can use the image we maintain in last step 2) after last successfully decoded frame in last step 1) , right ?

I also confused with the the comment of vpx_codec_decode() :
*\param[in] data Pointer to this block of new coded data. If
   *    NULL, a VPX_CODEC_CB_PUT_FRAME event is posted
   * for the previously decoded frame.

Does it mean that when the current frame being decoded is corrupt and then I can call the vpx_codec_decode() to tell the decoder to use last frame by setting the parameter data to NULL and data_sz to 0 ? Doese it mean that the applicaiton need not maintain the last successfully decoded frame by itself ?

Looking forwards to your help!

Regards,
Jim Wei

Original Message
Sender: Jim Bankoski<jimba...@google.com>
Recipient: apps-devel<apps-...@webmproject.org>
Cc: jim<j...@justek.us>
Date: Friday, May 15, 2015 08:36
Subject: Re: Question about vpx_codec_decode in libvpx

I'm not sure exactly what you are looking for.

This will get you one of 3 prior reference frames :

VP8_COPY_REFERENCE

You can see an example of how to use it in the examples, specifically vp8cx_set_ref.c

There are also some examples for dealing with udp dropped packets in the udp_sample project. Clone from here:
https://chromium.googlesource.com/webm/udpsample

and the far more full featured webrtc code base, here:
http://www.webrtc.org/native-code/development

I hope that helps.
Jim

On Thursday, May 14, 2015 at 2:32:16 AM UTC-7, jim wrote:
Hi vp8 experts,

Second try. Can anyone help?

Thanks,
Jim
Original Message
Sender: jim<j...@justek.us>
Recipient: apps-...@webmproject.org
Cc: wil...@justek.us
Date: Tuesday, May 12, 2015 18:13
Subject: Question about vpx_codec_decode in libvpx

Hi experts,

This is Jim and I am using libvpx for a voip project. Now I am confused with the method of


vpx_codec_err_t vpx_codec_decode(vpx_codec_ctx_t *ctx,

   const uint8_t *data,

   unsigned int data_sz,

   void *user_priv,

   long deadline);

How can I to get the previously decoded frame when I found something wrong with the current frame? According to the comment of
*\param[in] data Pointer to this block of new coded data. If

   * NULL, a VPX_CODEC_CB_PUT_FRAME event is posted

   * for the previously decoded frame.
Can I just call vpx_codec_decode() to tell the decoder to use last frame by setting the data to NULL and the data_sz to 0?
Or the application should buffer the correct frame, and to implement a call back for event of VPX_CODEC_CB_PUT_FRAME? If so, how to get that?

Looking forwards to your answer! Your help will be appreciated!

Regards
Jim

Jim Bankoski

unread,

Jun 22, 2015, 3:59:30 PM6/22/15

to jim, apps-devel

Sorry if you need to get the previously decoded frame after a frame that's gotten a garbage or broken frame you are stuck doing it yourself at the application level.

If we even partially decode a frame with the flag update last frame on the buffer is destroyed.

Its possible that using VP8_COPY_REFERENCE will let you get access to a frame that occurred before that time.

The comment refers to the fact that you call vpx_codec_decode multiple times without passing it a new buffer and get the same frame again. This is useful for dropped frames ( size 0 ) .