How to correctly seek frames in a webm file

[Eliya]

Hi all,

Please assist with supporting a seek functionality,

I Have finished implementing a webM , Alpha Encoder/Decoder.

I need assistant in the Parsing/Decoding part:

I am using the webm_parser and other inherited parsers from chrome source.

It works smoothly when playing in a sequential fashion, 

i.e looping on Parse(buf,buf_size) and then decoding :

StreamParserBuffer* buff = webm_stream_parser->cluster_parser_->video_.buffers_.back();

and then calling : 

vpx_codec_decode to each data_buff (it carries both the RGB data_buf and the alpha data_buf)

I then tried to create a Seek / GoToFrame(frame_index) functionality :

But It doesn't work.

So For example : a GoToFrame(index+1) loop (i.e starting from the second frame) will result with a black frame and after several frames it will start to output the correct frame.

If I try a GoToFrame(rand() % 5) (my example file contains 5 frames) it will out put this:

The parser seems to understand just fine the content of the bytes, i.e it finds the correct blocks and copy's them to a new StreamParserBuffer

but the result of the Decoder is not good..

[James Zern]

Hi,

This is the expected behavior. If you start decoding from an arbitrary frame you can expect corruption. You'll need to seek to a keyframe (cues are usually used for this) and start decoding from there. If you need frame perfect seeking, you'll want to seek to the nearest keyframe and discard the decoded output until you reach the frame you want.

[eliy...@gmail.com]

Hi James, 

Thanks for the quick and clear response.

I was missing the understanding that a cluster correlates to a single key frame.

>>You'll need to seek to a keyframe (cues are usually used for this)

Yes, thanks! aka "Cluster position"

>>and start decoding from there. If you need frame perfect seeking, you'll want to seek to the nearest keyframe and discard the decoded output until you reach the frame you want.

So I would need to parse & decode an entire cluster? 

In the end I Decode one frame:

int VpxDecode(const StreamParserBuffer& parser_buffers, VideoFrame& frame);

(where parser_buffers contains the RGB channel Block and the alpha channel BlockAdditions.)

So, you suggest that If I call VpxDecode on all buffers in a cluster from start to end the decoding will be fine?

Thanks!

[James Zern]

Hi,

On Tue, Oct 14, 2014 at 11:15 AM, <eliy...@gmail.com> wrote:

Hi James, 

Thanks for the quick and clear response.

I was missing the understanding that a cluster correlates to a single key frame.

>>You'll need to seek to a keyframe (cues are usually used for this)

Yes, thanks! aka "Cluster position"

>>and start decoding from there. If you need frame perfect seeking, you'll want to seek to the nearest keyframe and discard the decoded output until you reach the frame you want.

So I would need to parse & decode an entire cluster? 

Or a portion of one if the frame is mid-cluster. Ideally a cluster should start with a keyframe, but note that if you're using cues they are restricted to referencing frames at the beginning of the cluster, any prior keyframe will do as it resets the decoder state.

 

In the end I Decode one frame:

int VpxDecode(const StreamParserBuffer& parser_buffers, VideoFrame& frame);

(where parser_buffers contains the RGB channel Block and the alpha channel BlockAdditions.)

So, you suggest that If I call VpxDecode on all buffers in a cluster from start to end the decoding will be fine?

Yes unless there's some other issue with the file or decoder, you should be able to successfully starting decoding from any keyframe.

 

Thanks!

On Monday, October 13, 2014 7:38:03 PM UTC+3, James Zern wrote:

Hi,

On Mon, Oct 13, 2014 at 5:41 PM, Eliya <eliy...@gmail.com> wrote:

Hi all,

Please assist with supporting a seek functionality,

I Have finished implementing a webM , Alpha Encoder/Decoder.

I need assistant in the Parsing/Decoding part:

I am using the webm_parser and other inherited parsers from chrome source.

It works smoothly when playing in a sequential fashion, 

i.e looping on Parse(buf,buf_size) and then decoding :

StreamParserBuffer* buff = webm_stream_parser->cluster_parser_->video_.buffers_.back();

and then calling : 

vpx_codec_decode to each data_buff (it carries both the RGB data_buf and the alpha data_buf)

I then tried to create a Seek / GoToFrame(frame_index) functionality :

But It doesn't work.

So For example : a GoToFrame(index+1) loop (i.e starting from the second frame) will result with a black frame and after several frames it will start to output the correct frame.

If I try a GoToFrame(rand() % 5) (my example file contains 5 frames) it will out put this:

The parser seems to understand just fine the content of the bytes, i.e it finds the correct blocks and copy's them to a new StreamParserBuffer

but the result of the Decoder is not good..

This is the expected behavior. If you start decoding from an arbitrary frame you can expect corruption. You'll need to seek to a keyframe (cues are usually used for this) and start decoding from there. If you need frame perfect seeking, you'll want to seek to the nearest keyframe and discard the decoded output until you reach the frame you want.

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webm-discuss...@webmproject.org.
To post to this group, send email to webm-d...@webmproject.org.
Visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/.
For more options, visit https://groups.google.com/a/webmproject.org/d/optout.

[Brendan Bolles]

On Oct 15, 2014, at 3:11 AM, 'James Zern' via WebM Discussion wrote:

> Or a portion of one if the frame is mid-cluster. Ideally a cluster should start with a keyframe, but note that if you're using cues they are restricted to referencing frames at the beginning of the cluster, any prior keyframe will do as it resets the decoder state.

Is there some clever way to know if you have a keyframe mid-cluster?

So if I want frame 120, and the cluster starts at frame 100, there usually will be a keyframe at frame 100. But then as I march along, I may find another keyframe at 110, so I've decoded those first 10 frames for nothing. I could have just started at frame 110. Any way to prevent that from happening?


Brendan

[James Zern]

That's what cues [1] are meant for, it's up to the muxer to place them
intelligently.

[1] http://www.matroska.org/technical/specs/index.html