Possible Issue With mplayer/libavcodec

[Duane Robertson]

Has anyone noticed mplayer stuttering during slow pans in vp8 video,
especially after a scene change? This happens only when I set
--auto-alt-ref to on. The delay is just enough to be irritating to me.
I'm not even sure that anyone else has seen it, assuming it's not a
problem in my method.

I've updated and recompiled libvpx, ffmpeg, and everything in mplayer
over the last forty-eight hours. (I tried the latter with and without
the patch from the webm site.) I've tried one and two pass encoding,
mostly using the recommended settings. I've also viewed both the
naked .ivf files and matroska files with audio. Gstreamer doesn't show
the issue, raw video from the reference decoder doesn't, and neither
does video converted with ffmpeg.

I understand that mplayer uses libavcodec to decode vp8, and I seem to
remember that ffmpeg is using a faster implementation of the decoder. I
wonder if mplayer might have trouble dealing with the extra, undisplayed
alt-ref frames at some point in the decoding chain.

Is anyone else using mplayer much with vp8? Can anyone suggest anything
to narrow the issue down?

[Duane Robertson]

Is there any easy way to analyze a vp8 stream to determine if it's
legitimate and/or give more information about it? This is the one I have
in mind:

https://docs.google.com/a/duanerobertson.com/leaf?id=0B8fG4DFq4jlRNGFhNDBhZTAtMTEzYi00YjQyLWJiM2UtMGZhZjU1YjIxYmYw&sort=name&layout=list&num=50

I know that it has 379 frames, 358 of which are visible, but that's it
so far.

[Duane Robertson]

I've been going through the libavcodec/vp8* code, and struggling to
understand it, but I have noticed one interesting thing already--the
problem seems to be (directly or indirectly) associated with failing to
set the previous frame buffer to the current frame data.

When I force an update every frame, so (vp8.c):

//if (s->update_last) // move cur->prev
s->framep[VP56_FRAME_PREVIOUS] = s->framep[VP56_FRAME_CURRENT];

video created with "--auto-alt-ref=0" will play with no apparent
problems. However, the clip created with auto-alt-ref enabled shows
video breakup at the same points I would see skipping when using mplayer
compiled normally (latest svn).

I posted the issue on the libavcodec/mplayer list, but haven't gotten a
reply yet. Does anyone know off-hand if there's anything significant
about the times ivfenc prevents the previous frame buffer from being
updated? Does it only do so when auto-alt-ref is on? I'll keep looking
at both sets of code.

"""
I've noticed lately that video I create with webm's ivfenc, using the
"--auto-alt-ref=1" setting, stutters during pans (especially after a
scene change) when played in mplayer. When I view the clips in vlc or
using gstreamer, the video looks smooth. When I make video with
auto-alt-ref disabled, it looks normal in all players.
"""

[David Conrad]

On Aug 10, 2010, at 2:30 PM, Duane Robertson wrote:

> I've been going through the libavcodec/vp8* code, and struggling to
> understand it, but I have noticed one interesting thing already--the
> problem seems to be (directly or indirectly) associated with failing to
> set the previous frame buffer to the current frame data.
>
> When I force an update every frame, so (vp8.c):
>
> //if (s->update_last) // move cur->prev
> s->framep[VP56_FRAME_PREVIOUS] = s->framep[VP56_FRAME_CURRENT];
>
> video created with "--auto-alt-ref=0" will play with no apparent
> problems. However, the clip created with auto-alt-ref enabled shows
> video breakup at the same points I would see skipping when using mplayer
> compiled normally (latest svn).

The issue you're seeing is a result of two things:

1. in mplayer, decode/display is not pipelined, so each frame must be decoded in the time between displaying frames
2. altref frames are not displayed, so the next frame must be decoded also in that time slice between displaying frames, essentially resulting in spikes of nearly 2x the amount of work to be done per frame

Thus, for videos that are using more than about ~60-70% of CPU to play, a stutter at each altref can be expected in mplayer.

Also, because altrefs aren't meant to be displayed, the single frame of garbage you see when commenting out that line is expected.

Pipelining decode/display in mplayer would reduce this a bit, as would any decoder speedup including decoder threading (lavc's is currently single threaded.)

[Duane Robertson]

That sounds reasonable, David, but I suspect that what you're describing
isn't what I'm seeing, or at least not all of it. Mplayer is using an
average of 3-6% of one of my cpu's time playing video at 24 frames per
second. Even if several alt-refs showed up back to back, it's hard for
me to picture it bogging down the cpu enough that it couldn't display a
frame 1/24th of a second later. There could be a bottleneck elsewhere, I
suppose.

I agree that mplayer isn't handling non-displayed frames right though,
and I've given up trying to figure out how to deal with that, but
interestingly enough, I managed to fix the issue temporarily. I just had
libavcodec pass the previous frame instead of no frame when decoding an
alt-ref. I'm guessing mplayer is just sitting idle for one more 1/24th
of a second when it gets nothing back (as a best case).

http://bugzilla.mplayerhq.hu/show_bug.cgi?id=1759
"""
I can correct the problem (to my eyes) by forcing vp8_decode_frame()
(libavcodec/vp8.c) to return the previous (displayed) frame, rather than no
frame, each time it decodes a non-displayed (alt-ref) frame:

line 1645:
if (!s->invisible) {
*(AVFrame*)data = *s->framep[VP56_FRAME_CURRENT];
*data_size = sizeof(AVFrame);
+ } else {
+ *(AVFrame*)data = *s->framep[VP56_FRAME_PREVIOUS];
+ *data_size = sizeof(AVFrame);
+ //av_log(avctx, AV_LOG_WARNING, "invisible frame\n");
}

"""

This works for me, and doesn't even seem to make mplayer adjust the AV
sync, oddly enough (judging from the status line). Obviously I'm only
linking this into mplayer.