Effecient compression of looped frames with VP9

[Kagami Hiiragi]

Hi.

I'm trying to compress repeated sequence of frames using VP9 codec and
achieve the smallest output size of webm. Unfortunately I can't
effeciently compress sequence of more than 2 frames: output size become
too big. I'm using ffmpeg with libvpx to encode the webm and webm_info
to analyze the compressed frame sizes.

Here are the demo:

Input frames:
https://imgur.com/a/sqdcH (save them to 1.png, 2.png, 3.png)

Encoding commands:
$ ffmpeg -r 5 -loop 1 -i %d.png -c:v libvpx-vp9 -b:v 0 -crf 15 \
-t 10 -pass 1 -f null /dev/null
$ ffmpeg -r 5 -loop 1 -i %d.png -c:v libvpx-vp9 -b:v 0 -crf 15 \
-t 10 -pass 2 -y 3.webm

Analyzing the output:
$ webm_info -i 3.webm -all |\
sed -rn 's/.*(size_payload: *[0-9]+).*/\1/p'
http://pastebin.com/5Miphyj7

As you can see, only few compressed frames have minimal size. If I
remove 3.png and encode once more, webm_info will look like this:
http://pastebin.com/7DzazVw3

VP9 draft states:
https://tools.ietf.org/html/draft-grange-vp9-bitstream-00#section-2.2.2
> but for any particular frame the three available references are
> dynamically selectable from a pool of eight stored reference frames

So as far as I can understand it should work with sequence up to 8
frames length.

Am I missing some option or don't understand something? Thanks.

[Kagami Hiiragi]

[Paul Wilkins]

From the point of view of the vast majority of YouTube and natural video content the ultra efficient coding of short repeat sequences of frames is a bit of an edge case and has not been high thus far on our list of priorities. For their use case it is impractical to make any manual changes to encode parameters on a clip by clip basis because they have to encode millions of clips each day.

That said, we have noticed that some types of animated material use repeat patterns (for example 3 frames) to create a sense of motion in what would otherwise be static scenes. I can also see that there are other cases where other users might want to create short repeat sequences, so I am not ruling out addressing this in the future.

You are right that there are up to 8 reference buffers that could be used in VP9 (though only 3 of those can be used to predict any one frames). Hence, if we were to cycle through those buffers as we encoded a sequence, then for each new frame search to find the one that matches the new frame, then we coul basically get away with coding each frame of a repeat sequence up to 8 once at high quality and then have almost nothing to do there after.

However, this would require a lot of specialist code and would mean searching all 8 buffers each frame, which would have a massive hit on encode time if this type of cyclic buffer usage were generalized. Also in the more general case additional buffers may be used to support, for example multiple alt ref coding patterns and adaptive image scaling.

If we were to go down this route it would almost certainly have to be a specialized encoding parameter / option where the user indicates that they wish to code a repeat sequence and the length of the pattern.

For the more generalized animation case we have made some changes recently that try to detect cases where the current frame almost exactly matches either the golden frame or alternate reference buffer (but not the last frame) and then divert bits from these frames to other less well predicted frames in each group, to try and even out visual quality.  This will still mean you get some frames coded almosty perfectly at almost no cost and others that cost more because they are not as well predicted though.

Regards,

Paul Wilkins