We are experimenting with encoding longer video segments with libvpxvp9. But what we need are keyframes at certain positions to be able to split the output bitstream in a certain way. We use the flag VPX_FRAME_IS_KEY when pushing frames into the encoder and run 2 pass encoding. What we found is, that the frames, which are marked this way are not always coded as keyframes in the output. I experimented with the altref test-case and was able to reproduce it also with the latest version in the main branch (see attachment).
We investigated the rate control algorithm for 2 pass and it looks like the lookahead algorith that decides where to place the keyframes in the second pass does not consider the VPX_FRAME_IS_KEY flag at all. It just bases its decision on the stats from the first pass, where the flag was considered. While this results in keyframes in most of the cases, it is not guaranteed. We are currently testing how to fix it. Our idea is:
- Put a flag into the first pass stats file that indicates if this is a forced keyframe.
- In the second pass, check that the same frames are marked as keyframes. Having this different between the passes makes no sense in our eyes.
- In the function vp9_get_frames_to_next_key, consider the flag (get it from the stats) and thusly enforce a keyframe
Any input would be highly appreciated. Does this sound like a valid solution or are we not considering something? I will also be able to submit a PR.