Skipping B Frame and Vibration playback
MyCometG3
And found there is still some issue with B frame and reordering
frames.
Would you please investigate?
//
This issue was posted previously over a year before in: <a
href="http://groups.google.com/group/perian-discuss/browse_thread/
thread/76f92ab3177980f9/4709ad420977e721?
hl=en&lnk=gst&q=MyCometG3#4709ad420977e721">h264 decode order in mov
file</a>
It still remains under Perian decoder.
//
Sample files uploaded: <a href="http://groups.google.com/group/perian-
discuss/web/perian-1.1.1-error.zip?hl=en">perian-1.1.1-error.zip</a>
MyCometG3
- Perian does not implement reordering frames by itself for H.264
data.
Inside Perian-1.1/Patches/***.diff, libavcodec/h264.c is not patched
as "emit in draw order".
//
I think, the only solution is, to patch libavcodec/h264.c, and enforce
decoder to emit every frame in decode order, not draw order. Also,
reorder frames by Perian itself, instead of libavcodec/h.264.c.
//
As far as I know, current QuickTime BaseDecompressor will pass one
sample data only, to decompressor component at one time. (decompressor
can not seek prev/next sample data by itself)
So, suppose the case using QuickTime Player, and using right/left
arrow key to seek frames.
I think current Perian implementation does not support this feature,
"step seek".
decode order; I1 P2 B3 P4 B5
draw order; I1 B3 P2 B5 P4
(If codecCapability subCodecSupportsDrawInDecodeOrder is false)
1)When QT requires to decode second frame, it will call decodeBand I1,
P2, B3, then drawBand B3.
2)Next, pressing right arrow key, step forward to third frame. It will
simply call drawBand P2, without any decodeBand. What happens?
(If codecCapability subCodecSupportsDrawInDecodeOrder is false)
3)Next, pressing right arrow key, step forward to forth frame. It will
call decodeBand P4, B5, then drawBand B5.
4)Next, pressing right arrow key, step forward to third frame. It will
simply call drawBand P4, without any decodeBand. What happens?
5)Now, pressing left arrow key, step backward to forth frame. What
happens? Does it show B5? or P4?
To handle this situation, does Perian cache every decoded frames to
display later? Or does it cache source data stream to re-decode? How
do you deal with this issue? (Caching decompressed frame requires to
handle decode-draw order combination inside decompressor component)
So, I guess stepping seek feature is broken. (This could affect your
choppy exporting issue)
//
P.S. I have lost my trac account long time ago, name should be
"MyCometG3". Is it able to reset passwd?
Graham Booker
On Sep 30, 2008, at 8:56 PM, MyCometG3 wrote:
>
> I think the root problem is that:
> - Perian does not implement reordering frames by itself for H.264
> data.
>
It does where it has to. Data stored in avi files for example; but
this is not done for mkv nor mov files since those files have
information indicating decode and display order.
> Inside Perian-1.1/Patches/***.diff, libavcodec/h264.c is not patched
> as "emit in draw order".
>
huh?
> //
>
> I think, the only solution is, to patch libavcodec/h264.c, and enforce
> decoder to emit every frame in decode order, not draw order. Also,
> reorder frames by Perian itself, instead of libavcodec/h.264.c.
>
Uh, we always get frames in decode order. See the frame storage
method mentioned below.
> //
>
> As far as I know, current QuickTime BaseDecompressor will pass one
> sample data only, to decompressor component at one time. (decompressor
> can not seek prev/next sample data by itself)
>
Correct; a major issue for the AVI file format, but not so much with
others.
This is the reason I want to kill the moron who decided to stick
advanced codecs in a format that cannot handle them properly. The
other is that it should never have been done in the first place.
> So, suppose the case using QuickTime Player, and using right/left
> arrow key to seek frames.
> I think current Perian implementation does not support this feature,
> "step seek".
>
> decode order; I1 P2 B3 P4 B5
> draw order; I1 B3 P2 B5 P4
>
Actually, it does.
> (If codecCapability subCodecSupportsDrawInDecodeOrder is false)
What does this mean? You mention it twice. Was one supposed to be
with the flag is true? BTW, documentation on that flag (and numerous
other things in QuickTime) is severely lacking.
>
> 1)When QT requires to decode second frame, it will call decodeBand I1,
> P2, B3, then drawBand B3.
I prefer to number frames in draw order, but yeah:
I1, P3, B2. It will decode I1, P3, then B2, then draw B2.
>
> 2)Next, pressing right arrow key, step forward to third frame. It will
> simply call drawBand P2, without any decodeBand. What happens?
Yep, no decode is called, just draw.
>
> (If codecCapability subCodecSupportsDrawInDecodeOrder is false)
Again, what does this mean?
>
> 3)Next, pressing right arrow key, step forward to forth frame. It will
> call decodeBand P4, B5, then drawBand B5.
> 4)Next, pressing right arrow key, step forward to third frame. It will
> simply call drawBand P4, without any decodeBand. What happens?
Same as above? Yeah, if I'm reading you right.
>
> 5)Now, pressing left arrow key, step backward to forth frame. What
> happens? Does it show B5? or P4?
(draw order again)
QT calls decode on I1, P3, P5, B4, and then draw on B4. This is what
QT does; we cannot control that. This is why stepping backwards is
slow in comparison.
>
>
> To handle this situation, does Perian cache every decoded frames to
> display later? Or does it cache source data stream to re-decode? How
> do you deal with this issue? (Caching decompressed frame requires to
> handle decode-draw order combination inside decompressor component)
Depends on the situation, sorta. In intelligent file formats (mov and
mkv), it simply caches decoded frames.
So, decode of I1, P3, B2, it remembers decoded frames I1, P3, and B2.
This is what the FFusionGetBuffer, and FFusionReleaseBuffer functions
are for.
In unintelligent formats, we only cache source data for frames which
have yet to be decoded.
>
>
> So, I guess stepping seek feature is broken. (This could affect your
> choppy exporting issue)
>
The choppy export is a bug in QuickTime (rdar 5770288)! We mark which
frames can be dropped if playback is slow in the hopes that QuickTime
does something intelligent with that information. Sometimes is does,
but in the case of export, it does exactly the wrong thing (drops
droppable frames). Either the flags do something other than what we
are lead to believe, or QT is doing the wrong thing and no one seems
to have noticed.
Several of your questions, and ours, could be answered much better if
we were able to steal a QT engineer's times for about a day.
Unfortunately, since Apple is such a large company and has to worry
about legal issues, official channels are glacial, at best.
> //
>
> P.S. I have lost my trac account long time ago, name should be
> "MyCometG3". Is it able to reset passwd?
No such account exists. You likely never actually created one.
- Graham
MyCometG3
I'm sorry to bother you, with my poor English ability. European
languages are quite different from my one...
> > I think current Perian implementation does not support this feature,
> > "step seek".
Please try sample movie "B. SkipB and Vibration.mov" with Perian
1.1.1. It is inside perian-1.1.1-error.zip (uploaded in this
discussion area)
- a. Open "B. SkipB and Vibration.mov" inside QuickTime Player.
- b. Seek to 4:20, for example.
- c. Press right arrow key many times.
You can see Perian failed to draw 1 of each 2 consecutive frames. ex.
4:27, 4:33, 4:40, etc. It seems to fail on each P or B frame. This
does not happen on Apple H.264 Decoder.
//
Also, Perian plays that movie in incorrect order. (it was previously
posted issue)
- a. Open "B. SkipB and Vibration.mov" inside QuickTime Player.
- b. Seek to 4:20, for example.
- c. Play.
You can see Perian does choppy playback at last GOP. This does not
happen on Apple H.264 Decoder.
//
>No such account exists. You likely never actually created one.
Graham Booker
On Oct 2, 2008, at 4:36 AM, MyCometG3 wrote:
>
> Graham-san,
> I'm sorry to bother you, with my poor English ability. European
> languages are quite different from my one...
>
>>> I think current Perian implementation does not support this feature,
>>> "step seek".
>> Actually, it does.
>
> Please try sample movie "B. SkipB and Vibration.mov" with Perian
> 1.1.1. It is inside perian-1.1.1-error.zip (uploaded in this
> discussion area)
> - a. Open "B. SkipB and Vibration.mov" inside QuickTime Player.
> - b. Seek to 4:20, for example.
> - c. Press right arrow key many times.
> You can see Perian failed to draw 1 of each 2 consecutive frames. ex.
> 4:27, 4:33, 4:40, etc. It seems to fail on each P or B frame. This
> does not happen on Apple H.264 Decoder.
Arg, in the shortcut case where a band enters BeginBand that has
already been decoded, we didn't properly save the frame number, thus
it couldn't find the decoded frame to use during draw. Committed a
fix which does this, thus solving the skipping of B frames. Note:
this was not an issue with playback, as those bands were only began
and ended once, so no shortcut case was used.
>
>
> //
>
> Also, Perian plays that movie in incorrect order. (it was previously
> posted issue)
> - a. Open "B. SkipB and Vibration.mov" inside QuickTime Player.
> - b. Seek to 4:20, for example.
> - c. Play.
> You can see Perian does choppy playback at last GOP. This does not
> happen on Apple H.264 Decoder.
Quicktime does some really screwy things during playback of the last
GOP. It decides that it needs to re-decode the last frame, so it
proceeds to decode it from the last I frame, through the P frames,
until it reaches that frame. This is not so much an issue but that it
decides to do this operation *while* it is still drawing frames from
playback. FFMPEG aggressively re-uses frames, so when it sees an I
frame, it trashes all the other frame data since it is no longer needed.
I tried playing with the frame caching mechanism, doing a reference
counting on how many bands were using a particular frame, and only
freeing it when both ffmpeg, and all bands are through with it. This
does eliminate the problem, but I'm not sure if it is the correct
solution.
- Graham