Missing last two frames when decoding 264 video

[DavidP]

Hi all,

Im having a problem decoding an entire h264 video. I decode all
IVideoPictures, EXCEPT the last two. Below is the code (scala) I use
to count the number of complete IVideoPictures. (Im using Xuggler 3.4)


The sample.mp4 is created by:
wget http://www.fastvdo.com/264codec/streams/qcif/FVDO_Freeway_qcif.264
ffmpeg -i FVDO_Freeway_qcif.264 -vcodec libx264 -vpre slower
sample.mp4

If you print out the information (MP4Box -info sample.mp4) about the
clip, I can confirm that the clip has 232 frames...
* Movie Info *
Timescale 1000 - Duration 00:00:09.220
Fragmented File no - 1 track(s)
File Brand isom - version 512
Created: GMT Fri Jan 2 00:00:00 1970
Track # 1 Info - TrackID 1 - TimeScale 50 - Duration 00:00:09.220
Media Info: Language "Undetermined" - Type "vide:avc1" - *******232
samples*******
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 176 x 144 - Profile High @ Level 1.2




The below program prints out 230! Where are the last two frames???
(I can confirm its the last two that are missing because I made JPG of
each IVideoPicture) What am I doing stupid??


import _root_.com.xuggle.xuggler._
import _root_.com.xuggle.xuggler.video._

object VideoDecoder2 extends Application with Loggable {
var count = 0

val container = IContainer.make();
container.open("sample.mp4", IContainer.Type.READ, null)

val stream = container.getStream(0)
val videoCoder = stream.getStreamCoder()
videoCoder.open()

val packet = IPacket.make();
while (container.readNextPacket(packet) >= 0) {
val picture = IVideoPicture.make(videoCoder.getPixelType(),
videoCoder.getWidth(), videoCoder.getHeight());

var offset = 0;
while (offset < packet.getSize()) {
val bytesDecoded = videoCoder.decodeVideo(picture, packet,
offset);
offset += bytesDecoded;
if (picture.isComplete()) {
count += 1
}
}
}

logger.info("" + count)
}

[DavidP]

Looks like this guy has the same problem... but no fix:
http://groups.google.com/group/xuggler-users/browse_thread/thread/21602f675d7e21a7?fwc=1

[Joe Areeda]

I have the same issue but in my case it's only 1 frame.
Using video editing software (Pitivi) I can see that all the frames are
in the file.
If you find the solution please post the mystic incantation that does it.

Joe

[DavidP]

My gut feeling is that, just like when you encode video, you need to
pass null IVideoPictures until the encoder returns a non-complete
IPacket - you need to pass null IPackets to the decoder until it
returns a non-complete IVideoPicture. BUT, this doenst work.... it
seems logical thought??? Any thoughts??

> >http://groups.google.com/group/xuggler-users/browse_thread/thread/216...

[Joe Areeda]

Well, all my ideas so far are just conjecture.

I'm just starting to dig into the details but like you I'm interested in
all the frames. I can get around it by adding a couple of blank frames
at the end of encoding process but that seems just wrong and I've cussed
and sworn at people who leave me code like that.

The only clue I have right now is that the position and size of the last
packet read do not add up to the full length of the file. The call to
container.readNextPacket(packet) is returning a negative value. I
wonder if there might be some partial buffer with data that's not
getting processed.

The fact that you're missing 2 frames is changing my theories. Are you
sure both frames are in the stream and readable? I'm just trying to
understand how the reading terminates.

Joe

[Joe Areeda]

I've been playing with the DecodeAndPlayVideo demo which I used as a model for my programming.

I can see what's happening but don't know how to fix it and would appreciate any help.

Two things to mention if you are also using that code as a model.

First the display is behind one frame so that may be why you are missing 2 Frames.  You can fix this by adding a paint call to the updateJavaWindow method so it looks like:

  private static void updateJavaWindow(BufferedImage javaImage)
  {
    mScreen.setImage(javaImage);
    mScreen.paint(mScreen.getGraphics());
  }

The main problem is that the last frame is never marked complete.

The main loop looks like:

while(container.readNextPacket(packet) >= 0)
{
    offset = 0;
    while(offset < packet.getSize())
    {
        bytesDecoded = videoCoder.decodeVideo(picture,packet,offset);

        offset += bytesDecoded;
        if (picture.isComplete())
        {

            ...
        }
    }
}

The problem seems to be that  the picture.isComplete() does not return true until the next picture is started.  In my test video there is one packet per image and each packet.isComplete() returns true.  However, the picture.isComplete() returns true on the next packet, leaving the last picture incomplete.

Any suggestions are more than welcome.  I am happy to make the instrumented DecodeAndPlayVideo.java and the test video (32KB) available if anyone wants to get their hands dirty.

Joe

[DavidP]

Yes, I am sure that the two frames are indeed in the video. If I
extract each frame into a JPG using ffmpeg directly on the
commandline, I see the missing frames. Ive done some more testing, and
the missing 2 frame problem happens only with h264 video. If the
video is mjpeg or anything else, I get all the frames from xuggler.
SO...... Hmmmm

On May 28, 12:50 pm, Joe Areeda <newsre...@areeda.com> wrote:
> Well, all my ideas so far are just conjecture.
>
> I'm just starting to dig into the details but like you I'm interested in
> all the frames. I can get around it by adding a couple of blank frames
> at the end of encoding process but that seems just wrong and I've cussed
> and sworn at people who leave me code like that.

I agree - but Im the guy who has to maintain the code - so even more
reason to make my life easy by doing it properly.

[Stas Oskin]

Have you tried using the tip of tree version?

It has numerous fixes since 3.4, and that issue might have just been handled.

Stas.

--
You received this message because you are subscribed to the Google Groups "xuggler-users" group.
To post to this group, send email to xuggle...@googlegroups.com.
To unsubscribe from this group, send email to xuggler-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xuggler-users?hl=en.

[DavidP]

Yes.. I checked out the latest source from:
http://xuggle.googlecode.com/svn/trunk/java/xuggle-xuggler
and build it locally.. The version of xuggler is:
SVN-r22292-xuggle-4.0.845

Does that look right?

[DavidP]

Sorry, this was the version Im running...

ffmpeg version UNKNOWN-xuggle-4.0.959, Copyright (c) 2000-2011 the
FFmpeg developers
built on May 29 2011 01:41:42 with gcc 4.6.0 20110513 (prerelease)
configuration: --prefix=/usr/local/xuggler --extra-
version=xuggle-4.0.959 --extra-cflags=-I/home/david/svn/xuggle-
xuggler3/build/native/x86_64-unknown-linux-gnu/captive/usr/local/
xuggler/include --extra-ldflags=-L/home/david/svn/xuggle-xuggler3/
build/native/x86_64-unknown-linux-gnu/captive/usr/local/xuggler/lib --
enable-shared --enable-gpl --enable-nonfree --enable-version3 --enable-
libfaac --enable-libx264 --enable-libmp3lame --enable-libvorbis --
enable-libtheora --enable-libspeex --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-
pthreads
libavutil 51. 3. 0 / 51. 3. 0
libavcodec 53. 6. 1 / 53. 6. 1
libavformat 53. 2. 0 / 53. 2. 0
libavdevice 53. 1. 0 / 53. 1. 0
libavfilter 2. 11. 0 / 2. 11. 0
libswscale 0. 14. 0 / 0. 14. 0
libpostproc 51. 2. 0 / 51. 2. 0
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile
options] outfile}...


Does that look right?

[DavidP]

:) Partial success.... after decoding the video until the
container.readNextPacket(p) is done (see above code) I run the
following HACK:

var picture = IVideoPicture.make(coder.getPixelType(),
coder.getWidth(), coder.getHeight());
packet.setComplete(true, 10000);
var bytesDecoded = videoCoder.decodeVideo(picture, packet, 0);
logger.info(bytesDecoded + " " + picture.isComplete())

Which infact prints out
[h264 @ 0x7f7960350660] reference picture missing during reorder
[h264 @ 0x7f7960350660] Missing reference picture
[h264 @ 0x7f7960350660] AVC: nal size -2087364433
[h264 @ 0x7f7960350660] illegal short term buffer state detected
2383 true
and the picture outputted is actually complets and is the second to
last... :) yippie! If I do this again, it doesnt return true... so,
Im missing the last frame still. :(

BUT, this does CONFIRM that we have the right *idea* - that there is
still data left in the IVideoCoder that needs to be outputted to more
IVidoePictures.

So close! We can figure it out!!!

[Joe Areeda]

I had to update my sources and recompile to do it but I now match what
you have exactly.

Joe

[harriso...@gmail.com]

Have you guys looked at either setting the rc_lookahead option for x264 via ffmpeg. By default it buffers 40 frames as I recall. So you might want to set this to zero or ensure you flush buffer continually after reading of input is finished

Sent from my BlackBerry® wireless device

[Joe Areeda]

Sounded promising but no luck.
videoCoder.getProperty returned the default of 40 before setProperty and
0 afterwards but still missing the last frame

Joe

[Joe Areeda]

David,

What language are you using? Looks like JavaScript (which I didn't know
was supported).

Perhaps that's why you're missing 2 frames and I'm missing one while
ffmpeg gets them all.

I tried that and I get an error with the IVideoPicture containing the
next to last image.

I will now see if I can understand the source code of ffmpeg enough to
figure out what it's doing differently.

Joe

[Dan Stieglitz]

How do you flush the input buffer when decoding? My understanding was
avcodec_flush_buffers() was not available via Xuggler

[harriso...@gmail.com]

Are you flushing the output buffers correctly. You must loop continually calling encodeVideo with a Null IVideoPicture parameter, writing the returned packet to the output repeat until the packet returned from encodeVideo is not complete

[Joe Areeda]

Harrison,
You are the second person to suggest writing nulls but I don't see how
to do it.

I am using IMediaWriter to create the output file. If I call
encodeVideo with a null IVideoPicture parameter it throws an
IllegalArgument exception. Also IMediaWriter.encodeVideo has a void
return type and nothing comes back.

I do call IContainer.flushPackets() and IMediaWriter.close().

The output file displays all frames in MoviePlayer(ubuntu), Pitivi,
ffmpeg and Windows Media Player so I have a hard time believing the
problem is in the writer, but I've been wrong about stuff like this more
times than I can count so I am open to suggestions.

Joe

[DavidP]

Hi Harrison,

I dont think its a problem on the encode side... the video I am trying
to decode really has all the frames. ffmpeg, MP4BOX and vlc all
report the correct number.

Its the xuggler h264 decoder... and I have a workaround posted below
(PS: the language Im using is www.scala-lang.org) The problem with
this workaround is a few things:
- packet size 10000 works... anything much smaller or larger doesnt
- you need to use the same packet as the previous call? A new packet
doesnt work
- there is no termination condition... I just loop twice.
- a bunch of warnings/errors printed from xuggler (but it works none-
the-less)

I think this somehow proves that there is some data left in the
decoder, which CAN be 'pushed' out to new completed IVideoPictures...

Any ideas on how to do this nicely??? Looks to me we need the same
pattern as passing null IPackets until the decoder returns a non-
complete IVideoPicture, as we do on the encoding side.

Cherrio,
david




object VideoDecoder extends Application with Loggable {

var count = 0

val container = IContainer.make();
container.open("sample.mp4", IContainer.Type.READ, null)

val stream = container.getStream(0)
val videoCoder = stream.getStreamCoder()
videoCoder.open()

var packet = IPacket.make();

while (container.readNextPacket(packet) >= 0) {
val picture = IVideoPicture.make(videoCoder.getPixelType(),
videoCoder.getWidth(), videoCoder.getHeight());

var offset = 0;
while (offset < packet.getSize()) {
val bytesDecoded = videoCoder.decodeVideo(picture, packet,
offset);
offset += bytesDecoded;
if (picture.isComplete()) {
count += 1
}
}
}

//HACK HERE TO GET THE LAST TWO FRAMES :(
for (i ← 0 to 1) {

val picture = IVideoPicture.make(videoCoder.getPixelType(),
videoCoder.getWidth(), videoCoder.getHeight());

packet.setComplete(true, 10000);
val bytesDecoded = videoCoder.decodeVideo(picture, packet, 0);
if (picture.isComplete) {

count += 1
}
}

logger.info("" + count)
}

On May 29, 1:45 pm, harrison.dec...@gmail.com wrote:
> Are you flushing the output buffers correctly.  You must loop continually calling encodeVideo  with a Null IVideoPicture parameter, writing the returned packet to the output repeat until the packet returned from encodeVideo is not complete
>
> Sent from my BlackBerry® wireless device
>
>
>
>
>
>
>
> -----Original Message-----
> From: Joe Areeda <newsre...@areeda.com>
>
> Sender: xuggle...@googlegroups.com
> Date: Sun, 29 May 2011 10:41:48
> To: <xuggle...@googlegroups.com>
> Reply-To: xuggle...@googlegroups.com
> Subject: Re: [xuggler-users] Re: Missing last two frames when decoding 264 video
>
> Sounded promising but no luck.
> videoCoder.getProperty returned the default of 40 before setProperty and
> 0 afterwards but still missing the last frame
>
> Joe
>

[Stas Oskin]

Hi.

Can you submit a patch for this work-around, and open a Google code ticket?

Art will be happy to include this.

Stas.

[DavidP]

Hey Joe,

Im using Scala (awesome new language)... Its a JVM language, so its
exactly the same as java when things compile to byte code. I dont
think it has anything to do with the language... my guess is that its
the size of each frame, the encoding settings or even the type of
frame being decoded (B-P-I frames) that is different. The data being
'left' in the decoder varies, thus the # of missing frames varies
accordingly.

Regards,
dav

[Stas Oskin]

Hi Dave.

Off-topic, how do you find Scala for working with Xuggler?

I tried it a bit and it seem to really rock, much more suitable for modern programming then plain old Java.

But I wonder how it works with Xuggler, and if there any performance-wise issues?

Stas.

[DavidP]

This hack is not something that makes sense to patch... I think all it
proves is that the data is actually left in the decoder. That's all.
I have *not* looked into the xuggler code in detail - I assume the
correct patch would be located somewhere in csrc.

On May 30, 12:26 am, Stas Oskin <stas.os...@gmail.com> wrote:
> Hi.
>
> Can you submit a patch for this work-around, and open a Google code ticket?
>
> Art will be happy to include this.
>
> Stas.
>
>
>
>
>
>
>
> On Mon, May 30, 2011 at 10:23 AM, DavidP <daviddphill...@gmail.com> wrote:
> > Hi Harrison,
>
> > I dont think its a problem on the encode side... the video I am trying
> > to decode really has all the frames.  ffmpeg, MP4BOX and vlc all
> > report the correct number.
>
> > Its the xuggler h264 decoder... and I have a workaround posted below

> > (PS: the language Im using iswww.scala-lang.org)  The problem with

[Joe Areeda]

That makes sense, I have heard of Scala and didn't realized it uses the
same libraries and JVM. So I agree the difference is probably data
dependent. If you'd like to try my test file you can download it from
http://www.areeda.com/tmp/10counter.mp4 it's about 31KB and has 10
frames that just have numbers 1-10.

I still can't get the hack you suggest to have any effect.

We can figure out the amount of data unprocessed by the position and
size of the last packet processed and the size of the file.

Every thing I've tried seems to suggest the packet that returns the
error (for me) is empty or not valid.

The error code returned by container.readNextPacket(packet) is not a
valid error number for me, it's -541478725 which is type and description
unknown.

Joe

[DavidP]

Ive been using Scala for ~6months; however, it took me a good 4months
of using it before I actually believed it was worth it (better than
java). It initially feels more complex, less readable and way worse
tool support - however, *once* your eye starts understanding the
syntax, its just better! The tools support is much better with
2.9.0. Together with Xuggler, its been no problem - good
performance. Yourkit profiler works perfectly with scala, and as
expected most of the time is spend in encode/decode/colourspace

Cheers
dav

[DavidP]

I tried your 10counter.mp4 file, and I miss *one* frame without my
hack; however, the hack works.

Wierd... my hack works in my little test program; however, I cant get
it to actually work in my 'real' program. It never returns a
completed picture.... hmmmm

On May 30, 8:38 am, Joe Areeda <newsre...@areeda.com> wrote:
> That makes sense, I have heard of Scala and didn't realized it uses the
> same libraries and JVM.  So I agree the difference is probably data

> dependent.  If you'd like to try my test file you can download it fromhttp://www.areeda.com/tmp/10counter.mp4it's about 31KB and has 10

[Joe Areeda]

Fascinating. I'm pretty sure I understand what you're doing but haven't
had any luck.
I'll keep trying.

[DavidP]

Hey yippie!

I think I found some documentation in ffmpeg that supports passing
null packets to the decoder to get the last frames :)

http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/libavcodec_2utils_8c-source.html#l00691
http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/libavcodec_2utils_8c.html#eb2441bb9d76c881fc86a0085b2f13d2
function: avcodec_decode_video2


int attribute_align_arg avcodec_decode_video2 (
AVCodecContext * avctx,
AVFrame * picture,
int * got_picture_ptr,
AVPacket * avpkt

Decode the video frame of size avpkt->size from avpkt->data into
picture.

Some decoders may support multiple frames in a single AVPacket, such
decoders would then just decode the first frame.

Note: You might have to align the input buffer avpkt->data. The
alignment requirements depend on the CPU: on some CPUs it isn't
necessary at all, on others it won't work at all if not aligned and on
others it will work but it will have an impact on performance.

In practice, avpkt->data should have 4 byte alignment at minimum.

Note: Some codecs have a delay between input and output, these need to
be fed with avpkt->data=NULL, avpkt->size=0 at the end to return the
remaining frames.

Parameters:
avctx the codec context
[out] picture The AVFrame in which the decoded video frame will
be stored. Use avcodec_alloc_frame to get an AVFrame, the codec will
allocate memory for the actual bitmap. with default get/
release_buffer(), the decoder frees/reuses the bitmap as it sees fit.
with overridden get/release_buffer() (needs CODEC_CAP_DR1) the user
decides into what buffer the decoder decodes and the decoder tells the
user once it does not need the data anymore, the user app can at this
point free/reuse/keep the memory as it sees fit.
[in] avpkt The input AVpacket containing the input buffer. You
can create such packet with av_init_packet() and by then setting data
and size, some decoders might in addition need other fields like
flags&AV_PKT_FLAG_KEY. All decoders are designed to use the least
fields possible.
[in,out] got_picture_ptr Zero if no frame could be decompressed,
otherwise, it is nonzero.

Returns:
On error a negative value is returned, otherwise the number of
bytes used or zero if no frame could be decompressed.

[DavidP]

Ive make a patch that properly fixes the issue. It basically changes
the code to allow null packets to be passed in, instead of throwing
exceptions.




Index: csrc/com/xuggle/xuggler/StreamCoder.cpp
===================================================================
--- csrc/com/xuggle/xuggler/StreamCoder.cpp (revision 1081)
+++ csrc/com/xuggle/xuggler/StreamCoder.cpp (working copy)
@@ -856,6 +856,8 @@
StreamCoder::decodeVideo(IVideoPicture *pOutFrame, IPacket *pPacket,
int32_t byteOffset)
{
+VS_LOG_WARN("DAVE DEBUG VERSION 3");
+
int32_t retval = -1;
VideoPicture* frame = dynamic_cast<VideoPicture*> (pOutFrame);
Packet* packet = dynamic_cast<Packet*> (pPacket);
@@ -867,10 +869,6 @@
VS_LOG_WARN("Attempting to decode when not ready; no frame");
return retval;
}
- if (!packet) {
- VS_LOG_WARN("Attempting to decode when not ready; no packet");
- return retval;
- }
if (!mOpened) {
VS_LOG_WARN("Attempting to decode when not ready; codec not
opened");
return retval;
@@ -895,20 +893,11 @@
AVFrame *avFrame = avcodec_alloc_frame();
if (avFrame)
{
- RefPointer<IBuffer> buffer = packet->getData();
+ RefPointer<IBuffer> buffer = 0;
int frameFinished = 0;
int32_t inBufSize = 0;
uint8_t * inBuf = 0;

- inBufSize = packet->getSize() - byteOffset;
-
- VS_ASSERT(buffer, "no buffer in packet?");
- if (buffer)
- inBuf = (uint8_t*) buffer->getBytes(byteOffset, inBufSize);
-
- VS_ASSERT(inBuf, "incorrect size or no data in packet");
-
- if (inBufSize > 0 && inBuf)
{
VS_LOG_TRACE("Attempting decodeVideo(%p, %p, %d, %p, %d);",
mCodecContext,
@@ -916,15 +905,24 @@
frameFinished,
inBuf,
inBufSize);
+
AVPacket pkt;
av_init_packet(&pkt);
if (packet && packet->getAVPacket())
pkt = *packet->getAVPacket();
+
// copy in our buffer
+ if(packet) {
+ buffer = packet->getData();
+ inBufSize = packet->getSize() - byteOffset;
+ if (buffer)
+ inBuf = (uint8_t*) buffer->getBytes(byteOffset,
inBufSize);
+
+ mCodecContext->reordered_opaque = packet->getPts();
+ }
pkt.data = inBuf;
pkt.size = inBufSize;

- mCodecContext->reordered_opaque = packet->getPts();
retval = avcodec_decode_video2(mCodecContext, avFrame,
&frameFinished,
&pkt);
VS_LOG_TRACE("Finished %d decodeVideo(%p, %p, %d, %p, %d);",
@@ -934,6 +932,7 @@
frameFinished,
inBuf,
inBufSize);
+
if (frameFinished)
{
// copy FFMPEG's buffer into our buffer; don't try to get
efficient
@@ -948,10 +947,10 @@
int64_t packetTs = avFrame->reordered_opaque;
// if none, assume this packet's decode time, since
// it's presentation time should have been in
reordered_opaque
- if (packetTs == Global::NO_PTS)
+ if (packet && packetTs == Global::NO_PTS)
packetTs = packet->getDts();

- if (packetTs != Global::NO_PTS)
+ if (packet && packetTs != Global::NO_PTS)
{
if (timeBase->getNumerator() != 0)
{

[Joe Areeda]

Interesting.
I'm not sure we have that kind of control through the Java interfaces
though.

I think AV_PKT_FLAG_KEY is the Key Frame flag we can set.
I tried creating a new IPacket and setting it as a key frame, complete
with size 0 and trying to decode it several times without luck.

When we figure this out, I don't think the "fix" belongs in top level
code. I hope we can come up with a flushInput call some where lower
level, either in the Java interface or native library if necessary.

Joe

[Joe Areeda]

Excellent, I'll give it a try as soon as I get done sucking up to my
wife for the holidays.

thank you for your perseverance.

Joe

[Joe Areeda]

I'm having trouble applying your patch probably because my mail reader
is wrapping lines, could be my own stupidity.

Would you please add it as an attachment if this allows them or post it
somewhere?

Joe

On 05/30/2011 10:26 AM, DavidP wrote:

[DavidP]

http://pastebin.com/jBa4Gucc

Art... could you take a look at this patch, and apply it to
xuggle :)
Thanks a million - xuggler is great!

[DavidP]

Hi all,

Ive solved this problem by applying the attached patch, and calling:
do {
  videoCoder.decodeVideo(picture, null, 0) until
} while(picture.isComplete)

I *think* this makes sense as its necessary to pass null pictures while *encoding* to get the last few packets out, and thus it makes sense to pass null packets to get the last few pictures out!

Art, can you review and commit this?

Many thanks,
david

[Art Clarke]

I will do (probably next week), but do you by any chance also have a unit test that would show the problem before and after?

--

You received this message because you are subscribed to the Google Groups "xuggler-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/xuggler-users/-/80bDbXbmQk4J.

To post to this group, send email to xuggle...@googlegroups.com.
To unsubscribe from this group, send email to xuggler-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/xuggler-users?hl=en.

--
http://www.xuggle.com/
xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio and video.

Use Xuggle to get the power of FFmpeg in Java.

[Art Clarke]

Actually this seems very similar to Jakob's patches that he was working on (and which are more complete).  Jakob, were you able to finish the last work you wanted to do in your tree?

- Art

[Jakub Stachowski]

Yes. Patches for flushing both audio and video are complete. I have
created pull request for them.

W dniu 2012-04-25 21:50, Art Clarke pisze:

> <mailto:xuggle...@googlegroups.com>.

> To unsubscribe from this group, send email to
> xuggler-user...@googlegroups.com

> <mailto:xuggler-users%2Bunsu...@googlegroups.com>.

> For more options, visit this group at
> http://groups.google.com/group/xuggler-users?hl=en.
>
>
>
>
> --
> http://www.xuggle.com/
> xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio
> and video.
>
> Use Xuggle to get the power of FFmpeg in Java.
>
>
>
>
> --
> http://www.xuggle.com/
> xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio and
> video.
>
> Use Xuggle to get the power of FFmpeg in Java.
>

> --
> You received this message because you are subscribed to the Google
> Groups "xuggler-users" group.

[TazmanianD]

I know this thread is a bit old but could someone offer a summary of the current status of this problem? I seem to have encountered this as well although in my case I'm losing a full 10 frames off the end of my video (and I've noticed the the timestamp of the first decoded frame is also 10 - firstTimestampInStream in the example). I'm running the latest version that's on the website (5.4). Should the patches mentioned above already be included in what's been compiled and posted? If not, do I need to apply the patch and build xuggle myself? It's also a little unclear to me exactly what I need to do (if anything) to get the final frames. Do I need to continue calling decodeVideo after reading the last packet?

[TazmanianD]

Hmmm, I've discovered that I have the same problem with video encoded with Huffyuv v2.1.1 although it's dropping 8 frames. I can understand H.264 content because there are future frames to deal with but why would Huffyuv have that problem? If I save the video uncompressed, I get all the frames.

[TazmanianD]

I think I figured it out. It seems that the decoder somewhere is maintaining a buffer of frames and so when you read packets, it's always behind except that when it gets to the end of the stream, it just stops without returning those final frames (a bug in Xuggler?). However to get at them, all you have to do is execute a seek operation because all the frames that are decoded immediately following a seek are actually the buffered frames that had been decoded previously (which means that doing a seek, doesn't immediately put you where you expect).

As an example, if we assume a video with 500 frames (0-499) and an offset/buffer of 10 frames, the last frame that will be read before the reader stops is 489. If you want frame 490, you have to seek to frame 489 (seek to the nearest keyframe and then read up and including 489). Then just seek to frame 0. The very next frame decoded from the next packet will be frame 490 and 491 after that, etc. Of course if you actually wanted frame 0 from that seek, you'll have to read and throw away frames 490 through 499 before you'll get to frame 0 from the seek.

[declan harrison]

you need to flush out the decoded frames from the stream coder, passing in a null as per the encoder, stream coder. This has to do with delayed frames.

iStreamCoder.encodeVideo(iPacket, null, 0);

depending on the version the decoder maynot allow passing the syntax above like the encodeVideo does

On Fri, Sep 12, 2014 at 3:07 AM, TazmanianD <damon....@gmail.com> wrote:

I think I figured it out. It seems that the decoder somewhere is maintaining a buffer of frames and so when you read packets, it's always behind except that when it gets to the end of the stream, it just stops without returning those final frames (a bug in Xuggler?). However to get at them, all you have to do is execute a seek operation because all the frames that are decoded immediately following a seek are actually the buffered frames that had been decoded previously (which means that doing a seek, doesn't immediately put you where you expect).

As an example, if we assume a video with 500 frames (0-499) and an offset/buffer of 10 frames, the last frame that will be read before the reader stops is 489. If you want frame 490, you have to seek to frame 489 (seek to the nearest keyframe and then read up and including 489). Then just seek to frame 0. The very next frame decoded from the next packet will be frame 490 and 491 after that, etc. Of course if you actually wanted frame 0 from that seek, you'll have to read and throw away frames 490 through 499 before you'll get to frame 0 from the seek.

--

You received this message because you are subscribed to the Google Groups "xuggler-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to xuggler-user...@googlegroups.com.

To post to this group, send email to xuggle...@googlegroups.com.

Visit this group at http://groups.google.com/group/xuggler-users.
For more options, visit https://groups.google.com/d/optout.

[Matthew Townsend]

I've found a fix. After parsing all the packets of the video, seek back to the beginning and keep using the decoder until picture.getTimeStamp() is less than it was on the frame before.

As I go through the readNextPacket() loop I call getTimeStamp() inside the picture.isComplete() clause and save that value.

Then, after the loop ends once, I call container.seekKeyFrame(videoStreamId, 0, 0, 0, 0), and start the loop again. This time you're doing the same thing, except at some point picture.getTimeStamp() is going to return a value less than it did on the previous frame because you'll have started processing from the beginning again.

On Saturday, May 28, 2011 at 4:26:58 AM UTC-7, DavidP wrote:

Hi all,

Im having a problem decoding an entire h264 video.  I decode all
IVideoPictures, EXCEPT the last two.  Below is the code (scala) I use
to count the number of complete IVideoPictures. (Im using Xuggler 3.4)


The sample.mp4 is created by:
   wget http://www.fastvdo.com/264codec/streams/qcif/FVDO_Freeway_qcif.264
   ffmpeg -i FVDO_Freeway_qcif.264 -vcodec libx264 -vpre slower
sample.mp4

If you print out the information (MP4Box -info  sample.mp4) about the
clip, I can confirm that the clip has 232 frames...
* Movie Info *
        Timescale 1000 - Duration 00:00:09.220
        Fragmented File no - 1 track(s)
        File Brand isom - version 512
        Created: GMT Fri Jan  2 00:00:00 1970
Track # 1 Info - TrackID 1 - TimeScale 50 - Duration 00:00:09.220
Media Info: Language "Undetermined" - Type "vide:avc1" - *******232
samples*******
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x21
AVC/H264 Video - Visual Size 176 x 144 - Profile High @ Level 1.2




The below program prints out 230!  Where are the last two frames???
(I can confirm its the last two that are missing because I made JPG of
each IVideoPicture)  What am I doing stupid??


import _root_.com.xuggle.xuggler._
import _root_.com.xuggle.xuggler.video._

object VideoDecoder2  extends Application  with Loggable {

  var count = 0

  val container = IContainer.make();
  container.open("sample.mp4", IContainer.Type.READ, null)

  val stream = container.getStream(0)
  val videoCoder = stream.getStreamCoder()
  videoCoder.open()

  val packet = IPacket.make();

  while (container.readNextPacket(packet) >= 0) {
    val picture = IVideoPicture.make(videoCoder.getPixelType(),
videoCoder.getWidth(), videoCoder.getHeight());

    var offset = 0;
    while (offset < packet.getSize()) {
      val bytesDecoded = videoCoder.decodeVideo(picture, packet,
offset);
      offset += bytesDecoded;
      if (picture.isComplete()) {
        count += 1
      }
    }
  }

  logger.info("" + count)
}

[Matthew Townsend]

In order to do a similar thing with the decoder, you can call seekKeyFrame(id, 0, 0, 0, 0) on the container and read more packets until a decoded, complete IVideoPicture has a getTimeStamp() less than the last one.