Help with timebase and framerate (again)

[jez]

OK, this isn't too serious a question as things seem to be working as
they are now, but I just want to check that IO'm doing this right
anyway.

I'm receiving MPEG4 video data in at 10FPS, and encoding it out to
Sorenson Spark at 10FPS. Now, do I set the input and output stream
coder framerates equal to IRational.make(10, 1) and the timebases
equal to IRational.make(1, 10)? Is it that simple? (I'm doing that
now and it seems to work but I'm not sure whether these values are
meant to be in millisecs or what)

Best regards,
Jeremy Morton (Jez)

[Art Clarke]

I was wondering when this question would come :) And unfortunately I
fear this response won't clear it up.

To be honest when we first wrote the API we didn't fully understand
what a TimeBase was -- we just knew they needed to be there. Now we
have a better understanding, but it's still complicated.

For IStream objects, the TimeBase is the units that time stamps
increment with in the containing IContainer. An IStream is ALWAYS
associated with an IContainer. This TimeBase differs depending on the
IContainerFormat. For example, in Flash's FLV format, timestamps are
always in units of milliseconds, so an IStream in a FLV IContainer
will have a time base of 1/1000. For MPEG4 format, timestamps are in
units of 1/90000 seconds, and so an IStream is 1/90000.

For IStreamCoder objects, the TimeBase will differ depending on the
Codec and the Container. Usually (especially when decoding) it'll be
1/(frame-rate). So if you have video at 30 frames per second, the
TimeBase on the IStreamCoder will be 1/30.

For encoding, you should set TimeBase on the IStreamCoder to whatever
your frame rate is. For example for video 30-fps is 1/30, for audio,
22050 khz sample rate is 1/22050.

Now Xuggler, when you're calling IStreamCoder.encode..., will
translate between the time stamps that are in IStreamCoder's timebase
into the timebase of the IStream being written to if known. That
means if you're encoding video at 15 fps into a FLV container, and
let's say you're on the 2nd frame (starting at 0), by default
encodeAudio will first set a timetstamp of '2' on the packet when the
StreamCoder finishes encoding, but the StreamCoder will then look at
the attached IStream (if it was gotten from an IStream) and translate
"2" at 1/15 timebase into "1333" at 1/1000 timebase.

Lastly, IPacket timestamps are always in the time-base of the IStream
they were read from. If encoded with an IStreamCoder that is attached
to an IStream, they will be in the IStream's time base, otherwise
they'll be in the IStreamCoder's timebase. IVideoPictures and
IAudioSamples are ALWAYS in microseconds (or a timebase of 1/1000000)
because frankly, it makes things much easier then.

So much for theory. Here's what to do:

1) For decoding, in most cases you can ignore TimeBases as we'll
figure them out. In your case though since we have no headers, set
your IStreamCoder timebase to 1/10 to emulate the input frame rate.

2) For encoding, in most cases the IStream timebase from
IContainer.addNewStream(int) will be set correctly based on the set or
guesses IContainerFormat of the output IContainer, but you'll need to
set IStreamCoder.setTimeBase(IRational). Set it to 1 / frame-rate
(*).

- Art

*) You may ask why don't we always set TimeBase to 1 / frame-rate, and
I'll respond with (a) some codecs pay attention to frame-rate and some
to time base but more importantly (b) for variable frame-rate video
codecs (yup, they exist), frame-rate is the approximate effective
frame-rate, but TimeBase is the unit you want the IStreamCoder to
assume incremental timestamps are in. Don't worry about that for now.

--
http://www.xuggle.com/
xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio and video.

Use Xuggle to get the power of FFMPEG in Java.

[jez]

On Apr 8, 4:03 pm, Art Clarke <acla...@xuggle.com> wrote:
> I was wondering when this question would come  :)  And unfortunately I
> fear this response won't clear it up.

Not sure why that is... lots to grasp but this info's been pretty
helpful. Looks like I lucked out and set all I needed to correctly
with SteamStream 1.0, with one exception...

> So much for theory.  Here's what to do:
>
> 1) For decoding, in most cases you can ignore TimeBases as we'll
> figure them out.  In your case though since we have no headers, set
> your IStreamCoder timebase to 1/10 to emulate the input frame rate.

I don't set the input coder timebase at all, only its framerate (to
10/1)... but it still works anyway. Why's this?

Also,

> For IStream objects, the TimeBase is the units that time stamps
> increment with in the containing IContainer. An IStream is ALWAYS
> associated with an IContainer. This TimeBase differs depending on the
> IContainerFormat. For example, in Flash's FLV format, timestamps are
> always in units of milliseconds, so an IStream in a FLV IContainer
> will have a time base of 1/1000. For MPEG4 format, timestamps are in
> units of 1/90000 seconds, and so an IStream is 1/90000.

So I don't ever need to set the timebase for an IStream, right? It
will be automatically set by Xuggler when I get it via
IContainer#getStream?

And finally, an unrelated question I just thought of...
Why does most of your example code do IStreamCoder#setGlobalQuality
(0)? That's but one of the things you can set on an IStreamCoder,
seems a bit random, and doesn't seem to make any difference if you
remove it.

[Art Clarke]

On Thu, Apr 9, 2009 at 2:25 AM, jez <jez...@gmail.com> wrote:
>
> On Apr 8, 4:03 pm, Art Clarke <acla...@xuggle.com> wrote:
>> I was wondering when this question would come  :)  And unfortunately I
>> fear this response won't clear it up.
>
> Not sure why that is... lots to grasp but this info's been pretty
> helpful.  Looks like I lucked out and set all I needed to correctly
> with SteamStream 1.0, with one exception...

It's just complicated -- thanks to the wide variety of ways that
containers implement timestamps.

>
>> So much for theory.  Here's what to do:
>>
>> 1) For decoding, in most cases you can ignore TimeBases as we'll
>> figure them out.  In your case though since we have no headers, set
>> your IStreamCoder timebase to 1/10 to emulate the input frame rate.
>
> I don't set the input coder timebase at all, only its framerate (to
> 10/1)... but it still works anyway.  Why's this?

I think it depends. In your case you don't always have the
meta-information about your stream, and so the IStreamCoder isn't able
to guess all items for input (for a on-disk MOV file for example, you
don't even need to set the Frame Rate when reading -- we'll guess it
from the MOOV atom).

> So I don't ever need to set the timebase for an IStream, right?  It
> will be automatically set by Xuggler when I get it via
> IContainer#getStream?

Yes, if we know the ContainerFormat, then getStream(...) and
addNewStream(...) should set the right time base.

>
> And finally, an unrelated question I just thought of...
> Why does most of your example code do IStreamCoder#setGlobalQuality
> (0)?  That's but one of the things you can set on an IStreamCoder,
> seems a bit random, and doesn't seem to make any difference if you
> remove it.

It's ignored by some codecs (e.g. h.264), but the mpeg4 video codec
pays attention to it. To see it in action, try converting a file to a
Quicktime MOV with mpeg4 video. With the setGlobalQuality() ffmpeg
chooses a lower quality and the video looks degrated.

- Art

[Stas Oskin]

Hi.

>
> And finally, an unrelated question I just thought of...
> Why does most of your example code do IStreamCoder#setGlobalQuality
> (0)?  That's but one of the things you can set on an IStreamCoder,
> seems a bit random, and doesn't seem to make any difference if you
> remove it.

It's ignored by some codecs (e.g. h.264), but the mpeg4 video codec
pays attention to it.  To see it in action, try converting a file to a
Quicktime MOV with mpeg4 video.  With the setGlobalQuality() ffmpeg
chooses a lower quality and the video looks degrated.

 

Interesting, so the setGlobalQuality has to be set to 0, in order to provide the best quality on MPEG4 codec?

Can you tell what fields it sets in the FFMPEG itself? I mean, which code does this function calls?

Thanks!

[Art Clarke]

> Interesting, so the setGlobalQuality has to be set to 0, in order to provide
> the best quality on MPEG4 codec?
> Can you tell what fields it sets in the FFMPEG itself? I mean, which code
> does this function calls?

Assuming:
AVCodecContext *foo;

I set:
foo->global_quality

to what you set. Beyond that, take a look in ffmpeg/libavcodec and
see who uses 'global_quality'.

- Art

p.s. our code for setting this is in:
csrc/com/xuggle/xuggler/StreamCoder.cpp:500

[Stas Oskin]

Hi.

to what you set.  Beyond that, take a look in ffmpeg/libavcodec and
see who uses 'global_quality'.

I checked the latest svn, and it seems only being used in some still rare codecs as Dirac. The only relation to MPEG4 was the file mpegvideo_enc, where it only was present in debug line, which also was commented:
//printf("%d %d\n", s->avctx->global_quality, s->current_picture.quality);

Moreover, according to options.c file, which contains the default parameters for all of the settings, it seems to be set to 0 anyway:

{"global_quality", NULL, OFFSET(global_quality), FF_OPT_TYPE_INT, DEFAULT, INT_MIN, INT_MAX}, // DEFAULT is 0

Any idea if it's really still used anywhere?

Regards.
 

[Art Clarke]

On Fri, Apr 10, 2009 at 7:58 AM, Stas Oskin <stas....@gmail.com> wrote:
>
> Any idea if it's really still used anywhere?
>

Looks like not then. It was used back when we originally wrote the
code. We'll deprecate it.

- Art

[jez]

On Apr 10, 4:43 pm, Art Clarke <acla...@xuggle.com> wrote:

> On Fri, Apr 10, 2009 at 7:58 AM, Stas Oskin <stas.os...@gmail.com> wrote:
>
> > Any idea if it's really still used anywhere?
>
> Looks like not then.  It was used back when we originally wrote the
> code.  We'll deprecate it.

Unless ffmpeg is deprecating it also maybe you should just explain
this in the Javadocs instead.

[Stas Oskin]

From my checks, it always 0 as it's initialized by the options.c. It is the same way even in the version of 1/2 years ago.

So I think it pretty safe to remove  it - caveat emptor of course :)

2009/4/13 jez <jez...@gmail.com>

[jez]

Removing it breaks my code as I set it. :-)

Best regards,
Jeremy Morton (Jez)

On Apr 13, 10:45 am, Stas Oskin <stas.os...@gmail.com> wrote:
> From my checks, it always 0 as it's initialized by the options.c. It is the
> same way even in the version of 1/2 years ago.
>
> So I think it pretty safe to remove  it - caveat emptor of course :)
>

> 2009/4/13 jez <jez9...@gmail.com>

[Art Clarke]

I said deprecate it, not remove it. Although deprecation is a warning
it'll be removed in a future release.

- Art

[jez]

I don't think it should be removed at all, unless ffmpeg removes it.

What about SimpleMediaFile#setVideoGlobalQuality()? Is that to be
deprecated too?

Best regards,
Jeremy Morton (Jez)

On Apr 13, 3:40 pm, Art Clarke <acla...@xuggle.com> wrote:
> I said deprecate it, not remove it.  Although deprecation is a warning
> it'll be removed in a future release.
>
> - Art
>
>
>
> On Mon, Apr 13, 2009 at 3:04 AM, jez <jez9...@gmail.com> wrote:
>
> > Removing it breaks my code as I set it.  :-)
>
> > Best regards,
> > Jeremy Morton (Jez)
>
> > On Apr 13, 10:45 am, Stas Oskin <stas.os...@gmail.com> wrote:
> >> From my checks, it always 0 as it's initialized by the options.c. It is the
> >> same way even in the version of 1/2 years ago.
>
> >> So I think it pretty safe to remove  it - caveat emptor of course :)
>
> >> 2009/4/13 jez <jez9...@gmail.com>
>
> >> > On Apr 10, 4:43 pm, Art Clarke <acla...@xuggle.com> wrote:
> >> > > On Fri, Apr 10, 2009 at 7:58 AM, Stas Oskin <stas.os...@gmail.com>
> >> > wrote:
>
> >> > > > Any idea if it's really still used anywhere?
>
> >> > > Looks like not then.  It was used back when we originally wrote the
> >> > > code.  We'll deprecate it.
>
> >> > Unless ffmpeg is deprecating it also maybe you should just explain
> >> > this in the Javadocs instead.
>
> >> > Best regards,
> >> > Jeremy Morton (Jez)
>

> --http://www.xuggle.com/

[Art Clarke]

If/when we deprecate setGlobalQuality we'll also deprecate the
SimpleMediaFile interface.

As for when to remove it, it makes sense to wait for ffmpeg to
officially deprecate it first.

- Art

--

[Stas Oskin]

Hi.

I might have misunderstood, or what's worse, mis-instructed :).

What I meant was that global quality *seems* not be used in mpeg4 encoding, and moreover, *seems* to be always set to 0 anyway. But it does get used by other encoders, such as dirac, aac and bunch of others.

So, my question / advice was whether we really need to use the setVideoGlobalQuality(0) during initialization, or we can get away by using default values, for MPEG4 encoding only.

Because of the other uses above, I think the function should be kept.

Regards,
Stas.

2009/4/14 jez <jez...@gmail.com>