Needed: Video Decoding Unit Tests

[Nathan]

FIRST, does anyone know how/where I could find out what video formats
AVbin is _supposed_ to support? I'll search around in the ffmpeg
source, but I'm not sure what I'm even looking for, or where I'd look
for it.

SECOND, what video formats are people actually using pyglet to decode?

THIRD, can anyone donate a nice, small 3-second video with good sound
and a clear picture in some raw format that we could use to transcode
to different formats to make video test files for the varios formats?

FOURTH, can anyone help with the above? I'm not a video guy...

From what I can tell, the only test cases for the media module are
four attempts to play generated noise (white noise or sine waves).
This seems to be a glaring omission in an otherwise extensive set of
unit tests. If we can get some unit tests up and running, that will
go a long way in maintaining AVbin.

~ Nathan

[Richard Jones]

On 13 November 2011 17:05, Nathan <nathan...@gmail.com> wrote:
> FIRST, does anyone know how/where I could find out what video formats
> AVbin is _supposed_ to support?  I'll search around in the ffmpeg
> source, but I'm not sure what I'm even looking for, or where I'd look
> for it.

I don't believe there is such a list. If you don't mind me asking;
what's the purpose of the list? Is it that avbin can't support all the
ffmpeg formats? Or that there's no clear list of even the ffmpeg
supported formats?

Richard

[Nathan]

I don't mind at all. The purpose would be to know which formats to
transcode a test video to so that we can see if pyglet/avbin can
actually decode said formats. And then make unit tests that actually
exercise all that so that whenever we make a change to avbin, we can
see if we inadvertently broke one of the formats.

Not to mention, it would be nice to know when others come with bug
reports. For example, when Dmitry Chichkov recently reported problems
decoding a vp6f video file, I had no idea whether that was even a
supported format!

~ Nathan

[Rémy Sanchez]

On Sunday 13 November 2011 07:05:21 Nathan wrote:
> FIRST, does anyone know how/where I could find out what video formats

> AVbin is supposed to support? I'll search around in the ffmpeg

> source, but I'm not sure what I'm even looking for, or where I'd look
> for it.

"ffmpeg -formats" looks like a good place to start. The output will probably
depend on the compilation options though.

> SECOND, what video formats are people actually using pyglet to decode?

If I can make a suggestion, don't support too much formats and concentrate on
free stuffs with few IP problems, let's say WebM for instance. Just my
opinion...

--
Rémy Sanchez
http://hyperthese.net/

[Nathan]

On Sun, Nov 13, 2011 at 9:35 PM, Rémy Sanchez
<remy.s...@hyperthese.net> wrote:
> On Sunday 13 November 2011 07:05:21 Nathan wrote:
>> FIRST, does anyone know how/where I could find out what video formats
>> AVbin is supposed to support?  I'll search around in the ffmpeg
>> source, but I'm not sure what I'm even looking for, or where I'd look
>> for it.
>
> "ffmpeg -formats" looks like a good place to start. The output will probably
> depend on the compilation options though.

Interesting idea. Unfortunately, it looks like the build process of
ffmpeg from within avbin is completely customized, and no ffmpeg
binary is produced. Perhaps I will be able to find a way to get the
binary produced if I muck around in the build process for awhile.

>> SECOND, what video formats are people actually using pyglet to decode?
>
> If I can make a suggestion, don't support too much formats and concentrate on
> free stuffs with few IP problems, let's say WebM for instance. Just my
> opinion...

Well, I agree with the principle, but *I'm* not really supporting
formats or not. FFmpeg does...or not. AVbin is just a light wrapper
for some FFmpeg functions. What I'm really interested in here is what
video formats people are actually using, so that I can prioritize
efforts to make the more relevant unit tests first.

~ Nathan

[Rémy Sanchez]

On Mon, 14 Nov 2011 22:33:02 -0700, Nathan <nathan...@gmail.com>
wrote:

> On Sun, Nov 13, 2011 at 9:35 PM, Rémy Sanchez
> <remy.s...@hyperthese.net> wrote:
>> On Sunday 13 November 2011 07:05:21 Nathan wrote:
>>> FIRST, does anyone know how/where I could find out what video
>>> formats
>>> AVbin is supposed to support?  I'll search around in the ffmpeg
>>> source, but I'm not sure what I'm even looking for, or where I'd
>>> look
>>> for it.
>>
>> "ffmpeg -formats" looks like a good place to start. The output will
>> probably
>> depend on the compilation options though.
>
> Interesting idea. Unfortunately, it looks like the build process of
> ffmpeg from within avbin is completely customized, and no ffmpeg
> binary is produced. Perhaps I will be able to find a way to get the
> binary produced if I muck around in the build process for awhile.

Well, you'll find how to list the formats in the function behind that
(cmdutils.c, function "int opt_formats(const char *opt, const char
*arg)"). The codecs registration looks pretty static though :
http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/allformats_8c-source.html

>>> SECOND, what video formats are people actually using pyglet to
>>> decode?
>>
>> If I can make a suggestion, don't support too much formats and
>> concentrate on
>> free stuffs with few IP problems, let's say WebM for instance. Just
>> my
>> opinion...
>
> Well, I agree with the principle, but *I'm* not really supporting
> formats or not. FFmpeg does...or not. AVbin is just a light wrapper
> for some FFmpeg functions. What I'm really interested in here is
> what
> video formats people are actually using, so that I can prioritize
> efforts to make the more relevant unit tests first.

The idea behind is that no one will use Pyglet to make video players
sensu stricto, so for specific applications you generally make custom
videos, and then start wondering what bloody codec you'll use among the
hundred supported by ffmpeg. Then IMHO if your graphical library makes
the choice in your place, this is not too bad. Not to manually disable
other formats, but at least say "Optimized for codec XXX", which would
be true because there will be unit tests for that :)

And I suggested WebM as it is most probably the codec I am going to use
for my application. Don't know for the others though.

[Nathan]

On Tue, Nov 15, 2011 at 6:04 AM, Rémy Sanchez

<remy.s...@hyperthese.net> wrote:
> On Mon, 14 Nov 2011 22:33:02 -0700, Nathan <nathan...@gmail.com> wrote:
>>
>> On Sun, Nov 13, 2011 at 9:35 PM, Rémy Sanchez
>> <remy.s...@hyperthese.net> wrote:
>>>
>>> On Sunday 13 November 2011 07:05:21 Nathan wrote:
>>>>
>>>> FIRST, does anyone know how/where I could find out what video formats
>>>> AVbin is supposed to support?  I'll search around in the ffmpeg
>>>> source, but I'm not sure what I'm even looking for, or where I'd look
>>>> for it.
>>>
>>> "ffmpeg -formats" looks like a good place to start. The output will
>>> probably
>>> depend on the compilation options though.
>>
>> Interesting idea.  Unfortunately, it looks like the build process of
>> ffmpeg from within avbin is completely customized, and no ffmpeg
>> binary is produced.  Perhaps I will be able to find a way to get the
>> binary produced if I muck around in the build process for awhile.
>
> Well, you'll find how to list the formats in the function behind that
> (cmdutils.c, function "int opt_formats(const char *opt, const char *arg)").
> The codecs registration looks pretty static though :
> http://cekirdek.pardus.org.tr/~ismail/ffmpeg-docs/allformats_8c-source.html

That's a lot of formats!

>>>> SECOND, what video formats are people actually using pyglet to decode?
>>>
>>> If I can make a suggestion, don't support too much formats and
>>> concentrate on
>>> free stuffs with few IP problems, let's say WebM for instance. Just my
>>> opinion...
>>
>> Well, I agree with the principle, but *I'm* not really supporting
>> formats or not.  FFmpeg does...or not.  AVbin is just a light wrapper
>> for some FFmpeg functions.  What I'm really interested in here is what
>> video formats people are actually using, so that I can prioritize
>> efforts to make the more relevant unit tests first.
>
> The idea behind is that no one will use Pyglet to make video players sensu
> stricto, so for specific applications you generally make custom videos, and
> then start wondering what bloody codec you'll use among the hundred
> supported by ffmpeg. Then IMHO if your graphical library makes the choice in
> your place, this is not too bad. Not to manually disable other formats, but
> at least say "Optimized for codec XXX", which would be true because there
> will be unit tests for that :)
>
> And I suggested WebM as it is most probably the codec I am going to use for
> my application. Don't know for the others though.

Ah, I see. Great suggestion! A list of recommended formats would
definitely help steer pyglet users in the right direction. I'll put
WebM up at the top of the list, seeing as how no one else who reads
the mailing list apparently uses video in pyglet (that is willing to
talk about it).

~ Nathan

[Tristam MacDonald]

On Tue, Nov 15, 2011 at 11:26 AM, Nathan <nathan...@gmail.com> wrote:

>>>> SECOND, what video formats are people actually using pyglet to decode?

Ah, I see.  Great suggestion!  A list of recommended formats would
definitely help steer pyglet users in the right direction.  I'll put
WebM up at the top of the list, seeing as how no one else who reads
the mailing list apparently uses video in pyglet (that is willing to
talk about it).

~ Nathan

I have loaded video into pyglet before, mostly as a test, and at the time I always used H.264, because I am on a Mac, and that is the most widely used format there.

In general though, I think most people would be happy with the selection of supported video formats listed in the pyglet manual (plus WebM, I imagine):

http://pyglet.org/doc/programming_guide/supported_media_types.html

--

Tristam MacDonald

System Administrator, Suffolk University Math & CS Department

http://swiftcoder.wordpress.com/

[Nathan]

Sweet! The list has already been created! I obviously need to give
the Pyglet documentation a thorough read, again. I'll focus my
efforts on that list.

~ Nathan

[anatoly techtonik]

On Monday, November 14, 2011 7:35:12 AM UTC+3, Rémy wrote:

> SECOND, what video formats are people actually using pyglet to decode?

If I can make a suggestion, don't support too much formats and concentrate on
free stuffs with few IP problems, let's say WebM for instance. Just my
opinion...

WebM for sure, and preferably without FFmpeg at all.

[anatoly techtonik]

Forgot to add +1 to "don't support too much formats". The best approach is to create a list that starts with one supported - and preferably only one =) - supported format, and add others with explanation why they are added, i.e. what shortcomings of original format these additional formats are supposed to close.

This way people will also get a guidance for choosing the most appropriate format for their goal (or just get some reasons to choose the preferred one).

[Tristam MacDonald]

On Thu, Nov 17, 2011 at 4:16 AM, anatoly techtonik <tech...@gmail.com> wrote:

Forgot to add +1 to "don't support too much formats". The best approach is to create a list that starts with one supported - and preferably only one =) - supported format, and add others with explanation why they are added, i.e. what shortcomings of original format these additional formats are supposed to close.

This way people will also get a guidance for choosing the most appropriate format for their goal (or just get some reasons to choose the preferred one).

Unfortunately, in the world of video there is no one-format-to-rule-them-all. The three possible contenders for that title are probably H.264, Theora and WebM, but each comes with significant drawbacks (licensing on non-Apple platforms, poor quality, and poor encoding speed, respectively).

If cutting down the number of formats is a popular idea, then I would suggest making those 3 the base set of formats, and get someone with extensive video experience to benchmark quality/size/speed of all 3 and provide a discussion of their relative merits.

[Richard Jones]

I don't believe the discussion is about removing support for codecs
(they're all built into ffmpeg and avbin is just a thin, sane ABI
around ffmpeg). The discussion is about which codecs should be present
in the avbin test suite.

An avbin that restricts itself to supporting WebM (or even one that
supports WebM and h.264 and Theora) is useless.

Richard

[Tristam MacDonald]

On Thu, Nov 17, 2011 at 5:26 PM, Richard Jones <r1char...@gmail.com> wrote:

> Unfortunately, in the world of video there is no
> one-format-to-rule-them-all. The three possible contenders for that title
> are probably H.264, Theora and WebM, but each comes with significant
> drawbacks (licensing on non-Apple platforms, poor quality, and poor encoding
> speed, respectively).
> If cutting down the number of formats is a popular idea, then I would
> suggest making those 3 the base set of formats, and get someone with
> extensive video experience to benchmark quality/size/speed of all 3 and
> provide a discussion of their relative merits.

I don't believe the discussion is about removing support for codecs
(they're all built into ffmpeg and avbin is just a thin, sane ABI
around ffmpeg). The discussion is about which codecs should be present
in the avbin test suite.

An avbin that restricts itself to supporting WebM (or even one that
supports WebM and h.264 and Theora) is useless.

    Richard

Yes, I assumed we were talking about 'supported' as in 'tested'. I am arguing against the idea that we should recommend one format over another in the test suite and/or documentation.

It might be that we can define a small cluster of formats (for which my list of [H.264, Theora, WebM] is just a suggestion), which covers 90% of use cases, but there are always going to be cases where a different format provides a better fit.