variable frame rate on webm

[fishor]

Hallo all,
i hacking currently on Cheese (linux webcam recorder), and i have one
issue. Most webcams do variable frame rate depending on light. For
example my cam report say it send 30fps but vp8 will get only 4fps.
To capture stream i use this pipeline:

gst-launch-0.10 -e pulsesrc ! queue ! audioconvert ! vorbisenc !
queue ! webmmux name="muxer" v4l2src ! video/x-raw-
yuv,width=320,height=240 ! ffmpegcolorspace ! vp8enc ! muxer. muxer. !
filesink location=foo.webm

At the end i get only static 30fps webm file, so if you play it will
looks like "need for speed". There is no way to use "videorate", it
will consum useless cpu power.

My question is. Is it possible to produce webm with variable frame
rate?

Regards,
Alexey

[fishor]

I found theora can use some place holders for duplicate frames in ogg.
It reduce CPU usage and size. Are there any thing like this in webm?

Take a look here:
https://bugzilla.gnome.org/show_bug.cgi?id=627459

[Steve Lhomme]

Matroska can handle VFR just fine. The codec on the other hand may not be too happy with that because of motion handling.

My 2 cents

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To post to this group, send email to webm-d...@webmproject.org.
To unsubscribe from this group, send email to webm-discuss...@webmproject.org.
For more options, visit this group at http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.

[Alex Fisher]

My test show, vp8 muxed even with matroska can't do it. I had good
expirience with VFR mjpeg or theora muxed in matroska. This was the
reason why i ask here, may be i just do not seen some switch for VFR
by vp8.

[Alex Fisher]

I see there is VPX_FRAME_IS_DROPPABLE, any chance to write a dummy 0
byte frame and mark it as DROPPABLE?

[Frank Galligan]

I would think this should work. What happens if you run a file through mkvalidator?

http://www.matroska.org/downloads/mkvalidator.html

--

[Alex Fisher]

2010/8/23 Frank Galligan <fgal...@google.com>:

> I would think this should work. What happens if you run a file through
> mkvalidator?
> http://www.matroska.org/downloads/mkvalidator.html

mkvalidator 0.2.2: the file appears to be valid
Track #1 V_VP8 95815 bits/s
Track #2 A_VORBIS 345 bits/s
file created with GStreamer plugin version 0.10.24.3 / GStreamer Matroska muxer

[Frank Galligan]

What player are you using? And what is the behavior you are seeing?

--

[fishor]

On Aug 24, 6:02 pm, Frank Galligan <fgalli...@google.com> wrote:
> What player are you using? And what is the behavior you are seeing?

I tested two different player, mplayer and totem. On both the video
speed is too high, i'd say x2-x3.

[Andy Shaules]

Ok, time to chime in.

The solution which may be possible for you based on the method of capturing.

1) Instead of waiting for the camera to update, which is based often on
amount of light these days, you would spin a thread to push samples, and
repeating the last available frame in cases of low light.

2) When a frame is expected and does not arrive, you can try to push an
empty video frame, as if the compressor itself returned a zero length frame.
The video renderer will keep the sync by assuming a dropped frame at that
location. Same as AVI.

3) Force the mux filter/process to not wait for matching video samples,
effectivly producing the zero length payloads if required.

--

[Frank Galligan]

The VP8 codec and the WebM container should not care that you are VFR. Are the timestamps of the video frames wrong in the captured file?

--

[Frank Galligan]

Before doing any of these I would try and figure out why the VFR files are not correct. The codec an container should be able to handle VFR.

If the capture solution cannot handle VFR then I would try 1. But I would not change the mux filter to only work with a rate and scale.

[Andy Shaules]

Mostly the trouble comes from the cameras media type. It lies. The mux filter obeys, and if the capture filter provided by the manufacturer does not provide IAMDroppedFrames interface, the resulting file will be mangled by the mux filter, whether it is webm or not. Depending on the muxer, if it has its own sense of time, which they usually dont, your results will vary. The trouble shows up in the playback graph.

 

What is the tool being used?  Sounds like a job for a trans filter combined with solution number one.

[Frank Galligan]

I haven't seen where the video timestamps from the capture have been flat out wrong before (well at least to the point of easily noticable). If that is the case then I would definitely create another filter that sits between the capture and the encoder to re-timestamp the video. But if you use wallclock time your av will most likely drift overtime unless you have access to the audio.

If you do have to re-timestamp video another big problem I have seen in the past is variable sample rate coming form the audio capture. Depending on your needs you might have to address this too.

But before you do any of this development you should be sure the issue is the video capture on the camera and not an easily fixed bug somewhere else in the solution.

[Andy Shaules]

The timestamps are written by the allocator, and the issue occures when the camera fillBuffer method takes longer than it agreed upon. If the camera fails to update the IAmDropped Frames interface with accurate count, the rest of the graph assumes correct frame count and delivery rate, and thus, causes the fast-forward playback of the video. Likewise, in the case of software drivers based on Vivek's infamous VCam, that deliver samples too quick, the resulting mux plays in slow motion.

 

So the short of the long is that you wont see incorrect timestamps. You just wont get the count and frequency advertised at the IAMStreamConfig interface. Playback will reveal the misaligned zipper/ mux.

 

VFR cams and this issue affecting time is a common issue over at the directshow programmers community forum.

 

 Andy

[Andy Shaules]

It could be that the mux filter is missing the flag,

 

pMediaSample->SetDiscontinuity(true);

...Which is the expected flag in such a case. 

[Alex Fisher]

I use gstreamer to create this files. It has normally no problems with
time stamps us i can see on theore+matroska or mjpeg+matrsoka samples.

This is pipe i use:

gst-launch-0.10 -e pulsesrc ! queue ! audioconvert ! vorbisenc ! queue
! webmmux name="muxer" v4l2src ! video/x-raw-yuv,width=320,height=240
! ffmpegcolorspace ! vp8enc ! muxer. muxer. ! filesink
location=foo.webm

Your solution to copies of frames will look like this:
... ! videorate ! vp8enc ! webmmux ! ...

but this is exactly what i do not need. If web cam provide 30fps but
do only 5fps this will make huge CPU load, this will make realtime
capture impossible.

[Alex Fisher]

In attachment is file sample

[Alex Fisher]

Please take a look to this bu report:
https://bugzilla.gnome.org/show_bug.cgi?id=627459

it handle same issue about theora+ogg container.

[Frank Galligan]

Nothing looks to bad in the file so most likely something is messing up the timestamps before it gets to the vp8 encoder.

On Wed, Aug 25, 2010 at 2:01 AM, Alex Fisher <lexa....@gmail.com> wrote:

In attachment is file sample

[Frank Galligan]

I looked and from the bug report theora in ogg is a fixed framerate format. Ogg and Webm are different so there might be 2 different bugs.

Did you file a specific bug about webm capture in gstreamer?

[Alex Fisher]

I did some extra debuging. Seem like vp8 encoder regenerate new
timestamps. The question is, in gst or libvpx part.

[Alex Fisher]

Here is new bug report. Will see what will come out
https://bugzilla.gnome.org/show_bug.cgi?id=627971

[Alex Fisher]

Looks like this is problem of libvpx.

[Frank Galligan]

Hi Alex,

It looked to me like Sebastian said it was a problem with video encoder base class of gstreamer. Did I misunderstand the comment?

Frank

On Thu, Aug 26, 2010 at 11:59 AM, Alex Fisher <lexa....@gmail.com> wrote:

Looks like this is problem of libvpx.

[Alex Fisher]

He said "It's a problem in the video encoder base class.". But it's
true and i interpreted this and probably wrong. I will ask it again.
Can you confirm that VFR work on your system?

[Frank Galligan]

Yes VFR should work for VP8 and WebM.

--