Webm audio/video sync

[bra...@collabramusic.com]

I have a webm that when played back in VLC and Firefox has the audio and video in sync. However, if I view the video in Chrome the audio and video fall out of sync. Does anyone have any advice on what might be causing this to occur?

Below is the video that when downloaded and opened in Chrome has the issue occurring: https://drive.google.com/file/d/0B8sIqIlMgB94M2h5NzlTNzR0QWM/view?usp=sharing

[Brendan Bolles]

On Monday, February 8, 2016 at 10:48:01 AM UTC-8, bra...@collabramusic.com wrote:

I have a webm that when played back in VLC and Firefox has the audio and video in sync. However, if I view the video in Chrome the audio and video fall out of sync. Does anyone have any advice on what might be causing this to occur
?

I took a hard look at your file. It's a little disconcerting, given that it I think it comes from FFmpeg?

My hunch is that the reason Firefox plays it back in sync is that it ignores timestamps while maybe Chrome doesn't.

I wrote a little tool to examine the file, checking which video frames and audio samples are in the file and comparing them to the frame/sample you might expect at that timestamp. The program and the output are attached.

I think the main problem is that the number of audio samples at any point in time are fewer than what I'd expect given the timestamp and the sample rate. The gap increases over time, so that by the end of the video there are about 43000 fewer samples, nearly one second off given the 48 kHz sample rate.

Other issues I see:

1. There is no frame rate specified and the frames have uneven timecode separations, confusing a program that tries to guess the framerate. My program came up with 13.672 frames per second, although the overall rate is more like 27.5 fps.

2. The first video frame has timecode 30000000, not 0.

[Brandon Kobel]

Brendan,

I can't thank you enough for spending so much effort on my problem.

The file was created using kurento-media-server which is a gstreamer based application that records audio/video from our user's browsers using WebRtc, hence the sporatic framerates and audio samples. I then ran the file through mkvmerge and mkclean to make sure it was completely up to 'spec' with ordering of elements, etc.

We've been having trouble internally recreating the circumstances that lead to these videos being recreated; and more so we've been having trouble determining whether kurento-media-server is doing something wrong when creating the files, or whether Google Chrome isn't playing back the files properly, or whether there's some sort of post-processing that we can do to bridge the gap.

Would you happen to know whether the video that is being created abides by all the webm specifications, with the framerate not being specified and the sporatic timecode seperations? Or is this something that's definitely not appropriate and needs to be fixed when creating the file, or in some sort of post-processing step?

Thanks again,

Brandon

[Brendan Bolles]

On Tuesday, February 9, 2016 at 2:46:28 AM UTC-8, Brandon Kobel wrote:

We've been having trouble internally recreating the circumstances that lead to these videos being recreated; and more so we've been having trouble determining whether kurento-media-server is doing something wrong when creating the files, or whether Google Chrome isn't playing back the files properly, or whether there's some sort of post-processing that we can do to bridge the gap.

Would you happen to know whether the video that is being created abides by all the webm specifications, with the framerate not being specified and the sporatic timecode seperations? Or is this something that's definitely not appropriate and needs to be fixed when creating the file, or in some sort of post-processing step?

I'd guess if you had the computer try to encode the movie while some other process was using up CPU and disk resources, that might help replicate it.

Hopefully a real expert will chime in, but WebM is simple format. It doesn't actually require that your video frames fall at even time intervals, although some software may be expecting that.

That audio samples issue may be a different beast though. What is the right thing for the player to do if the samples don't match the timing? Seems like the best choice would be to play the audio straight through and adjust the timings to match. This would adjust the frame timings, and then who knows if they would still sync?