Webm file generated using ffmpeg contains errors and doesn't play properly.

[Ravi Kiran]

Hi,

I am trying to mux rtp audio(opus) and video(vp8) streams into a webm file using ffmpeg's libavformat. Though I am able to get an output file, but it plays properly only for first few seconds. After that it becomes junk. No image can be seen. Why is this happening ?  And, there seems to be lot of noise in audio if no headphones are used on the browser client. 

2. I am receiving audio and video streams from firefox/chrome browser client. What is the default avg bitrate of vp8 and opus encoded by firefox/chrome ? Right now I am setting video bitrate at 1000Kbps for vp8 and audio bitrate at 40Kbps for opus mono channel. Is this okay ? 

3. In the vp8 video stream I am receiving from browser, the key frame spacing seems to be very high. For a video of around 13-14 secs I am getting only 3 key frames. At 30 fps, its around 13*30 =    390 frames. So, 3 keyframes for around 390 frames. Is this normal or should I be getting more frequent key frames from the browser ?

4. I tried to find errors on the webm file by running the following ffmpeg command : 

      ffmpeg -v error -i test.webm -f null - 2>error.log

This command tries to recode the input webm file to nothing. In the process it reads the input file and reports any errors if present in the file. The following is the error log generated : 

[vp8 @ 0x7fe976003600] Header size larger than data provided

[opus @ 0x7fe975802000] Error decoding a SILK frame.

[opus @ 0x7fe975802000] Error decoding an Opus frame.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

[NULL @ 0x7fe973821600] Invalid sync code 7fd624.

[NULL @ 0x7fe973821600] Invalid sync code 000200.

[opus @ 0x7fe975802000] Error decoding a SILK frame.

[opus @ 0x7fe975802000] Error decoding an Opus frame.

[NULL @ 0x7fe973822200] Error parsing Opus packet header.

Error while decoding stream #0:0: Invalid data found when processing input

    Last message repeated 1 times

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid profile 4.

[NULL @ 0x7fe973821600] Invalid profile 6.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

[NULL @ 0x7fe973821600] Invalid profile 6.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973822200] Error parsing Opus packet header.

Error while decoding stream #0:0: Invalid data found when processing input

    Last message repeated 1 times

[NULL @ 0x7fe973821600] Invalid sync code 000000.

    Last message repeated 1 times

[NULL @ 0x7fe973821600] Invalid sync code 00012a.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

    Last message repeated 4 times

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

[NULL @ 0x7fe973821600] Invalid profile 4.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

[NULL @ 0x7fe973821600] Invalid profile 4.

[NULL @ 0x7fe973821600] Invalid sync code 7fd624.

[NULL @ 0x7fe973821600] Invalid profile 6.

    Last message repeated 1 times

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

[NULL @ 0x7fe973821600] Invalid profile 4.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid profile 6.

[NULL @ 0x7fe973821600] Invalid profile 7.

Error while decoding stream #0:0: Invalid data found when processing input

    Last message repeated 1 times

[NULL @ 0x7fe973821600] Invalid sync code 000000.

    Last message repeated 2 times

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

    Last message repeated 1 times

[NULL @ 0x7fe973822200] Error parsing Opus packet header.

[NULL @ 0x7fe973821600] Invalid sync code afe0e4.

Error while decoding stream #0:0: Invalid data found when processing input

   Last message repeated 1 times

[NULL @ 0x7fe973821600] Invalid profile 4.

[NULL @ 0x7fe973822200] Error parsing Opus packet header.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

   Last message repeated 3 times

[NULL @ 0x7fe973821600] Invalid sync code 0fb1e4.

[NULL @ 0x7fe973821600] Invalid sync code b01d8e.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973822200] Error parsing Opus packet header.

[NULL @ 0x7fe973821600] Invalid sync code 000000.

   Last message repeated 1 times

Error while decoding stream #0:0: Invalid data found when processing input

[NULL @ 0x7fe973821600] Invalid sync code 000000.

Error while decoding stream #0:0: Invalid data found when processing input

Too many packets buffered for output stream 0:1.

I am converting RTP packets to vp8 frames correctly as far as I have checked, with each vp8 frame consisting of "Uncompressed data chunk part" + "Compressed payload". And, timestamp values are also being set correctly by calculating timestamp deltas with firstframe rtp timestamp. Any help regarding this is really helpful. 

Thanks

-ravi kiran

[James Zern]

On Fri, Mar 2, 2018 at 5:56 AM, Ravi Kiran <kiran...@gmail.com> wrote:
> Hi,
>
> I am trying to mux rtp audio(opus) and video(vp8) streams into a webm file
> using ffmpeg's libavformat. Though I am able to get an output file, but it
> plays properly only for first few seconds. After that it becomes junk. No
> image can be seen. Why is this happening ? And, there seems to be lot of
> noise in audio if no headphones are used on the browser client.
>
> 2. I am receiving audio and video streams from firefox/chrome browser
> client. What is the default avg bitrate of vp8 and opus encoded by
> firefox/chrome ? Right now I am setting video bitrate at 1000Kbps for vp8
> and audio bitrate at 40Kbps for opus mono channel. Is this okay ?
>

Those are common bitrates, though ti does depend on the resolution of the video.

> 3. In the vp8 video stream I am receiving from browser, the key frame
> spacing seems to be very high. For a video of around 13-14 secs I am getting
> only 3 key frames. At 30 fps, its around 13*30 = 390 frames. So, 3
> keyframes for around 390 frames. Is this normal or should I be getting more
> frequent key frames from the browser ?
>

That frequency is reasonable to make a seekable stream.

> 4. I tried to find errors on the webm file by running the following ffmpeg
> command :
>
> ffmpeg -v error -i test.webm -f null - 2>error.log
>
> This command tries to recode the input webm file to nothing. In the process
> it reads the input file and reports any errors if present in the file. The
> following is the error log generated :
>
>
> [vp8 @ 0x7fe976003600] Header size larger than data provided
>
>

> [...]

>
>
>
> I am converting RTP packets to vp8 frames correctly as far as I have
> checked, with each vp8 frame consisting of "Uncompressed data chunk part" +
> "Compressed payload". And, timestamp values are also being set correctly by
> calculating timestamp deltas with firstframe rtp timestamp. Any help
> regarding this is really helpful.
>

Given the errors it seems there is an issue with perhaps your
concatenation or the size stored in the container assuming you're
capturing on a reliable link and there isn't some intermittent packet
loss.
You may want to try to get one stream working on its own rather than
debugging both at the same time.

[Ravi Kiran]

>>Those are common bitrates, though ti does depend on the resolution of the video. 

My video resolution is 640 X 480.

>>Given the errors it seems there is an issue with perhaps your 
>>concatenation or the size stored in the container assuming you're 
>>capturing on a reliable link and there isn't some intermittent packet 
>>loss. 

You mean concatenation of vp8 into individual frames ? And, I am not storing any size on the container. Maybe ffmpeg is doing that. Have to check that. Actually

>>You may want to try to get one stream working on its own rather than 
>>debugging both at the same time. 

Actually I did the same. I tried muxing video only and audio files. And, the quality is better when I do that compared to when I mux both audio and video. Means it doesn't go junk so early in the video. But, it does go junk after some period of time in that too. 

[James Zern]

On Fri, Mar 2, 2018, 22:10 Ravi Kiran <kiran...@gmail.com> wrote:

>>Those are common bitrates, though ti does depend on the resolution of the video. 

My video resolution is 640 X 480.

>>Given the errors it seems there is an issue with perhaps your 
>>concatenation or the size stored in the container assuming you're 
>>capturing on a reliable link and there isn't some intermittent packet 
>>loss. 

You mean concatenation of vp8 into individual frames ? And, I am not storing any size on the container. Maybe ffmpeg is doing that. Have to check that. Actually

Yes that's what I meant. You can use ffprobe, mkvinfo or webm_info to inspect the final output.

--
You received this message because you are subscribed to the Google Groups "WebM Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webm-discuss...@webmproject.org.
To post to this group, send email to webm-d...@webmproject.org.
Visit this group at https://groups.google.com/a/webmproject.org/group/webm-discuss/.
For more options, visit https://groups.google.com/a/webmproject.org/d/optout.

[Ravi Kiran]

Hi,

Sorry for the delayed response. I tried all the options we discussed above. That is, I tried muxing video only file (vp8 video), audio only file (opus audio) and video-audio file using FFmpeg's libavformat library. Audio only file seems to be working okay. In video only file, the same issue persists, i.e for the initial few seconds the video is fine. But, after that the video goes blur. 

I validated the output file with ffprobe. All the meta info seems to be fine. But, when I checked with mkvalidator tool, the following error messages were shown : 

ERR201: Invalid 'CueRelativePosition' for profile 'webm' in CueTrackPositions at 588307

ERR0B1: SimpleBlock at 764 track #2 is not a keyframe

I am attaching the output test webm file here. I am unable to proceed any further in this. My vp8 frames seem to be correct and all other data also seem to be correct. As I mentioned earlier I suspect very few keyframes received and decoder not able to decode correctly with available keyframes as a issue. But, what to do about that I have no clue. Please help.

-ravi kiran

On Friday, 2 March 2018 19:26:00 UTC+5:30, Ravi Kiran wrote:

[James Zern]

Hi,

On Fri, Mar 23, 2018 at 12:58 AM Ravi Kiran <kiran...@gmail.com> wrote:

Hi,

Sorry for the delayed response. I tried all the options we discussed above. That is, I tried muxing video only file (vp8 video), audio only file (opus audio) and video-audio file using FFmpeg's libavformat library. Audio only file seems to be working okay. In video only file, the same issue persists, i.e for the initial few seconds the video is fine. But, after that the video goes blur. 

I validated the output file with ffprobe. All the meta info seems to be fine. But, when I checked with mkvalidator tool, the following error messages were shown : 

ERR201: Invalid 'CueRelativePosition' for profile 'webm' in CueTrackPositions at 588307

ERR0B1: SimpleBlock at 764 track #2 is not a keyframe

This is because the audio stream is not marked as a keyframe -- all audio blocks can be marked this way.

 

I am attaching the output test webm file here. I am unable to proceed any further in this. My vp8 frames seem to be correct and all other data also seem to be correct. As I mentioned earlier I suspect very few keyframes received and decoder not able to decode correctly with available keyframes as a issue. But, what to do about that I have no clue. Please help.

How about one without the audio so it's a bit easier to analyze?

 

--

[Ravi Kiran]

>How about one without the audio so it's a bit easier to analyze?

I have tried without audio also. Same issue there too - video pixelates and blurs out after few seconds. I am attaching a video only output file here. Following error shown when this file is validated with mkvalidator tool :

ERR201: Invalid 'CueRelativePosition' for profile 'webm' in CueTrackPositions at 355368

Thank you.

-ravi kiran.

On Friday, 2 March 2018 19:26:00 UTC+5:30, Ravi Kiran wrote:

[James Zern]

Hi,

On Fri, Mar 23, 2018 at 2:16 AM Ravi Kiran <kiran...@gmail.com> wrote:

> >How about one without the audio so it's a bit easier to analyze?

> I have tried without audio also. Same issue there too - video pixelates
and blurs out after few seconds. I am attaching a video only output file
here. Following error shown when this file is validated with mkvalidator
tool :

> ERR201: Invalid 'CueRelativePosition' for profile 'webm' in
CueTrackPositions at 355368

I don't think this is the cause of the corruption, cues are only used when
seeking in the stream. If you start playback from the beginning with ffplay
you'll see the following as it progresses as you did with ffmpeg:
...
[NULL @ 0x7f0efc003240] Invalid sync code 23b9f8. sq= 0B f=0/0
[NULL @ 0x7f0efc003240] Invalid sync code 8d8ddc. sq= 0B f=0/0
[vp8 @ 0x7f0efc012e80] Header size larger than data providedf=0/0
[vp8 @ 0x7f0efc0059e0] Header size larger than data providedf=0/0

You're probably losing some data along the way. Is the rtp data obtained
over a lossy link like udp? Are you able to access the data at the point of
capture? You could then write that out to compare what you're seeing
downstream. If you have issues at the point of capture then there's likely
a logic error in the code doing the muxing.

[Ravi Kiran]

Hi James, 

Sorry for the delayed response. You were right. There was a bug in the way i was converting incoming RTP packets into individual vp8 frames. And there was no data loss, even if I am receiving RTP on UDP port. I am now successfully able to generate a complete webm file, which plays properly. But I need a little light on few things.

As you have seen, mine is a live video case, and I am recording a live video stream into webm file in realtime. Now, if I want to stream the webm file over http in realtime itself without much latency, what would be the best approach to go ? I got few options here :

1. Generate multiple small webm files of over 10-30 secs length, in realtime and fetch them over http for streaming. But it will have few challenges :

           

           a) Later if I want a recorded whole file, I will have to stitch all these files together into a whole file. 

           

           b) In order to generate multiple small files, I should be receiving frequent vp8 keyframes at correct intervals, so that I can begin each webm file at the keyframe. But I have noticed, the                   keyframe frequency is not dependable. Sometimes there is only 1 keyframe for a video of about 1-2 mins. This wouldn't help if I want to mux multiple small webms which can be      

               streamed as http payloads.

2. Generate a single whole webm file, and fetch specified segment/section of video using range fetching from the file over http. It will have the overhead of understanding and traversing the EBML and matroska container format by myself.

3. Use a tool like ffmpeg to generate streamable chunks of video separately and stream them over http using MPEG-DASH or HLS. Not sure if it can be done in realtime.

So, my requirement here is to stream the file over http in 'realtime', instead of sending the vp8 packetized in RTP, while having recorded webm intact. Which would be the best way to go forward ? Also, is webm suitable for http streaming of video or is it better to switch to someother format ? (This i am just asking for confirmation, I know youtube already streams webm over http using MPEG DASH ).

Any suggestion will be much helpful. Thanks.

-ravi kiran.

On Friday, 2 March 2018 19:26:00 UTC+5:30, Ravi Kiran wrote:

[James Zern]

On Sat, Mar 31, 2018 at 3:17 AM Ravi Kiran <kiran...@gmail.com> wrote:

Hi James, 

Sorry for the delayed response. You were right. There was a bug in the way i was converting incoming RTP packets into individual vp8 frames. And there was no data loss, even if I am receiving RTP on UDP port. I am now successfully able to generate a complete webm file, which plays properly. But I need a little light on few things.

As you have seen, mine is a live video case, and I am recording a live video stream into webm file in realtime. Now, if I want to stream the webm file over http in realtime itself without much latency, what would be the best approach to go ? I got few options here :

1. Generate multiple small webm files of over 10-30 secs length, in realtime and fetch them over http for streaming. But it will have few challenges :

           

           a) Later if I want a recorded whole file, I will have to stitch all these files together into a whole file. 

           

           b) In order to generate multiple small files, I should be receiving frequent vp8 keyframes at correct intervals, so that I can begin each webm file at the keyframe. But I have noticed, the                   keyframe frequency is not dependable. Sometimes there is only 1 keyframe for a video of about 1-2 mins. This wouldn't help if I want to mux multiple small webms which can be      

               streamed as http payloads.

You can force a more regular frequency of keyframes to produce a more seekable stream (-g with ffmpeg).

 

2. Generate a single whole webm file, and fetch specified segment/section of video using range fetching from the file over http. It will have the overhead of understanding and traversing the EBML and matroska container format by myself.

3. Use a tool like ffmpeg to generate streamable chunks of video separately and stream them over http using MPEG-DASH or HLS. Not sure if it can be done in realtime.

So, my requirement here is to stream the file over http in 'realtime', instead of sending the vp8 packetized in RTP, while having recorded webm intact. Which would be the best way to go forward ? Also, is webm suitable for http streaming of video or is it better to switch to someother format ? (This i am just asking for confirmation, I know youtube already streams webm over http using MPEG DASH ).

Fragmented mp4 or mkv/webm will work for this. For matroska elements (segments, clusters) can be given a size of 'unknown' to allow them to be streamed in realtime.

 

--

[Huong Nguyen Xuan]

Hi, Ravi,
I am getting same issue, what did you fix with rtp packet handling?
Thank you.