VP8 drops frames with p frames

[Kevin Wang]

I'm having some trouble with a ffmpeg-produced VP8 RTP stream playing in Chrome WebRTC via pion. If I set the GOP size to a small number (<4) it works, however if I set it to anything larger, chrome will eventually start dropping frames when playing one of the p frames. This doesn't happen when I use gstreamer to generate the frames, however.

I don't see any errors in the chrome_debug.log with webrtc's verbose logging enabled, nevertheless framesDropped steadily increases:

What's the best way to debug this further? I've tried ffplay'ing the exact RTP stream that's sent to WebRTC and it plays without issue.

[Kevin Wang]

Although actually I found this in the log, I do seem to get this message at the same frequency as a dropped frame:

[135772:77324:0322/113535.434:WARNING:frame_buffer2.cc(416)] A jump in frame id was detected, clearing buffer.

How is the frame id set? It looks like it's in the encoded bitstream, is it this field (which is currently -1) that I need to manually set? https://www.ffmpeg.org/doxygen/trunk/structAVPacket.html#ab5793d8195cf4789dfb3913b7a693903

[Kevin Wang]

Ok I did a bit more debugging, turns out there's another warning with the right periodicity:

[134300:137112:0322/122902.927:WARNING:frame_buffer2.cc(421)] Frame 6 inserted after frame 127 was handed off for decoding, dropping frame.
[134300:137112:0322/122902.961:WARNING:frame_buffer2.cc(421)] Frame 7 inserted after frame 127 was handed off for decoding, dropping frame.
[134300:137112:0322/122902.995:WARNING:frame_buffer2.cc(416)] A jump in frame id was detected, clearing buffer.

I then looked at ffmpeg vs gstreamer's rtp payloader and it seems that gstreamer defaults to not setting the picture id: https://github.com/GStreamer/gst-plugins-good/blob/master/gst/rtp/gstrtpvp8pay.c#L42

Whereas ffmpeg always uses a 7-bit picture id: http://ffmpeg.org/doxygen/2.1/rtpenc__vp8_8c_source.html

If I set picture-id-mode=7-bit on gstreamer, I can reproduce the issue. Is webrtc not handling picture id's correctly? In particular, it seems that:

https://chromium.googlesource.com/external/webrtc/+/HEAD/modules/video_coding/frame_buffer2.cc#424

accommodates a full 64-bit picture id: https://chromium.googlesource.com/external/webrtc/+/HEAD/api/video/encoded_frame.h#46

and when a smaller picture id is placed in that field, it will drop the buffer size of frames?

Although this is my first exposure to the webrtc codebase so apologies if it is not that simple :)

[Philipp Hancke]

seven bit picture ids are broken since 2017:

  https://bugs.chromium.org/p/webrtc/issues/detail?id=7675

IMO not supporting seven bit picture ids is violating a MUST-level requirement from RFC 7741 but as of the fix for

  https://bugs.chromium.org/p/webrtc/issues/detail?id=7877

this is hardly relevant in practice, just use two bytes.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAPeaFSTTN%2BF4qy-QUDxXC7dp-50ftwmxPPFnZTqssFVdruhouw%40mail.gmail.com.

[Kevin Wang]

I see, ok that's good to know. I'll update ffmpeg to produce 15-bit picture id's.

It's a shame that the RFC is violated here, I spent a day trying to figure out what was going on since there's no warning or anything similar. :(

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKiJMgsAOhOL6wrtxFBSG-cV-bqcpNO_jbh6r1tT2hgNvVg%40mail.gmail.com.