H264 Video Freeze on Chrome M128 when using OBS for streaming

[Jeremy Mao]

Repro steps:

1. Use OBS for streaming, and x264 as the encoder, may need a RTMP gateway to convert the RTMP protocol to RTP protocol.

2. Use WebRTC as the viewer.

3. Introduce a 5% downlink packet loss, and you will see the video is freezing on Chrome M128, but smooth on Chrome M127.

Has anyone encountered a similar issue?? I can see there are some H264-related modifications in https://webrtc.googlesource.com/src.git/+log/refs/branch-heads/6613, but don't why, could anyone provide any help on this, thanks! 

 

[Philipp Hancke]

running a bisect using https://www.chromium.org/developers/bisect-builds-py/ will most likely tell you what commit caused this.

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/b99008f2-9ec4-4e56-8e0d-8cae27c53fdfn%40googlegroups.com.

[Sean DuBois]

Do you still see this problem if you use WHIP (WebRTC) instead of RTMP out of OBS.

You can pass the NACKs back to OBS directly 

On Sep 13, 2024, at 09:52, 'Philipp Hancke' via discuss-webrtc <discuss...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKiLZ8nQ%3DDdgWxwpy9b0z7gTBWkFdCMBdHsrbgiCgG%2BxNvQ%40mail.gmail.com.

[shunbo li]

After conducting some tests, we found that when using Chrome 128 with hardware encoding enabled, if the H.264 profile-level-id is set to 64001f, subscribing to this stream on Chrome 128 (Windows & Mac platforms) and applying a 5% downstream weak network (using Win-clumsy/Win-NLC) can reproduce the H.264 decoding freeze（Only Keyframe decode success）, and there are a large number of log outputs like this in chrome_debug.log: 

"[4116:17920:0914/100518.272:WARNING:rtp_seq_num_only_ref_finder.cc(65)] Generic frame with packet range [17062, 17073] has no GoP, dropping frame."

However, when subscribing to this stream on Chrome 127 (Windows & Mac platforms), even applying a 20% downstream weak network remains very smooth (using Win-clumsy/Win-NLC).

Therefore, I believe this is not an issue with OBS streaming, but rather a fallback in WebRTC's compatibility with H.264.

[shunbo li]

The attached log file may help you analyze the issue.

[Sergio Garcia Murillo]

what are the timestamps for each packet?  and do you know what  packet loss triggered the issue? 

El sáb, 14 sept 2024, 11:23, 'shunbo li' via discuss-webrtc <discuss...@googlegroups.com> escribió:

After debugging the WebRTC source code, we found that the following commit changed the default behavior in the  ProcessStapAOrSingleNalu  function  .
( https://source.chromium.org/chromium/_/webrtc/src/+/3753c8190e3f0aca6758a5521e33f8b5d4f09ab4 )

This commit assumes that the RTP packet contains at least an SPS, PPS, I frame, and P frame. However, in reality, there can be cases where only an AUD frame is encapsulated using Strap-A.

For example, when Chrome 128 sends an H264 stream at 640x480, 15fps, and 500kbps, the output sequence from the encoder is as follows:

In this situation, the frame continuity check based on the RTP sequence number will fail, which can lead to video freeze due to the absence of decoded frames.(last_picture_id_with_padding_gop)  

在2024年9月13日星期五 UTC+8 23:52:31<Sean DuBois> 写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/b696a252-29be-49c2-9a19-c2633c0cd499n%40googlegroups.com.

[shunbo li]

Here are two sets of tests I conducted, each containing a chrome_debug.log, a browser-side packet capture, and a server-side record of the NALUs in each packet.

The only variable between the two sets of tests is the encoding method used by the same sending device for H.264: software encoding (file prefix “[sw-encode]”) and hardware encoding (file prefix “[hw-encode]”). 

When using the clumsy tool to introduce a 5% random packet drop, the test case using hardware encoding experienced significant stuttering on the receiving end, while the test case using software encoding was very smooth.

In the test case using hardware encoding, there are numerous entries in the chrome_debug.log stating "Generic frame with packet range [x, x] has no GoP, dropping frame." It appears that the frame continuity check mechanism before decoding is not functioning, or there might be an issue with the FindFrame method in h26x_packet_buffer?

I look forward to receiving your further analysis results. Thank you very much!

[Sergio Garcia Murillo]

h26x_packet_buffer? In was not aware it was enabled by default now. Are you compiling your own chromium for this tests? 

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/1f7ff1c1-5e41-4466-b1dd-5aa15c9e21ccn%40googlegroups.com.

[shunbo li]

The above tests were conducted using the automatically updated Chrome 128.

I am trying to download the source code for the Windows platform and compile the corresponding version of Chromium, but I am currently facing some issues. If I’m lucky and succeed, I will indeed try to debug it.

At the same time, if anyone can help look into this issue, it could be fixed more quickly.

[Philipp Hancke]

You may find the video_replay tool described at https://webrtchacks.com/video_replay/ to be more useful.

One of the comments at the bottom has good instructions for getting unencrypted RTP/RTCP out of Chrome

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/97cb658f-d889-4131-a63b-6032ba7110b9n%40googlegroups.com.

[Muhammad Usman Bashir]

For a temporary workaround, consider using software encoding instead of hardware encoding as it seems to perform better under packet loss conditions.

As per requirement: The FindFrame in h26x_packet_buffer method is crucial for locating specific frames within the packet buffer. Enhancing its efficiency and accuracy could lead to better frame retrieval and reduce the chances of frame mismatches or drops that might contribute to video freezes.

I have tested M127, and M128 branches with fresh webrtc compilations on windows 11, and didn't face any video freeze issue using OBS Studio.

[shunbo li]

It is indeed strange. 

The Chromium I compiled with M128, when compared to Chrome 128/Chrome 129 on the same machine decoding the same H.264 RTP stream under 5% weak network conditions, caused Chrome release 128 to freeze (it was upgraded to 129 after a restart today), while Chromium ran smoothly. 

However, during the testing process, Chromium did experience several crashes. 

The corresponding stack trace can be referenced in the image below.

I confirm that chromium running without use h26x_packet_buffer, but rather the older version of packet_buffer. This may be related to not finding “first_packet” after the frame is completed, and the commit related to the first_packet marker is this one:  https://source.chromium.org/chromium/_/webrtc/src/+/3753c8190e3f0aca6758a5521e33f8b5d4f09ab4. This commit changed the logic of the first_packet flag.

I am currently compiling a version with RTC_CHECK disabled to see if there will be any differences.

[shunbo li]

Attached is a set of comparison videos that evaluate the decoding of H.264 streamed by OBS under the same test conditions for Chrome 129, Edge 128, Firefox 130, and Chromium 128. Under 5% random packet loss, Chrome 129 and Edge 128 freeze, but recover when the weak network is removed. In contrast, Firefox 130 and Chromium 128 had no issues throughout.

The compilation options for Chromium 128 are:  is_debug = false is_component_build = true blink_symbol_level = 0 v8_symbol_level = 0 ffmpeg_branding = "Chrome" rtc_use_h264 = true proprietary_codecs = true rtc_disable_check_msg = true.
I am not sure which compilation options in these release versions caused the issue, but I can be certain that hardware-encoded H.264 is not the root cause of the video stuttering, as Chrome 127 and earlier versions, as well as Firefox, do not have this problem.

For various reasons, we cannot reject the streaming end from sending hardware-encoded H.264. 

From the browser's perspective, a good product should be able to handle such basic issues. 

Therefore, we need your help to see if there is a way to pinpoint the problem and resolve it in future versions. 

Thank you all.

[Sergio Garcia Murillo]

Just got back from a business trip, so won't be able to take a look at it this week, but just a couple of comments.

When reporting issues it is always good to open an issue on the libwebrtc issue tracker.

When you compile chromium it will use the new h26x_packet_bufffer by default, which is not used on the official chrome builds yet, so you will be testing something different. You can revert this behaviour as stated here:

https://issuetracker.google.com/issues/41480904#comment89

Best regards

Sergio

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/3fd13bc2-0302-4531-9b13-b881072d0128n%40googlegroups.com.

[shunbo li]

Thank you very much. Your reply and the related posts have been very helpful and have pointed the investigation in the right direction. 

It can be confirmed that the compatibility fallback issue with H.264 hardware-encoded streams is related to the latest changes in depacketizer_h264( https://source.chromium.org/chromium/_/webrtc/src/+/3753c8190e3f0aca6758a5521e33f8b5d4f09ab4 ).

The latest code causes a freeze when 5% packet loss is applied after disabling h26x_packet_buffer. Rolling back this change allows playback regardless of whether h26x_packet_buffer is enabled or not.

I will submit a new issue report or take other measures to push for the resolution of this H.264 compatibility problem. 

Thank you!