Framerate drops down using iOS hardware H.264

[Ray Wang]

Hi:

I am developing iOS apps based on webrtc, i set the framerate of videotoolbox encoder explicitly to 15.0, however, at the very beginning sendframerate came out to be 30.0.

and after a while, sendframerate drops down dramatically to about 1.0 ~ 5.0, and there is no packet loss or cpu overuse, the rtt is also very low.

I guess  most of the captured frames was dropped in the frame dropper module.

could anyone help to elaborate how frame dropper works and  a brief description of the frame rate control algorithm of Webrtc?

[Ray Wang]

Hello, anybody have an idea?

According to logs, frame dropper drops most I420 frames when explicitly using iOS hardware encoder.

在 2016年3月2日星期三 UTC+8上午11:50:26，Ray Wang写道：

[Zeke Chin]

See:

https://code.google.com/p/chromium/codesearch#chromium/src/third_party/webrtc/modules/video_coding/video_sender.cc&l=285

Peter knows more about that than I do.

What's likely happening is that the encoder is outputting a much higher bitrate than what we expected from it. This causes us to drop frames because we think we've saturated the bandwidth allocation. I made a change recently that adapts the bitrate set on the encoder - is this not working for you?

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/44a023fc-7e40-4706-96c5-c4b0abceecfb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Ray Wang]

No, i merged your latest commit, the problem still exists.

But, framerate behaves different on different devices.

I tested between two device, one is iPhone 5C, iOS 9.2; another is iPhone 6, iOS 8.4;

the resolution is 360*640, and target bitrate is 512Kbps.

the framerate running on iPhone 5C will drop down from 30 fps to about 2~5 fps after a moment. and the output bitrate is actually above 512Kbps mostly.

however iPhone 6 was running at 30fps consistently, and output bitrate is below 512Kbps mostly. 

both of two device use Videotoolbox.

and besides, although iPhone 6 can run at 30 fps consistently, the rendering image quality is not so good, i think it's because framerate at 30 fps is too high,

but changing framerate to 15 fps does not take any effect on Videotoolbox(even explicitly set videotoolbox property).

在 2016年3月10日星期四 UTC+8上午6:59:44，Zeke Chin写道：

[Zeke Chin]

I opened crbug/webrtc/5650. I'll look into it once I get a 5C device.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/f7d143c5-fd3c-471f-86df-7758415ce3d9%40googlegroups.com.

[Ray Wang]

Thanks, Zeke

在 2016年3月11日星期五 UTC+8上午4:30:16，Zeke Chin写道：