MediaRecorder VP9 Bitrate spikes

[Robert Jamieson]

Hi,

We are seeing ~1% of all our user's recordings, across tens of thousands of samples, having a bitrate significantly higher than the peak bitrate set when using the MediaRecorder to record VP9 from a 1080p canvas. Debugging on Windows, I see the average (2.5Mb/s) and peak (5.0Mb/s) bitrates being set correctly in the encoder. In the affected recordings we see bitrates as high as 40Mb/s. I analyzed one such recording and saw that for parts of the recording the bitrate was jumping by 16x and the frame rate was simultaneously dropping (30 down to 20fps). The frame sizes were ~1.1MB per frame for p-frames. The frames during this period of high bitrate had very little motion and shouldn't have stressed the encoder.

So far I been unable to re-create this issue locally. We have seen this issue across both Mac and Windows. If I disable hardware encoding on Windows I see the bitrate is 3x higher for the same static image being rendered at 30fps, 30Mb for 30secs vs 10Mb for 30secs with the HW encoder, but I don't see the huge spikes we see in our customer's recordings.

My current theory is overall system pressure is affecting the encoder causing the compression to drop. Because we are seeing it on both Mac and Windows, I don't think it is an issue with the hardware encoder directly.

Is anyone aware of conditions that would cause the bitrate to spike so far beyond what is set as the peak bitrate? Or should this be considered a bug?

Thanks,
Robert

[Dale Curtis]

+Markus Handell +Eugene Zemtsov +

My best guess would be some issue with hardware encoders. I'd try to see if you can correlate with the unmasked GPU information.

We're in the middle of transitioning the encoder backends for MediaRecorder. So you might see if this reproduces/does not reproduce for a customer using --enable-blink-feauters=MediaRecorderUseMediaVideoEncoder or --disable-blink-features=MediaRecorderUseMediaVideoEncoder

- dale

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/b13d7178-4023-4c9a-af22-197c836bc5d0n%40chromium.org.

[Wojciech Dzierżanowski]

Could the encoder latency mode affect this?

I've just noticed (by accident, while working on something else
related to MediaRecorder) that enabling
MediaRecorderUseMediaVideoEncoder changes the latency mode for
hardware encoders within MediaRecorder:
https://issues.chromium.org/issues/373267375.

Cheers,
Wojciech

> You received this message because you are subscribed to the Google Groups "media-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to media-dev+...@chromium.org.
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/CAPUDrwd%3DAB%3DiJY1EwU%3DPybVh0S-cGL%2BtkoXASzt62EQ8Ey3cHg%40mail.gmail.com.

[Robert Jamieson]

Hi,

Thanks for the suggestions. I tested with both MediaRecorderUseMediaVideoEncoder enabled and disabled but didn't see a difference.

We did however manage to recreate the issue internally. We saw the issue occur on a 2020 MacBook Pro and also on a Windows Lenovo machine. The common point between both machines was neither had hardware encoding support for VP9.

I have now been able to recreate the issue on my dev Windows machine by disabling hardware encode in flags. The test I am performing is rendering a static image to a canvas at 30fps and using that as the video track for the media recorder. The videoBitsPerSecond are being set to 2.5Mb/s in the MediaRecorder ctor, with the peak bitrate being set to 5Mb/s based on a magnitude of 2 in vea_encoder.cc (discussed here https://issues.chromium.org/issues/40851972). I am then recording for 40secs.

When hardware encoding is enabled I see the encoder (media\gpu\windows\media_foundation_video_encode_accelerator_win.cc) configured to Variable BitRate with One Pass. The max QP is set to 56 and the min QP set to 2 with CODECAPI_AVEncCommonMeanBitRate and MF_MT_AVG_BITRATE set to 2500000 and CODECAPI_AVEncCommonMaxBitRate set to 5000000.
It results in a 40 sec file with a bitrate of 3Mb/s.

When hardware encoding is disabled I see encoder (media\video\vpx_video_encoder.cc) set to Variable BitRate with One Pass. The max QP set to 58 and the min QP set to 2 with rc_target_bitrate set to 2500 (kb rather than bytes). There is no peak bitrate set, however rc_overshoot_pct and rc_undershoot_pct are set to 25%.
It results in a 40 sec file with a bitrate of 9.5Mb/s almost 4x the desired bitrate.

With a static image I would have expected maximal quality with almost maximal compression since there is zero motion between frames. I did notice however that there is a hint for identifying screen capture scenarios. According to the comment this is due to bitrates way overshooting the desired bitrates in certain screen capture scenarios (https://issues.webrtc.org/issues/42226143). Since the Media Recorder can be recording any type of content, from high motion to static images, is it always in danger of large bitrate variance or is it possible to set a peak to prevent large spikes?

Thanks,
Robert

[Robert Jamieson]

I have a minimal, reproducible sample app that demonstrates high bitrates with VP9 when using software encoding.

Should I raise a bug and provide the sample app?

Thanks,
Robert

---

From: Markus Handell <hand...@google.com>
Sent: 25 October 2024 09:31
To: Robert Jamieson <rjam...@all-turtles.com>; Sergey Silkin <ssi...@google.com>
Cc: Wojciech Dzierżanowski <wdzierz...@opera.com>; Dale Curtis <dalec...@chromium.org>; Eugene Zemtsov <eug...@chromium.org>; Chromium-dev <chromi...@chromium.org>; media-dev <medi...@chromium.org>
Subject: Re: [chromium-dev] MediaRecorder VP9 Bitrate spikes

 

I don't think bitrate management has gotten a lot of attention in MediaRecorder. Values you get out seem way out of wack.

CC @Sergey Silkin 

On Thu, Oct 24, 2024 at 7:47 PM Robert Jamieson <rjam...@all-turtles.com> wrote:

Hi,

Thanks for the suggestions. I tested with both MediaRecorderUseMediaVideoEncoder enabled and disabled but didn't see a difference.

We did however manage to recreate the issue internally. We saw the issue occur on a 2020 MacBook Pro and also on a Windows Lenovo machine. The common point between both machines was neither had hardware encoding support for VP9.

I have now been able to recreate the issue on my dev Windows machine by disabling hardware encode in flags. The test I am performing is rendering a static image to a canvas at 30fps and using that as the video track for the media recorder. The videoBitsPerSecond are being set to 2.5Mb/s in the MediaRecorder ctor, with the peak bitrate being set to 5Mb/s based on a magnitude of 2 in vea_encoder.cc (discussed here https://issues.chromium.org/issues/40851972). I am then recording for 40secs.

When hardware encoding is enabled I see the encoder (media\gpu\windows\media_foundation_video_encode_accelerator_win.cc) configured to Variable BitRate with One Pass. The max QP is set to 56 and the min QP set to 2 with CODECAPI_AVEncCommonMeanBitRate and MF_MT_AVG_BITRATE set to 2500000 and CODECAPI_AVEncCommonMaxBitRate set to 5000000.
It results in a 40 sec file with a bitrate of 3Mb/s.

When hardware encoding is disabled I see encoder (media\video\vpx_video_encoder.cc) set to Variable BitRate with One Pass. The max QP set to 58 and the min QP set to 2 with rc_target_bitrate set to 2500 (kb rather than bytes). There is no peak bitrate set, however rc_overshoot_pct and rc_undershoot_pct are set to 25%.
It results in a 40 sec file with a bitrate of 9.5Mb/s almost 4x the desired bitrate.

With a static image I would have expected maximal quality with almost maximal compression since there is zero motion between frames. I did notice however that there is a hint for identifying screen capture scenarios. According to the comment this is due to bitrates way overshooting the desired bitrates in certain screen capture scenarios (https://issues.webrtc.org/issues/42226143). Since the Media Recorder can be recording any type of content, from high motion to static images, is it always in danger of large bitrate variance or is it possible to set a peak to prevent large spikes?

Thanks,
Robert

> On 14 Oct 2024, at 16:39, Wojciech Dzierżanowski <wdzierz...@opera.com> wrote:
>
> Could the encoder latency mode affect this?

[mark a. foltz]

Yes.  Please file an issue here:

https://issues.chromium.org/u/1/issues/new?component=1456201&template=0

I don't see an auto-CC for this component, so please assign it to mfo...@chromium.org and I will route.

m.

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/LO2P265MB717110336B0F81906D74CF79A0552%40LO2P265MB7171.GBRP265.PROD.OUTLOOK.COM.

[Robert Jamieson]

I've filed the issue here https://issues.chromium.org/u/0/issues/376707227

Thanks,
Robert