Reland "Support SW BRC for HEVC encoding on Windows." [chromium/src : main]

0 views
Skip to first unread message

Hirokazu Honda (Gerrit)

unread,
Jul 3, 2024, 4:50:57 AM (2 days ago) Jul 3
to Jianlin Qiu, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
Attention needed from Eugene Zemtsov, Jianlin Qiu and Sergey Silkin

Hirokazu Honda added 1 comment

Patchset-level comments
File-level comment, Patchset 2 (Latest):
Hirokazu Honda . resolved

Could you provide the quality and performance values with software rate controller?

Open in Gerrit

Related details

Attention is currently required from:
  • Eugene Zemtsov
  • Jianlin Qiu
  • Sergey Silkin
Submit Requirements:
  • requirement satisfiedCode-Coverage
  • requirement is not satisfiedCode-Review
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
Gerrit-MessageType: comment
Gerrit-Project: chromium/src
Gerrit-Branch: main
Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
Gerrit-Change-Number: 5673763
Gerrit-PatchSet: 2
Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
Gerrit-CC: Erik Språng <spr...@chromium.org>
Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
Gerrit-Attention: Jianlin Qiu <jianl...@intel.com>
Gerrit-Comment-Date: Wed, 03 Jul 2024 08:50:47 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
satisfied_requirement
unsatisfied_requirement
open
diffy

Jianlin Qiu (Gerrit)

unread,
Jul 3, 2024, 10:11:47 PM (2 days ago) Jul 3
to Hirokazu Honda, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
Attention needed from Eugene Zemtsov, Hirokazu Honda and Sergey Silkin

Jianlin Qiu added 1 comment

Patchset-level comments
Hirokazu Honda . resolved

Could you provide the quality and performance values with software rate controller?

Jianlin Qiu

I think we need to rework video_encode_accelerator_test to get it working for Windows as well. I can report PSNR/SSIM back once that is done.

Encoding time is nearly the same at 720p on NV/Intel platform comparing to driver's own BRC at the same target bitrate(1.9Mbps), and average QP is 21 for SW BRC vs 20.9 without SW BRC on NV, while for Intel SW BRC is a must as latest driver does not support changing bitrate in CBR mode(fix likely to be rolled out in July); For Intel at the same 1.9Mbps target bitrate, QP is around 22.5.

Open in Gerrit

Related details

Attention is currently required from:
  • Eugene Zemtsov
  • Hirokazu Honda
  • Sergey Silkin
Submit Requirements:
  • requirement satisfiedCode-Coverage
  • requirement is not satisfiedCode-Review
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
Gerrit-MessageType: comment
Gerrit-Project: chromium/src
Gerrit-Branch: main
Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
Gerrit-Change-Number: 5673763
Gerrit-PatchSet: 4
Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
Gerrit-CC: Erik Språng <spr...@chromium.org>
Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
Gerrit-Attention: Hirokazu Honda <hi...@chromium.org>
Gerrit-Comment-Date: Thu, 04 Jul 2024 02:11:33 +0000
Gerrit-HasComments: Yes
Gerrit-Has-Labels: No
Comment-In-Reply-To: Hirokazu Honda <hi...@chromium.org>
satisfied_requirement
unsatisfied_requirement
open
diffy

Hirokazu Honda (Gerrit)

unread,
Jul 4, 2024, 1:18:08 AM (yesterday) Jul 4
to Jianlin Qiu, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
Attention needed from Eugene Zemtsov, Jianlin Qiu and Sergey Silkin

Hirokazu Honda added 2 comments

Patchset-level comments
Hirokazu Honda . resolved

Could you provide the quality and performance values with software rate controller?

Jianlin Qiu

I think we need to rework video_encode_accelerator_test to get it working for Windows as well. I can report PSNR/SSIM back once that is done.

Encoding time is nearly the same at 720p on NV/Intel platform comparing to driver's own BRC at the same target bitrate(1.9Mbps), and average QP is 21 for SW BRC vs 20.9 without SW BRC on NV, while for Intel SW BRC is a must as latest driver does not support changing bitrate in CBR mode(fix likely to be rolled out in July); For Intel at the same 1.9Mbps target bitrate, QP is around 22.5.

Hirokazu Honda

How is the produced bitrate?
Is it sufficiently near to the target bitrate?

I need to learn the software rate controller implementation.
Could you give the rationale why the h264 rate controller can be used for h265?

File media/gpu/windows/media_foundation_video_encode_accelerator_win.cc
Line 1727, Patchset 4 (Latest): } else {
quantizer = QindextoAVEncQP(metadata_qp.value());
}
#else
quantizer = QindextoAVEncQP(metadata_qp.value());
#endif // BUILDFLAG(ENABLE_PLATFORM_HEVC)
Hirokazu Honda . unresolved
```
} else
#endif //
{
quantizer = QindextoAVEncQP(metadata_qp.value());
}
```
Open in Gerrit

Related details

Attention is currently required from:
  • Eugene Zemtsov
  • Jianlin Qiu
  • Sergey Silkin
Submit Requirements:
    • requirement satisfiedCode-Coverage
    • requirement is not satisfiedCode-Review
    • requirement is not satisfiedNo-Unresolved-Comments
    Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
    Gerrit-MessageType: comment
    Gerrit-Project: chromium/src
    Gerrit-Branch: main
    Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
    Gerrit-Change-Number: 5673763
    Gerrit-PatchSet: 4
    Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
    Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-CC: Erik Språng <spr...@chromium.org>
    Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
    Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
    Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Attention: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Comment-Date: Thu, 04 Jul 2024 05:17:58 +0000
    Gerrit-HasComments: Yes
    Gerrit-Has-Labels: No
    Comment-In-Reply-To: Jianlin Qiu <jianl...@intel.com>
    Comment-In-Reply-To: Hirokazu Honda <hi...@chromium.org>
    satisfied_requirement
    unsatisfied_requirement
    open
    diffy

    Jianlin Qiu (Gerrit)

    unread,
    Jul 4, 2024, 2:21:04 AM (yesterday) Jul 4
    to Hirokazu Honda, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
    Attention needed from Eugene Zemtsov, Hirokazu Honda and Sergey Silkin

    Jianlin Qiu added 2 comments

    Patchset-level comments
    Hirokazu Honda . resolved

    Could you provide the quality and performance values with software rate controller?

    Jianlin Qiu

    I think we need to rework video_encode_accelerator_test to get it working for Windows as well. I can report PSNR/SSIM back once that is done.

    Encoding time is nearly the same at 720p on NV/Intel platform comparing to driver's own BRC at the same target bitrate(1.9Mbps), and average QP is 21 for SW BRC vs 20.9 without SW BRC on NV, while for Intel SW BRC is a must as latest driver does not support changing bitrate in CBR mode(fix likely to be rolled out in July); For Intel at the same 1.9Mbps target bitrate, QP is around 22.5.

    Hirokazu Honda

    How is the produced bitrate?
    Is it sufficiently near to the target bitrate?

    I need to learn the software rate controller implementation.
    Could you give the rationale why the h264 rate controller can be used for h265?

    Jianlin Qiu

    Histograms captured for a 2-minutes' HEVC session shown below. According to the result, SW BRC performs better than HW BRC.

    The reason that the rate controller can be reused is that they follow the similar HRD requirement. RD model could be different for intra-frames due to coding efficiency, but should basically be less different for inter-frames. It would be ideal for WebRTC usage which uses open-GOP for H.265.

    With SW BRC:

    • Histogram: WebRTC.Video.RMSEOfEncodingBitrateInKbps.H265 recorded 4 samples, mean = 13.2 (flags = 0x41) [#]
    • 0 ...
    • 4 -O (1 = 25.0%) {0.0%}
    • 5 ...
    • 12 -O (1 = 25.0%) {25.0%}
    • 14 O (0 = 0.0%) {50.0%}
    • 17 -O (1 = 25.0%) {50.0%}
    • 20 -O (1 = 25.0%) {75.0%}
    • 24 ...

    With HW BRC:

    • Histogram: WebRTC.Video.RMSEOfEncodingBitrateInKbps.H265 recorded 4 samples, mean = 16.8 (flags = 0x41) [#]
    • 0 ...
    • 8 -O (1 = 25.0%) {0.0%}
    • 10 ...
    • 14 -O (1 = 25.0%) {25.0%}
    • 17 -O (1 = 25.0%) {50.0%}
    • 20 -O (1 = 25.0%) {75.0%}
    • 24 ...
    File media/gpu/windows/media_foundation_video_encode_accelerator_win.cc

    quantizer = QindextoAVEncQP(metadata_qp.value());
    }
    #else
    quantizer = QindextoAVEncQP(metadata_qp.value());
    #endif // BUILDFLAG(ENABLE_PLATFORM_HEVC)
    Hirokazu Honda . resolved
    ```
    } else
    #endif //
    {
    quantizer = QindextoAVEncQP(metadata_qp.value());
    }
    ```
    Jianlin Qiu

    Done

    Open in Gerrit

    Related details

    Attention is currently required from:
    • Eugene Zemtsov
    • Hirokazu Honda
    • Sergey Silkin
    Submit Requirements:
    • requirement satisfiedCode-Coverage
    • requirement is not satisfiedCode-Review
    Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
    Gerrit-MessageType: comment
    Gerrit-Project: chromium/src
    Gerrit-Branch: main
    Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
    Gerrit-Change-Number: 5673763
    Gerrit-PatchSet: 5
    Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
    Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-CC: Erik Språng <spr...@chromium.org>
    Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
    Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
    Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Attention: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Comment-Date: Thu, 04 Jul 2024 06:20:53 +0000
    satisfied_requirement
    unsatisfied_requirement
    open
    diffy

    Hirokazu Honda (Gerrit)

    unread,
    Jul 4, 2024, 4:31:26 AM (24 hours ago) Jul 4
    to Jianlin Qiu, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
    Attention needed from Eugene Zemtsov, Jianlin Qiu and Sergey Silkin

    Hirokazu Honda voted and added 4 comments

    Votes added by Hirokazu Honda

    Code-Review+1

    4 comments

    Patchset-level comments
    Hirokazu Honda

    Thanks. It would be great to have more evaluation by launch.

    File-level comment, Patchset 5 (Latest):
    Hirokazu Honda . resolved

    LGTM. I defer to Eugene for the final approval.

    File media/gpu/windows/media_foundation_video_encode_accelerator_win.cc
    Line 399, Patchset 5 (Latest): }
    Hirokazu Honda . unresolved

    Change the supported temporal layers for H265?

    Line 1725, Patchset 5 (Latest): metadata_qp = std::clamp(metadata_qp.value(), 1, kH26xMaxQp);
    Hirokazu Honda . unresolved

    Why not 0?

    Open in Gerrit

    Related details

    Attention is currently required from:
    • Eugene Zemtsov
    • Jianlin Qiu
    • Sergey Silkin
    Submit Requirements:
    • requirement satisfiedCode-Coverage
    • requirement satisfiedCode-Review
    • requirement is not satisfiedNo-Unresolved-Comments
    Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
    Gerrit-MessageType: comment
    Gerrit-Project: chromium/src
    Gerrit-Branch: main
    Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
    Gerrit-Change-Number: 5673763
    Gerrit-PatchSet: 5
    Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
    Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-CC: Erik Språng <spr...@chromium.org>
    Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
    Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
    Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Attention: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Comment-Date: Thu, 04 Jul 2024 08:31:14 +0000
    Gerrit-HasComments: Yes
    Gerrit-Has-Labels: Yes
    satisfied_requirement
    unsatisfied_requirement
    open
    diffy

    Jianlin Qiu (Gerrit)

    unread,
    Jul 4, 2024, 6:11:56 AM (22 hours ago) Jul 4
    to Hirokazu Honda, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
    Attention needed from Eugene Zemtsov, Hirokazu Honda and Sergey Silkin

    Jianlin Qiu added 2 comments

    File media/gpu/windows/media_foundation_video_encode_accelerator_win.cc
    Hirokazu Honda . unresolved

    Change the supported temporal layers for H265?

    Jianlin Qiu

    This will be done in a separate CL. It's not simply setting it to 2 or 3.

    Line 1725, Patchset 5 (Latest): metadata_qp = std::clamp(metadata_qp.value(), 1, kH26xMaxQp);
    Hirokazu Honda . unresolved

    Why not 0?

    Jianlin Qiu

    We're just following H.264 approach for WebCodecs CQP encoding above. Unless we want the encode to work in lossless mode, this should not be 0.

    Open in Gerrit

    Related details

    Attention is currently required from:
    • Eugene Zemtsov
    • Hirokazu Honda
    • Sergey Silkin
    Gerrit-Attention: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Comment-Date: Thu, 04 Jul 2024 10:11:42 +0000
    Gerrit-HasComments: Yes
    Gerrit-Has-Labels: No
    Comment-In-Reply-To: Hirokazu Honda <hi...@chromium.org>
    satisfied_requirement
    unsatisfied_requirement
    open
    diffy

    Hirokazu Honda (Gerrit)

    unread,
    2:12 AM (2 hours ago) 2:12 AM
    to Jianlin Qiu, chromium...@chromium.org, Sergey Silkin, Eugene Zemtsov, Chromium LUCI CQ, Erik Språng, Philip Eliasson, Mosa Morosev, AyeAye, chromeos-gfx-...@google.com, media-cro...@chromium.org, feature-me...@chromium.org, media-wi...@chromium.org
    Attention needed from Eugene Zemtsov, Jianlin Qiu and Sergey Silkin

    Hirokazu Honda added 2 comments

    File media/gpu/windows/media_foundation_video_encode_accelerator_win.cc
    Hirokazu Honda . resolved

    Change the supported temporal layers for H265?

    Jianlin Qiu

    This will be done in a separate CL. It's not simply setting it to 2 or 3.

    Hirokazu Honda

    Acknowledged

    Line 1725, Patchset 5 (Latest): metadata_qp = std::clamp(metadata_qp.value(), 1, kH26xMaxQp);
    Hirokazu Honda . resolved

    Why not 0?

    Jianlin Qiu

    We're just following H.264 approach for WebCodecs CQP encoding above. Unless we want the encode to work in lossless mode, this should not be 0.

    Hirokazu Honda

    Acknowledged

    Open in Gerrit

    Related details

    Attention is currently required from:
    • Eugene Zemtsov
    • Jianlin Qiu
    • Sergey Silkin
    Submit Requirements:
    • requirement satisfiedCode-Coverage
    • requirement satisfiedCode-Review
    Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. DiffyGerrit
    Gerrit-MessageType: comment
    Gerrit-Project: chromium/src
    Gerrit-Branch: main
    Gerrit-Change-Id: If34865cf7573a1579aa9d83a988345bfd8c0f579
    Gerrit-Change-Number: 5673763
    Gerrit-PatchSet: 5
    Gerrit-Owner: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>
    Gerrit-Reviewer: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Reviewer: Hirokazu Honda <hi...@chromium.org>
    Gerrit-Reviewer: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Reviewer: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-CC: Erik Språng <spr...@chromium.org>
    Gerrit-CC: Mosa Morosev <mosam...@microsoft.com>
    Gerrit-CC: Philip Eliasson <phil...@webrtc.org>
    Gerrit-Attention: Sergey Silkin <ssi...@webrtc.org>
    Gerrit-Attention: Eugene Zemtsov <eug...@chromium.org>
    Gerrit-Attention: Jianlin Qiu <jianl...@intel.com>
    Gerrit-Comment-Date: Fri, 05 Jul 2024 06:12:04 +0000
    Gerrit-HasComments: Yes
    Gerrit-Has-Labels: No
    satisfied_requirement
    open
    diffy
    Reply all
    Reply to author
    Forward
    0 new messages