Video: Enable HEVC bitstream converter for Media Foundation for clear [chromium/src : main]

Jiawei Chen (Gerrit)

unread,

Jul 16, 2023, 2:48:42 PM7/16/23

to Dale Curtis, feature-me...@chromium.org, poscia...@chromium.org

Attention is currently required from: Dale Curtis.

Jiawei Chen would like Dale Curtis to review this change.

Video: Enable HEVC bitstream converter for Media Foundation for clear

By default, bitstream converter(to Annex-B) is not enabled for HEVC when
use media foundation for clear, but it's reqiurenment of HEVC deocder in
MediaFoundation.
Refer to detail in this page:
https://learn.microsoft.com/en-us/windows/win32/medfound/h-265---hevc-video-decoder

This change only affect ffmpeg_demuxer used case.

Bug: 1465217
Change-Id: Ie8f44aea08cd2fe7d2e0d98e84f177e7abc375bd
---
M media/renderers/win/media_foundation_video_stream.cc
M media/renderers/win/media_foundation_video_stream.h
2 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/media/renderers/win/media_foundation_video_stream.cc b/media/renderers/win/media_foundation_video_stream.cc
index f92d8db..a6e2685 100644
--- a/media/renderers/win/media_foundation_video_stream.cc
+++ b/media/renderers/win/media_foundation_video_stream.cc
@@ -381,6 +381,14 @@
 #endif  // BUILDFLAG(USE_PROPRIETARY_CODECS)
 
 #if BUILDFLAG(ENABLE_PLATFORM_HEVC)
+HRESULT MediaFoundationHEVCVideoStream::GetMediaType(
+    IMFMediaType** media_type_out) {
+  RETURN_IF_FAILED(MediaFoundationVideoStream::GetMediaType(media_type_out));
+  // Enable conversion to Annex-B
+  demuxer_stream_->EnableBitstreamConverter();
+  return S_OK;
+}
+
 bool MediaFoundationHEVCVideoStream::AreFormatChangesEnabled() {
   // Disable explicit format change event for HEVC to allow switching to the
   // new stream without a full re-create, which will be much faster. This is
diff --git a/media/renderers/win/media_foundation_video_stream.h b/media/renderers/win/media_foundation_video_stream.h
index c2c044e0..a356c03 100644
--- a/media/renderers/win/media_foundation_video_stream.h
+++ b/media/renderers/win/media_foundation_video_stream.h
@@ -42,6 +42,7 @@
 // The HEVC specific video stream.
 class MediaFoundationHEVCVideoStream : public MediaFoundationVideoStream {
  protected:
+  HRESULT GetMediaType(IMFMediaType** media_type_out) override;
   bool AreFormatChangesEnabled() override;
 };
 #endif  // BUILDFLAG(ENABLE_PLATFORM_HEVC) ||

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Jiawei Chen (Gerrit)

unread,

Jul 16, 2023, 2:48:45 PM7/16/23

to feature-me...@chromium.org, poscia...@chromium.org, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Dale Curtis.

View Change

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Dale Curtis (Gerrit)

unread,

Jul 17, 2023, 12:48:58 PM7/17/23

to William Carr, feature-me...@chromium.org, poscia...@chromium.org, Jiawei Chen

Attention is currently required from: William Carr.

Dale Curtis would like William Carr to review this change authored by Jiawei Chen.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Dale Curtis (Gerrit)

unread,

Jul 17, 2023, 12:49:25 PM7/17/23

to Xiaohan Wang, feature-me...@chromium.org, poscia...@chromium.org, Jiawei Chen, William Carr

Attention is currently required from: Jiawei Chen, William Carr, Xiaohan Wang.

Dale Curtis would like Xiaohan Wang to review this change authored by Jiawei Chen.

Dale Curtis (Gerrit)

unread,

Jul 17, 2023, 12:49:32 PM7/17/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr, Xiaohan Wang.

View Change

1 comment:

Patchset:
- Patch Set #1:
  I'm surprised Moho would work if this was true. Are we not trying any HEVC + Protected Content?

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Demo (Gerrit)

unread,

Jul 17, 2023, 2:28:27 PM7/17/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Dale Curtis, Jiawei Chen, William Carr, Xiaohan Wang.

View Change

1 comment:

Patchset:
- Patch Set #1:
  I'm surprised Moho would work if this was true. […]
  Yes, I didn't test protected content. the use case might be:
  1. User open a local HEVC video in browser;
  2. User open a online mp4 video hosted on network storage in browser.
  However, cases will be "ffmpeg_demuxer" used and "media foundation for clear" enabled.
  If chunk_demuxer is used, it will has no such problem in this CL's bug.
  BTW, what is Moho, thanks.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Jiawei Chen (Gerrit)

unread,

Jul 17, 2023, 2:30:54 PM7/17/23

to Demo, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Dale Curtis, Jiawei Chen, William Carr, Xiaohan Wang.

Jiawei Chen removed Demo from this change.

View Change

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 17, 2023, 5:25:34 PM7/17/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Dale Curtis, Jiawei Chen, William Carr.

View Change

3 comments:

Commit Message:
- Patch Set #1, Line 7: for clear
  This change isn't specific to clear content, it'll affect protected content as well if FFmpegDemuxer is used.
Patchset:
- Patch Set #1:
  Yes, I didn't test protected content. the use case might be: […]
  Almost all protected content playbacks use MSE with ChunkDemuxer, which doesn't need the conversion. This would explain why we are not seeing this issue so far.
- Patch Set #1:
  Thanks for the fix! The change looks good to me. One question, do we have any test covering the combination of FFmpegDemuxer + HEVC + MediaFoundationRenderer?

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Dale Curtis (Gerrit)

unread,

Jul 17, 2023, 6:47:37 PM7/17/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr, Xiaohan Wang.

View Change

1 comment:

Commit Message:

- Patch Set #1, Line 7: for clear

- This change isn't specific to clear content, it'll affect protected content as well if FFmpegDemuxer […]
  Ah, that makes sense.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Jiawei Chen (Gerrit)

unread,

Jul 18, 2023, 1:40:29 AM7/18/23

to feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: William Carr, Xiaohan Wang.

View Change

1 comment:

Patchset:
- Patch Set #1:
  Thanks for the fix! The change looks good to me. […]
  do you mean unit test or something else?

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 18, 2023, 11:21:45 AM7/18/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr.

View Change

1 comment:

Patchset:
- Patch Set #1:
  do you mean unit test or something else?
  +wic...@microsoft.com
  A pipeline integration test or browser test would work.
  - media/renderers/win/media_foundation_renderer_integration_test.cc
  - https://crbug.com/1442997

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 18, 2023, 1:21:24 PM7/18/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr.

View Change

1 comment:

Patchset:
- Patch Set #1:
  +wic...@microsoft.com […]
  In media_foundation_renderer_integration_test.cc, we should be using FFmpegDemuxer:
  https://source.chromium.org/chromium/chromium/src/+/refs/heads/main:media/test/pipeline_integration_test_base.cc;drc=00599035f99a89298fb1eccf7ee9f5c7706291c9;l=481

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

William Carr (Gerrit)

unread,

Jul 19, 2023, 7:45:45 AM7/19/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen.

View Change

1 comment:

Patchset:
- Patch Set #1:
  Adding Frank.
  Change LGTM in general, but one thing I'm not familiar with is whether we only need to do this for certain streams. (There's plenty of HEVC streams that work with MF right now, and I'm just not sure the interaction for those streams of enabling the converter. Frank - do you happen to know?)

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

William Carr (Gerrit)

unread,

Jul 19, 2023, 7:49:43 AM7/19/23

to Frank Li, feature-me...@chromium.org, poscia...@chromium.org, Jiawei Chen, Xiaohan Wang, Dale Curtis

Attention is currently required from: Frank Li, Jiawei Chen.

William Carr would like Frank Li to review this change authored by Jiawei Chen.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 19, 2023, 11:56:26 AM7/19/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Frank Li, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Frank Li, Jiawei Chen, William Carr.

View Change

1 comment:

Patchset:
- Patch Set #1:
  Adding Frank. […]
  For MSE playback, where ChunkDemuxer is used, the streams are already converted so this CL won't have impact on those. For FFmpegDemuxer it seems like this is needed.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Jiawei Chen (Gerrit)

unread,

Jul 19, 2023, 12:51:42 PM7/19/23

to feature-me...@chromium.org, poscia...@chromium.org, Frank Li, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Frank Li, William Carr, Xiaohan Wang.

View Change

1 comment:

Patchset:
- Patch Set #1:
  In media_foundation_renderer_integration_test.cc, we should be using FFmpegDemuxer: […]
  Thanks for your help, CL updated.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 19, 2023, 5:33:58 PM7/19/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Frank Li, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Frank Li, Jiawei Chen, William Carr.

View Change

3 comments:

Patchset:
- Patch Set #2:
  Great to see more tests!
  lg with two questions. Thanks!
File media/renderers/win/media_foundation_renderer_integration_test.cc:
- Patch Set #2, Line 138: }
  OOC: Did this test fail without your change?
File media/test/data/README.md:
- Patch Set #2, Line 1458: ffmpeg -i bear-1280x720.mp4 -vf "scale=3840:2160,setpts=4*PTS" -c:v libx265 -crf 28 -c:a copy bear-3840x2160-hevc.mp4
  OOC, why this need to be scaled for testing? Will it still work if we don't scale?

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Frank Li (Gerrit)

unread,

Jul 19, 2023, 5:46:48 PM7/19/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr.

View Change

1 comment:

File media/renderers/win/media_foundation_renderer_integration_test.cc:
- Patch Set #2, Line 33: GetAllVideoCodecs
  GetVideoCodecsMap ?

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Jiawei Chen (Gerrit)

unread,

Jul 20, 2023, 1:48:16 AM7/20/23

to feature-me...@chromium.org, poscia...@chromium.org, Frank Li, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Frank Li, William Carr, Xiaohan Wang.

View Change

3 comments:

File media/renderers/win/media_foundation_renderer_integration_test.cc:

- Patch Set #2, Line 33: GetAllVideoCodecs
  GetVideoCodecsMap ?

- Done

- Patch Set #2, Line 138: }
  OOC: Did this test fail without your change?

- Yes, if has no my changes, MediaFoundationRenderer will throw "The Media format is recognized but is invalid. (0xC00D3E8C)", it makes pipeline decode error.
File media/test/data/README.md:

- Patch Set #2, Line 1458: ffmpeg -i bear-1280x720.mp4 -vf "scale=3840:2160,setpts=4*PTS" -c:v libx265 -crf 28 -c:a copy bear-3840x2160-hevc.mp4
  OOC, why this need to be scaled for testing? Will it still work if we don't scale?

- if just use already exist HEVC video(bear-xxx.mp4) in data folder, and without my changes in this CL:
  MediaFoundationRenderer will not throw any error in playback(just no picture present).
  But if I scale the video to 3840x2160, and use setpts to extend it's duration time,
  MediaFoundationRenderer will throw our expected error("kOnPlaybackError (The Media format is recognized but is invalid. (0xC00D3E8C))"
  Then I tried below combinations to use FFMPEG generate videos,
  scale=1920:1080,
  scale=1920:1080,setpts=2*PTS
  scale=3840:2160,
  scale=3840:2160,setpts=2*PTS
  only "scale=3840:2160,setpts=2*PTS" will let MediaFoundation throw our expected error.
  I'm curious about this.
  Hi Frank(@fra...@microsoft.com), do you familiar about HEVC decoder in windows,
  if yes, may you can help to give us some tips here. Thanks.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Frank Li (Gerrit)

unread,

Jul 20, 2023, 10:26:16 AM7/20/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr, Xiaohan Wang.

View Change

1 comment:

File media/test/data/README.md:

- Patch Set #2, Line 1458: ffmpeg -i bear-1280x720.mp4 -vf "scale=3840:2160,setpts=4*PTS" -c:v libx265 -crf 28 -c:a copy bear-3840x2160-hevc.mp4

- if just use already exist HEVC video(bear-xxx. […]
  I have asked Matt from OS decoder team.

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Xiaohan Wang (Gerrit)

unread,

Jul 20, 2023, 8:26:40 PM7/20/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Xiaohan Wang, Frank Li, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr.

Patch set 3:Code-Review +1

View Change

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

朱思达 (Gerrit)

unread,

Jul 21, 2023, 1:39:31 PM7/21/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, Chromium LUCI CQ, Xiaohan Wang, Frank Li, William Carr, Dale Curtis, chromium...@chromium.org

Attention is currently required from: Jiawei Chen, William Carr, 朱思达.

Patch set 3:Commit-Queue +2

View Change

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Chromium LUCI CQ (Gerrit)

unread,

Jul 21, 2023, 1:54:58 PM7/21/23

to Jiawei Chen, feature-me...@chromium.org, poscia...@chromium.org, 朱思达, Xiaohan Wang, Frank Li, William Carr, Dale Curtis, chromium...@chromium.org

Chromium LUCI CQ submitted this change.

View Change

Approvals:
  朱思达: Commit
  Xiaohan Wang: Looks good to me

Video: Enable HEVC bitstream converter for Media Foundation for clear

By default, bitstream converter(to Annex-B) is not enabled for HEVC when
use media foundation for clear, but it's reqiurenment of HEVC deocder in
MediaFoundation.
Refer to detail in this page:
https://learn.microsoft.com/en-us/windows/win32/medfound/h-265---hevc-video-decoder

This change only affect ffmpeg_demuxer used case.

Bug: 1465217
Change-Id: Ie8f44aea08cd2fe7d2e0d98e84f177e7abc375bd

Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4688957
Reviewed-by: Xiaohan Wang <xhw...@chromium.org>
Commit-Queue: 朱思达 <zhu...@bytedance.com>
Cr-Commit-Position: refs/heads/main@{#1173613}
---
M media/renderers/win/media_foundation_renderer_integration_test.cc
M media/renderers/win/media_foundation_video_stream.cc
M media/renderers/win/media_foundation_video_stream.h
M media/test/data/README.md
A media/test/data/bear-3840x2160-hevc.mp4
M media/test/media_bundle_data.filelist
M media/unit_tests_bundle_data.filelist
7 files changed, 52 insertions(+), 5 deletions(-)

To view, visit change 4688957. To unsubscribe, or for help writing mail filters, visit settings.

Reply all

Reply to author

Forward