hw deocding of h264 : linux, vaapi, wayland

[Robin Carlisle]

Hi,

Not sure if this is the right place to ask this ... please direct me elsewhere if not.

Looking at this merged work : https://chromium-review.googlesource.com/c/chromium/src/+/3646633

it looks like hardware decoding of h264 under linux/wayland should be operational?   Is this the correct assumption?    

I tried this on ubuntu 22.04 and 24.04.   22.04 has a runtime vaapi that is older than the buildtime one - so that failed (IsLibVACompatible).   24.04 has a new enough version so I continued testing from there

I have tried the chromium snap as well as my own debug build and release build.   I have tried 133 and 135.

I have tried with the following :

tmp=$(mktemp -d)

bindir=$HOME/distrib/chrome/135.0.7021.0/Debug

$bindir/chrome \

  --no-sandbox \

  --ozone-platform=wayland \

  --use-gl=angle --use-angle=gl \

  --user-data-dir=${tmp} \

  --enable-features=VaapiVideoDecodeLinuxGL \

  --enable-logging=stderr --v=0 --vmodule='*media*=1,*vaapi*=3' \

  --remote-allow-origins='*' --remote-debugging-port=9222 \

  --kiosk file:///home/labs/videos/gradtest_v1_h264_main_20MBit.mp4

I have tried many other arguments, enabling/disabling features etc.

intel_gpu_top confirms the lack of HW decoding, and top backs this up with high chrome CPU usage.

Vaapi Info as reported by chrome (I added the BUILDTIME log message for debugging purposes) :

VAAPI RUNTIME version: 1.20 Intel iHD driver for Intel(R) Gen Graphics - 24.1.0 ()

VAAPI BUILDTIME version: 1.17

I am quite new to debugging chromium although I have built it (and CEF) quite a few times.   I am struggling to get any more out of the logs that will help me understand why this isnt working.   If I enable too much logging it's a flood of info and I don't know how to filter it down as I don't know where the location of the log message might be - and therefore its vmodule search pattern.  The most relevant logs I have from the above command is :

[334305:334305:0226/105707.450513:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"event":"kLoad","url":"file:///home/labs/videos/gradtest_v1_h264_main_20MBit.mp4"}
[334305:334305:0226/105707.453421:VERBOSE1:web_media_player_impl.cc(1253)] SetLatencyHint(nan)
[334163:334163:0226/105707.759954:WARNING:idle_linux.cc(110)] None of the known D-Bus ScreenSaver services could be used.
[334163:334163:0226/105707.852907:ERROR:wayland_surface.cc(499)] Not implemented reached in std::optional<bool> ui::WaylandSurface::SetExplicitSync()
[334163:334163:0226/105707.852957:ERROR:wayland_surface.cc(439)] Not implemented reached in bool ui::WaylandSurface::SetExplicitSyncLegacy()
[334305:334305:0226/105707.998835:VERBOSE1:web_media_player_impl.cc(2869)] MultiBufferDataSourceInitialized
[334305:334305:0226/105707.999207:VERBOSE1:web_media_player_impl.cc(2851)] DataSourceInitialized
[334163:334163:0226/105708.001289:VERBOSE1:frameless_media_interface_proxy.cc(40)] Add
[334305:334339:0226/105708.033516:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"event":"kPipelineStateChange","pipeline_state":"kStarting"}
[334305:334339:0226/105708.038179:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"FFmpegDemuxer: created video stream, config codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}"}
[334305:334339:0226/105708.038686:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"FFmpegDemuxer: created audio stream, config codec: aac, profile: unknown, bytes_per_channel: 4, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Float 32-bit planar, bytes_per_frame: 8, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true, target_output_channel_layout: NONE, target_output_sample_format: Unknown sample format, has aac extra data: true"}
[334305:334339:0226/105708.038915:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"duration":30.0,"event":"kDurationChanged"}
[334163:334182:0226/105708.040624:VERBOSE1:media_stream_manager.cc(1501)] RFAOSF::RequestDeviceAuthorization({device_id=}) [process_id=5, frame_id=1]
[334305:334305:0226/105708.045985:VERBOSE1:web_media_player_impl.cc(2079)] OnMetadata
[334305:334305:0226/105708.049789:VERBOSE1:web_media_player_impl.cc(3103)] SetReadyState(1)
[334305:334305:0226/105708.050539:VERBOSE1:web_media_player_impl.cc(1054)] Pause
[334305:334305:0226/105708.050750:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"event":"kPause"}
[334163:334182:0226/105708.467451:VERBOSE1:media_stream_manager.cc(1501)] RFAOSF::AuthorizationCompleted({status=OK}, {params=format: PCM_LOW_LATENCY, channel_layout: 3, channels: 2, sample_rate: 44100, frames_per_buffer: 512, effects: NONE, mic_positions: }, {device_id=default}) [process_id=5, frame_id=1]
[334163:334182:0226/105708.467511:VERBOSE1:media_stream_manager.cc(1501)] RFAOSF::AuthorizationCompleted => (authorization time=426 ms) [process_id=5, frame_id=1]
[334305:334339:0226/105708.469073:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Selected FFmpegAudioDecoder for audio decoding, config: codec: aac, profile: unknown, bytes_per_channel: 4, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Float 32-bit planar, bytes_per_frame: 8, seek_preroll: 0us, codec_delay: 0, has extra data: true, encryption scheme: Unencrypted, discard decoder delay: true, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true"}
[334305:334339:0226/105708.470003:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select VaapiVideoDecoder for video decoding"}
[334305:334339:0226/105708.470303:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select DecryptingVideoDecoder for video decoding"}
[334305:334305:0226/105708.470527:VERBOSE1:media_interface_factory.cc(70)] CreateVideoDecoder
[334305:334339:0226/105708.470622:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select VpxVideoDecoder for video decoding"}
[334305:334305:0226/105708.470634:VERBOSE1:media_interface_factory.cc(225)] GetMediaInterfaceFactory
[334305:334339:0226/105708.470855:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select Dav1dVideoDecoder for video decoding"}
[334163:334163:0226/105708.471421:VERBOSE1:media_interface_proxy.cc(235)] MediaInterfaceProxy
[334163:334182:0226/105708.474170:VERBOSE1:media_stream_manager.cc(1501)] AMB::MakeAudioOutputStream({device_id=}, {params=[format: PCM_LOW_LATENCY, channel_layout: 3, channels: 2, sample_rate: 44100, frames_per_buffer: 1024, effects: MULTIZONE, mic_positions: ]})
[334163:334182:0226/105708.474306:VERBOSE1:media_stream_manager.cc(1501)] PAOS::PulseAudioOutputStream({device_id=default}, {params=[format: PCM_LOW_LATENCY, channel_layout: 3, channels: 2, sample_rate: 44100, frames_per_buffer: 1024, effects: MULTIZONE, mic_positions: ]}) [this=0x3384000a6e80]
[334163:334182:0226/105708.474592:VERBOSE1:media_stream_manager.cc(1501)] AMB::MakeAudioOutputStream => (number of streams=1)
[334163:334182:0226/105708.474684:VERBOSE1:media_stream_manager.cc(1501)] PAOS::Open() [this=0x3384000a6e80]
[334305:334339:0226/105708.475865:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Selected FFmpegVideoDecoder for video decoding, config: codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}"}
[334305:334339:0226/105708.476302:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"event":"kPipelineStateChange","pipeline_state":"kPlaying"}

Anyone have any ideas? I do not know where to go from here to find out why the VaapiDecoder is not being selected

Thanks v much,

Robin

[Dale Curtis]

+Ted (Chromium) Meyer 

Unfortunately, hardware decoding on Linux has proved quite difficult to keep running under the variety of Linux configurations out there. We've really only focused on getting Intel iGPU systems working. The distro builds are generally not provided by Google and often have custom patches to make this work to some degree of success.

If you're not using an official build does one from https://www.google.com/chrome work any better? I'd guess not, but it's worth trying if you're on the narrow path we've lit up (Intel iGPU).

- dale

--
You received this message because you are subscribed to the Google Groups "media-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to media-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/be4340f3-6b08-4010-a6d0-ec2c1987fe68n%40chromium.org.

[Robin Carlisle]

Hi,  Thanks for your reply.

I did try the official build from google and that also did not work :-(

I have been running HW accel on AMD and intel since 113 using some patches from geniuses at Igalia - but am now eager to wean myself off those and upgrade chromium.  

Weird thing is ... even on the narrow path (Intel iGPU) that you support - it still doesnt work.    Is there some other component that also needs updating?  Some other driver or something.. mesa?  libva?

Thanks again,

Robin

[Robin Carlisle]

Grabbed a few more logs from where VAAPI is being rejected ...  but I cannot figure out why.    I think a return code of 0 would mean the decoder is accepted, a 202 looks to have rejected VAAPI.   I have been digging into the code but it is quite deep so I am making very slow progress.   I guess I could learn how to get the debugger out!

[252577:252608:0227/145015.298942:VERBOSE1:renderer_impl.cc(445)] InitializeVideoRenderer
[252577:252608:0227/145015.299148:VERBOSE1:decoder_stream.cc(120)] DecoderStream<video>
[252577:252608:0227/145015.299273:VERBOSE1:decoder_stream.cc(157)] Initialize<video>
[252577:252608:0227/145015.299450:VERBOSE2:decoder_selector.cc(98)] Initialize
[252577:252608:0227/145015.299627:VERBOSE2:decoder_selector.cc(113)] SelectDecoderInternal
[252577:252608:0227/145015.299888:VERBOSE1:mojo_video_decoder.cc(105)] MojoVideoDecoder
[252577:252608:0227/145015.300035:VERBOSE1:ffmpeg_video_decoder.cc(135)] FFmpegVideoDecoder
[252577:252608:0227/145015.300140:VERBOSE2:decoder_selector.cc(198)] GetAndInitializeNextDecoder
[252577:252608:0227/145015.300248:VERBOSE2:decoder_selector.cc(224)] GetAndInitializeNextDecoder: initializing Unknown Video Decoder
[252577:252608:0227/145015.300426:VERBOSE1:mojo_video_decoder.cc(145)] Initialize
[252577:252608:0227/145015.300716:VERBOSE2:decoder_stream_traits.cc(238)] Decoder initialization failed.
[252577:252608:0227/145015.300818:VERBOSE2:decoder_selector.cc(237)] OnDecoderInitializeDone: VaapiVideoDecoder success=202
[252577:252608:0227/145015.300921:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select VaapiVideoDecoder for video decoding"}
[252577:252608:0227/145015.301020:VERBOSE1:mojo_video_decoder.cc(111)] ~MojoVideoDecoder
[252577:252608:0227/145015.301124:VERBOSE2:decoder_selector.cc(198)] GetAndInitializeNextDecoder
[252577:252608:0227/145015.301201:VERBOSE2:decoder_selector.cc(224)] GetAndInitializeNextDecoder: initializing DecryptingVideoDecoder
[252577:252608:0227/145015.301293:VERBOSE2:decrypting_video_decoder.cc(41)] Initialize: codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}
[252577:252608:0227/145015.301574:VERBOSE2:decoder_stream_traits.cc(238)] Decoder initialization failed.
[252624:252624:0227/145015.301651:VERBOSE1:audio_latency.cc(168)] Using sink output buffer size: 441
[252577:252608:0227/145015.301685:VERBOSE2:decoder_selector.cc(237)] OnDecoderInitializeDone: DecryptingVideoDecoder success=203
[252577:252608:0227/145015.301851:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select DecryptingVideoDecoder for video decoding"}
[252577:252608:0227/145015.301968:VERBOSE2:decoder_selector.cc(198)] GetAndInitializeNextDecoder
[252577:252608:0227/145015.302014:VERBOSE2:decoder_selector.cc(224)] GetAndInitializeNextDecoder: initializing VpxVideoDecoder
[252577:252608:0227/145015.302060:VERBOSE1:vpx_video_decoder.cc(147)] Initialize: codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}
[252577:252608:0227/145015.302205:VERBOSE2:decoder_stream_traits.cc(238)] Decoder initialization failed.
[252577:252608:0227/145015.302254:VERBOSE2:decoder_selector.cc(237)] OnDecoderInitializeDone: VpxVideoDecoder success=202
[252577:252608:0227/145015.302297:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select VpxVideoDecoder for video decoding"}
[252577:252608:0227/145015.302352:VERBOSE2:decoder_selector.cc(198)] GetAndInitializeNextDecoder
[252577:252608:0227/145015.302382:VERBOSE2:decoder_selector.cc(224)] GetAndInitializeNextDecoder: initializing Dav1dVideoDecoder
[252577:252608:0227/145015.302511:VERBOSE2:decoder_stream_traits.cc(238)] Decoder initialization failed.
[252577:252608:0227/145015.302550:VERBOSE2:decoder_selector.cc(237)] OnDecoderInitializeDone: Dav1dVideoDecoder success=201
[252577:252608:0227/145015.302590:VERBOSE1:batching_media_log.cc(37)] MediaEvent: {"info":"Cannot select Dav1dVideoDecoder for video decoding"}
[252577:252608:0227/145015.302637:VERBOSE2:decoder_selector.cc(198)] GetAndInitializeNextDecoder
[252577:252608:0227/145015.302667:VERBOSE2:decoder_selector.cc(224)] GetAndInitializeNextDecoder: initializing FFmpegVideoDecoder
[252577:252608:0227/145015.302698:VERBOSE1:ffmpeg_video_decoder.cc(234)] Initialize: codec: h264, profile: h264 main, level: not available, alpha_mode: is_opaque, coded size: [3840,2160], visible rect: [0,0,3840,2160], natural size: [3840,2160], has extra data: true, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

[Robin Carlisle]

SOLVED.   

I got gdb out, stepped through some stuff - which led me to the feature flags .. which led to the correct set of features required to get this working.  The Vaapi specific flags I was using appear to be deprecated.  

In case it helps anyone ... here are the args for HW decoding on Intel iGPU  (build  133.0.6943.138).   built with gn args : "is_chrome_branded=false is_official_build=false symbol_level=2 is_debug=true use_sysroot=true proprietary_codecs=true enable_nacl=false ffmpeg_branding=\"Chrome\" chrome_pgo_phase=0 treat_warnings_as_errors=false is_component_build=true",

This also works on the ubuntu 24.04 chromium snap (Chromium 133.0.6943.126 snap)  and probably the official google chrome build - although I haven't tested this

$HOME/build/chrome/133_debug_symbols/src/out/Debug/chrome \
--no-sandbox \
--ozone-platform=wayland \
--use-gl=angle --use-angle=gl \
--enable-features=AcceleratedVideoDecoder,AcceleratedVideoDecodeLinuxGL \
--v=-3 \
--kiosk file:///home/labs/videos/gradtest_v1_h264_main_20MBit.mp4

Best,

Robin

[Ted (Chromium) Meyer]

Hi Robin, sorry for getting back to you so late,

I'm glad you solved the problem on your machine - the Linux HW Decode project has always been a bit of a problem honestly, since it requires totally different configurations based on what versions of drivers are installed (i965 vs iHD), and there is a huge array of configurations across distros, and especially across individual users. The flags we need for a corporate linux install differ from the ones used on my personal laptop, for example, despite being identical hardware.

I do sometimes try to keep wiki pages up to date for this issue, like:

https://wiki.archlinux.org/title/Chromium#Hardware_video_acceleration

Generally, you should always need:

--enable-features=AcceleratedVideoDecoder,AcceleratedVideoDecodeLinuxGL

And then use
--use-gl=angle --use-angle=gl
Depending on what GL or vulkan drivers you have installed.

Ted

[Robin Carlisle]

Hi Ted, thanks for your help and the link - really useful.

In my tests I have noticed that, as you said, some things work and some do not.  Here are my results if they might of interest to anyone out there.  I ventured into h265 too.    Using build 135.0.7037.2  on ubuntu 24.04 (noble) using flags

      chrome \
      --ozone-platform=wayland \
      --use-gl=angle --use-angle=gl \
      --enable-features=AcceleratedVideoDecoder,AcceleratedVideoDecodeLinuxGL,AcceleratedVideoDecodeLinuxZeroCopyGL \

• h264 - 8-bit

  • HW decode on

    • amd (requires flags)

    • intel nuc (requires flags)

  • SW decode on 

    • amd

    • intel nuc

• h264 - 10-bit

  • HW decode on

    • amd 

    • intel nuc

  • SW decode on 

    • amd (works, but still a bit bandy..  not sure if this is better than 8 bit h264)

    • intel nuc (works, but still a bit bandy..  not sure if this is better than 8 bit h264)

• h265 - 8-bit

  • HW decode on

    • amd (requires flags)

    • intel nuc (does not even require the flags that h264 does - very weird)

  • SW decode on 

    • amd

    • intel nuc

• h265 - 10-bit 

  • HW decode on

    • amd (requires flags) 

    • intel nuc (does not even require the flags that h264 does - very weird)

  • SW decode on 

    • amd

    • intel nuc (will not play at all in SW mode)

Some notes

1. on ubuntu 22.04 (jammy) - HW accel does not work as VAAPI buildtime version > runtime version  ... and this silently fails (at all log levels).   I backported vaapi from noble to jammy and this fixed this issue
2. The feature flag AcceleratedVideoDecodeLinuxZeroCopyGL appears to give a good efficiency boost - only evident to me in intel_gpu_top
3. It seems that AcceleratedVideoDecoder is default ON and not required - I will continue to use it anyway in case things change
4. h265 HW decode works very well - but SW decode doesnt.  Fine, but weird maybe.
5. It's great that 10-bit decoding works as it does look a bit better in my gradient tests - but I guess this is not true 10-bit through to the compositor as the quality is not as good as when I set up a true 10-bit video chain from video file through to monitor (outside of chrome)
6. Great this works on AMD gpu (discrete card - W68000) but for 10-bit decode on ubuntu 22.04 (jammy) it does require backported mesa from 24.04 (noble) or it fails with an error and I think it might even crash chrome gpu process.   I need to test this on more modern AMD cards next and integrated AMD gpu too
7. Weird that   h265 HW decode on intel does not even require the flags to work - I will continue to use it anyway in case things change

Again, thanks for your help

Robin