Android Pie and Chrome/80: pixel format problem during h264 video playback

[Michael Goffioul]

I was syncing my pie-x86 build with android-x86, and at the same time update the system webview to Chrome/80, but a new problem popped up when playing h264 videos in the browser. There's no picture and the logs show this:

03-07 17:50:57.128  4997  5103 E NdkImageReader: acquireImageLocked: Output buffer format: 0x4, ImageReader configured format: 0x23
03-07 17:50:57.128  4997  5103 E chromium: [ERROR:image_reader_gl_owner.cc(262)] method fails for some other reasons

My previous build, based on android-x86/android_9.0.0_r48 and using Chrome/77, didn't have that problem. Also youtube doesn't the problem. Any suggestion?

[Michael Goffioul]

Well, that's a bummer. I reverted my build to webview from Chrome/77, but now the webview just plain crashes whenever a video (or maybe audio) is played, even on youtube. Crash happens in thread Chrome_InProcGp, afaik it's the GPU thread, which may indicate the problem lies with mesa. That illustrates very well the proverb: if it ain't broke, don't fix it... I should have stayed with my pie-x86 repo from a couple of months ago.

[Chih-Wei Huang]

Michael Goffioul <michael....@gmail.com> 於 2020年3月8日 週日 上午9:04寫道：

I can play video in youtube from the Chrome browser in 9.0-r1.

How to reproduce the issue?


--
Chih-Wei
Android-x86 project
http://www.android-x86.org

[Michael Goffioul]

My problem is when using a webview, so the closest is probably to use the webview shell app instead of regular chrome browser. But that also means you need an up to date system webview apk. When I use Chrome/80 based system webview, I only get problem with h264 videos, that is, videos decoded by the ffmpeg omx layer. Youtube usually uses vp9, which uses standard aosp codec, so the few ones I've tested play just fine. From what I could gather, Chrome uses libmediandk to produce video frames, but the ffmpeg codec generates frames in RGB565 format, while Chrome expects YV12 (I suspect this is related to this change: https://osdn.net/projects/android-x86/scm/git/frameworks-av/commits/3c55a6971f909814691b8508487b89e41a357b72). The switch to libmediandk by default also seems to be a recent change (https://github.com/chromium/chromium/commit/81d7d8e661419461bf115d144617ed3ab0d824af).

Now the annoying thing is that if I revert to Chrome/77, which used to work fine, any videos (including youtube) crashes my webview-based browser, after a couple of seconds. This is probably a different problem.

Michael.

[Michael Goffioul]

For the record, I reverted mesa to the version I used prior syncing my repo (8b36d87a8f55891b15864aef9022b2533ccedddd) and the crash with Chrome/77 does not occur anymore.

[Michael Goffioul]

One final note: I finally got h264 video to play in the Chrome/80 webview on my device. First I resynced mesa to current pie-x86, this is needed to have proper gralloc1 support. Then I reverted this patch: https://osdn.net/projects/android-x86/scm/git/frameworks-av/commits/3c55a6971f909814691b8508487b89e41a357b72 to allow the ffmpeg codec to produce buffers in YV12 format. Everything seems to work fine: youtube videos, h264 videos in browser, h264 videos in ExoPlayer. Note that I'm using intel's minigbm (gralloc1) combined with drmfb-composer, which may be a reason why the patch that disables YV12 format is not needed anymore.

I wanted to provide a patch for frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1, but given that the code uses Android.bp, creating such conditional is widely unnecessarily painful, so I'll just live with a local dirty patch.

Michael.

 

[youling 257]

"frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1"

the gralloc1 is not the only one choose.

you can test minigbm gralloc0, also can get yv12 work.

minigbm gralloc0 called gralloc.minigbm, gralloc1 called minigbm.intel.

minigbm gralloc0 and hwcomposer.minigbm, https://github.com/maurossi/minigbm/commits/minigbm_amdgpu_v2 https://github.com/maurossi/drm_hwcomposer/commits/minigbm_cros

在 2020年3月17日星期二 UTC+8下午11:56:30，Michael Goffioul写道：

On Tue, Mar 17, 2020 at 8:27 AM Michael Goffioul <michael...@gmail.com> wrote:

On Mon, Mar 16, 2020 at 9:24 PM Michael Goffioul <michael...@gmail.com> wrote:

On Mon, Mar 9, 2020 at 4:01 AM Chih-Wei Huang <cwh...@android-x86.org> wrote:

Michael Goffioul <michael...@gmail.com> 於 2020年3月8日 週日 上午9:04寫道：

>
> On Sat, Mar 7, 2020 at 6:11 PM Michael Goffioul <michael...@gmail.com> wrote:
>>
>> I was syncing my pie-x86 build with android-x86, and at the same time update the system webview to Chrome/80, but a new problem popped up when playing h264 videos in the browser. There's no picture and the logs show this:
>>
>> 03-07 17:50:57.128  4997  5103 E NdkImageReader: acquireImageLocked: Output buffer format: 0x4, ImageReader configured format: 0x23
>> 03-07 17:50:57.128  4997  5103 E chromium: [ERROR:image_reader_gl_owner.cc(262)] method fails for some other reasons
>>
>> My previous build, based on android-x86/android_9.0.0_r48 and using Chrome/77, didn't have that problem. Also youtube doesn't the problem. Any suggestion?
>
> Well, that's a bummer. I reverted my build to webview from Chrome/77, but now the webview just plain crashes whenever a video (or maybe audio) is played, even on youtube. Crash happens in thread Chrome_InProcGp, afaik it's the GPU thread, which may indicate the problem lies with mesa. That illustrates very well the proverb: if it ain't broke, don't fix it... I should have stayed with my pie-x86 repo from a couple of months ago.


I can play video in youtube from the Chrome browser in 9.0-r1. 

How to reproduce the issue?

My problem is when using a webview, so the closest is probably to use the webview shell app instead of regular chrome browser. But that also means you need an up to date system webview apk. When I use Chrome/80 based system webview, I only get problem with h264 videos, that is, videos decoded by the ffmpeg omx layer. Youtube usually uses vp9, which uses standard aosp codec, so the few ones I've tested play just fine. From what I could gather, Chrome uses libmediandk to produce video frames, but the ffmpeg codec generates frames in RGB565 format, while Chrome expects YV12 (I suspect this is related to this change: https://osdn.net/projects/android-x86/scm/git/frameworks-av/commits/3c55a6971f909814691b8508487b89e41a357b72). The switch to libmediandk by default also seems to be a recent change (https://github.com/chromium/chromium/commit/81d7d8e661419461bf115d144617ed3ab0d824af).

Now the annoying thing is that if I revert to Chrome/77, which used to work fine, any videos (including youtube) crashes my webview-based browser, after a couple of seconds. This is probably a different problem.

For the record, I reverted mesa to the version I used prior syncing my repo (8b36d87a8f55891b15864aef9022b2533ccedddd) and the crash with Chrome/77 does not occur anymore.

[Mauro Rossi]

I wanted to provide a patch for frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1, but given that the code uses Android.bp, creating such conditional is widely unnecessarily painful, so I'll just live with a local dirty patch.

Michael.

A possible technical way, it could be to use the following runtime conditional in frameworks/av code, 

but for the moment I would suggest to just evaluate it, because it would be a constrain.

char gralloc_prop[PROPERTY_VALUE_MAX];

if (property_get("ro.hardware.gralloc", gralloc_prop, nullptr) > 4)

...

else

...

gralloc.drm, gralloc.gbm, gralloc.minigbm and gralloc.intel (gralloc1) would all need to be tested

Mauro

[Chih-Wei Huang]

Michael Goffioul <michael....@gmail.com> 於 2020年3月17日 週二 下午11:56寫道：

>
> One final note: I finally got h264 video to play in the Chrome/80 webview on my device. First I resynced mesa to current pie-x86, this is needed to have proper gralloc1 support. Then I reverted this patch: https://osdn.net/projects/android-x86/scm/git/frameworks-av/commits/3c55a6971f909814691b8508487b89e41a357b72 to allow the ffmpeg codec to produce buffers in YV12 format. Everything seems to work fine: youtube videos, h264 videos in browser, h264 videos in ExoPlayer. Note that I'm using intel's minigbm (gralloc1) combined with drmfb-composer, which may be a reason why the patch that disables YV12 format is not needed anymore.

Sounds good. Thank you!

> I wanted to provide a patch for frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1, but given that the code uses Android.bp, creating such conditional is widely unnecessarily painful, so I'll just live with a local dirty patch.

It would be better to use runtime detection
instead of build time config if possible.

[Mauro Rossi]

On Mon, Mar 16, 2020 at 9:24 PM Michael Goffioul <michael...@gmail.com> wrote:

On Mon, Mar 9, 2020 at 4:01 AM Chih-Wei Huang <cwh...@android-x86.org> wrote:

Michael Goffioul <michael...@gmail.com> 於 2020年3月8日 週日 上午9:04寫道：

>
> On Sat, Mar 7, 2020 at 6:11 PM Michael Goffioul <michael...@gmail.com> wrote:
>>
>> I was syncing my pie-x86 build with android-x86, and at the same time update the system webview to Chrome/80, but a new problem popped up when playing h264 videos in the browser. There's no picture and the logs show this:
>>
>> 03-07 17:50:57.128  4997  5103 E NdkImageReader: acquireImageLocked: Output buffer format: 0x4, ImageReader configured format: 0x23
>> 03-07 17:50:57.128  4997  5103 E chromium: [ERROR:image_reader_gl_owner.cc(262)] method fails for some other reasons

Hello Micheal, this problem is affecting gralloc.drm and gralloc.gbm, I still have to try with my latest gralloc.minigbm (branch aosp_pie-x86)

but on the current main graphics stack it is anyway a problem

Could we hope, with gralloc0, to be able to use the YV12 nearest pixel format available in mesa NV12 with some HW assisted conversion,

or HW assisted conversion to RGBX, the former (NV12) seems more efficient

 

One final note: I finally got h264 video to play in the Chrome/80 webview on my device. First I resynced mesa to current pie-x86, this is needed to have proper gralloc1 support. Then I reverted this patch: https://osdn.net/projects/android-x86/scm/git/frameworks-av/commits/3c55a6971f909814691b8508487b89e41a357b72 to allow the ffmpeg codec to produce buffers in YV12 format. Everything seems to work fine: youtube videos, h264 videos in browser, h264 videos in ExoPlayer. Note that I'm using intel's minigbm (gralloc1) combined with drmfb-composer, which may be a reason why the patch that disables YV12 format is not needed anymore.

I wanted to provide a patch for frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1, but given that the code uses Android.bp, creating such conditional is widely unnecessarily painful, so I'll just live with a local dirty patch.

Michael.

In this commit there is an example of how to open the gralloc module and check the gralloc_version and the common.name

Similar implementation could be applied in frameworks/av and it would be a clean way. 

 

[Mauro Rossi]

I wanted to provide a patch for frameworks/av/ that allows YV12 conditionally based on BOARD_USES_GRALLOC1, but given that the code uses Android.bp, creating such conditional is widely unnecessarily painful, so I'll just live with a local dirty patch.

Michael.

In this commit there is an example of how to open the gralloc module and check the gralloc_version and the common.name

Similar implementation could be applied in frameworks/av and it would be a clean way. 

This commit:

http://git.osdn.net/view?p=android-x86/external-mesa.git;a=commitdiff;h=1cc0e591b2abfb3e126b1d0d45a95b44c7e22ce9;hp=97528a021a28fab9d69f74b4c98cb8687df64786 

[youling 257]

I don't know how to test HW yuv, test some app use sw YUV on cm14.1-x86.

mxplayer setting sw YUV, play video black screen with gralloc.drm and gralloc.gbm, not blackscreen with gralloc.minigbm.

taobao live app, play video blackscreen with gralloc.drm, not blackscreen with gralloc.minigbm.

在 2020年5月22日星期五 UTC+8下午2:38:29，Mauro Rossi写道：

[youling 257]

revert "Don't use YV12 color format for video decoding" on cm14.1-x86, mxplayer HW play video blackscreen with gralloc.drm, HW play video not blackscreen with gralloc.minigbm.

gralloc.minigbm is chromium minigbm gralloc0.

在 2020年5月22日星期五 UTC+8下午3:47:23，youling 257写道：

I don't know how to test HW yuv, test some app use sw YUV on cm14.1-x86.

mxplayer setting sw YUV, play video black screen with gralloc.drm and gralloc.gbm, not blackscreen with gralloc.minigbm.

taobao live app, sw play video blackscreen with gralloc.drm, not blackscreen with gralloc.minigbm.

[youling 257]

why i revert "Don't use YV12 color format for video decoding", because

Kernel: Linux version 5.6.0-rc2-android-x86_64+ (root@localhost) (gcc version 9.2.1 20200203 (Ubuntu 9.2.1-28ubuntu1)) #1 SMP PREEMPT Tue Feb 18 21:39:43 CST 2020

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

LineageOS Version: '14.1.0-android_x86'

Build fingerprint: 'Android-x86/cm_android_x86/android_x86:7.1.2/NJH47F/939dd5481a:userdebug/test-keys'

Revision: '0'

ABI: 'x86'

pid: 29667, tid: 9821, name: MediaCodec_loop >>> com.chrome.canary:privileged_process0 <<<

signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xcae04000

    eax 00000010 ebx f773eaa0 ecx cad93530 edx cadcbc00

    esi 00038400 edi e7544340

    xcs 00000023 xds 0000002b xes 0000002b xfs 0000006b xss 0000002b

    eip f76ba487 ebp cb0011f8 esp cb0011d0 flags 00210202

backtrace:

    #00 pc 00179487 /system/lib/libstagefright.so (I422ToARGBRow_SSSE3+55)

    #01 pc 0015cd47 /system/lib/libstagefright.so (I420ToRGB565Dither+519)

    #02 pc 001588f5 /system/lib/libstagefright.so (_ZN7android14ColorConverter28convertYUV420PlanarUseLibYUVERKNS0_12BitmapParamsES3_+293)

[youling 257]

I see the coming soon pie-x86 r3 include build minigbm and minigbm_intel, they support yv12 pixel format.

the patch for framework/av allow yv12 use property_get is possible?

在 2020年3月18日星期三 UTC+8上午2:23:43，Mauro Rossi写道：

[Chih-Wei Huang]

youling 257 <youli...@gmail.com> 於 2020年6月5日 週五 下午12:54寫道：

>
> I see the coming soon pie-x86 r3 include build minigbm and minigbm_intel, they support yv12 pixel format.
> the patch for framework/av allow yv12 use property_get is possible?

So far minigbm & minigbm_intel don't work for me.
Maybe I miss something...

[Mauro Rossi]

In order to have minigbm and mingbm_intel in working state is to compile mesa without BOARD_USES_DRM_GRALLOC,

but we cannot do that with a single libGLES_mesa.so

That's why I had to split libGLES_mesa in two modules libGLES_mesa_drm_gralloc and libGLES_mesa_prime_fd

ro.hardware.egl property can be used to induce the frameworks/native EGL Loader to load the correct libGLES_mesa_* module

Regarding the YV12 issue with Chrome 80 and later the fact of detecting that minigbm or minigbm_intel are in use,

may enable frameworks/av to play videos, but it will not solve the black video problem for gralloc.drm and gralloc.gbm

We need a different approach either based on YV12 to NV12 conversion or on a yuv locking mechanism in the default gralloc.drm like in minigbm, which is implemented also in gralloc0 version, just selecting RGBX_8888 format does not work anymore with Chrome 80 and later change.

Mauro

[Chih-Wei Huang]

Chih-Wei Huang <cwh...@android-x86.org> 於 2020年6月6日 週六 上午10:24寫道：

>
> youling 257 <youli...@gmail.com> 於 2020年6月5日 週五 下午12:54寫道：
> >
> > I see the coming soon pie-x86 r3 include build minigbm and minigbm_intel, they support yv12 pixel format.
> > the patch for framework/av allow yv12 use property_get is possible?
>
> So far minigbm & minigbm_intel don't work for me.
> Maybe I miss something...

I've fixed it. Now all grallocs can work seemingly.
Will check the yv12 issue later.

For those who are interested in the new graphical stack,
you may sync the latest pie-x86 branch to build and test it.
Welcome to provide feedback.

[Chih-Wei Huang]

Chih-Wei Huang <cwh...@android-x86.org> 於 2020年6月8日 週一 上午8:32寫道：

> Chih-Wei Huang <cwh...@android-x86.org> 於 2020年6月6日 週六 上午10:24寫道：
> > youling 257 <youli...@gmail.com> 於 2020年6月5日 週五 下午12:54寫道：
> > >
> > > I see the coming soon pie-x86 r3 include build minigbm and minigbm_intel, they support yv12 pixel format.
> > > the patch for framework/av allow yv12 use property_get is possible?
> >
> > So far minigbm & minigbm_intel don't work for me.
> > Maybe I miss something...
>
> I've fixed it. Now all grallocs can work seemingly.

Credit to Mauro who finished most of the hard work.
I just completed the last piece of the jigsaw puzzle.

[Huy Minh Bui]

Can I use these gralloc and minigbm for nougat ? Since in nougat we are having problem with Chrome 80+ video playback too

Especially when compile with newer Mesa.

Vào 07:51:53 UTC+7 Thứ Hai, ngày 08 tháng 6 năm 2020, Chih-Wei Huang đã viết:

[youling 257]

https://groups.google.com/d/msg/android-x86/QfIMLBYNJxI/3M8LvlheAwAJ
BOARD_USES_DRM_GRALLOC := true

BOARD_USES_MINIGBM := true

BOARD_USES_MINIGBM_INTEL := false

BOARD_USES_GRALLOC1 := false

BOARD_USES_IA_HWCOMPOSER := false

TARGET_USES_HWC2 := false

BOARD_USES_DRM_HWCOMPOSER ?= true

hwcomposer.minigbm(hwc1) and gralloc.minigbm(gralloc0) is best choose for androidx86 7.

在 2020年6月8日星期一 UTC+8下午12:31:11，Huy Minh Bui写道：

[Huy Minh Bui]

Thanks :D

Vào 11:36:05 UTC+7 Thứ Hai, ngày 08 tháng 6 năm 2020, youling 257 đã viết:

[Huy Minh Bui]

by the way I read that post and saw that you can use Vulkan now ?

[fguy]

I compiled and tested the current pie-x86 branch with kernel 4.19.122 and mesa 20.1.0. Unfortunately, none of the proposed configurations led to the video showing in the current version of chrome. Option c drmfb did not load on my core m3 6y30. Intel celadon 9 with kernel 4.19.106 and mesa 19.3.3 displays video in chrome without any problems.

[youling 257]

The problem is android 7 only support vulkan 1.0

mesa20, radv: enable Vulkan 1.2, radv: limit the Vulkan version to 1.1 for Android, anv: Enable Vulkan 1.2 support

在 2020年6月8日星期一 UTC+8下午1:17:14，Huy Minh Bui写道：

[Huy Minh Bui]

So can we limit Mesa to use 1.0 only ?

Vào 01:47:05 UTC+7 Thứ Tư, ngày 10 tháng 6 năm 2020, youling 257 đã viết:

[Mauro Rossi]

On Tuesday, June 9, 2020 at 8:47:05 PM UTC+2, youling 257 wrote:

The problem is android 7 only support vulkan 1.0

That's another story, if you actually encountered issues,

you may try to HACK and artificially lower the Vulkan API version in 

https://cgit.freedesktop.org/mesa/mesa/tree/src/amd/vulkan/radv_extensions.py

line 45:

API_VERSIONS = [

    ApiVersion('1.0.68',  True),

-   ApiVersion('1.1.107', True),

+   ApiVersion('1.1.107', '!ANDROID'),

    ApiVersion('1.2.131', '!ANDROID'),

]

Similar attempt in anv https://cgit.freedesktop.org/mesa/mesa/tree/src/intel/vulkan/anv_extensions.py

Warning: I've tried this

Mauro

[Mauro Rossi]

Correction

On Tuesday, June 9, 2020 at 9:16:07 PM UTC+2, Mauro Rossi wrote:

On Tuesday, June 9, 2020 at 8:47:05 PM UTC+2, youling 257 wrote:

The problem is android 7 only support vulkan 1.0

That's another story, if you actually encountered issues,

you may try to HACK and artificially lower the Vulkan API version in 

https://cgit.freedesktop.org/mesa/mesa/tree/src/amd/vulkan/radv_extensions.py

line 45:

API_VERSIONS = [

    ApiVersion('1.0.68',  True),

-   ApiVersion('1.1.107', True),

+   ApiVersion('1.1.107', '!ANDROID'),

    ApiVersion('1.2.131', '!ANDROID'),

]

Similar attempt in anv https://cgit.freedesktop.org/mesa/mesa/tree/src/intel/vulkan/anv_extensions.py

Warning: I've NEVER tried this

[Huy Minh Bui]

Thanks for the information, I'll try it tomorrow.

Vào 02:16:58 UTC+7 Thứ Tư, ngày 10 tháng 6 năm 2020, Mauro Rossi đã viết:

[youling 257]

mesa 19.3 radv/android: Add android hardware buffer support, but android hardware buffer added in api level 26, 

https://github.com/mesa3d/mesa/commits/master/src/amd/vulkan/radv_android.c

https://developer.android.com/reference/android/hardware/HardwareBuffer

if want to amdgpu vulkan work on nougat-x86, please use mesa 19.2.8 or 18.3.6

在 2020年6月10日星期三 UTC+8上午2:47:05，youling 257写道：

[Huy Minh Bui]

So we can't revert or do anything for future build ?

[Mauro Rossi]

On Monday, July 20, 2020 at 4:27:08 AM UTC+2, Huy Minh Bui wrote:

So we can't revert or do anything for future build ?

I don't think there's anything to revert, because RADV_SUPPORT_ANDROID_HARDWARE_BUFFER it's not hardcoded,

it is enabled only for ANDROID_API_LEVEL >= 26,

please have a look at line 92

https://cgit.freedesktop.org/mesa/mesa/tree/src/amd/vulkan/radv_private.h 

While Android 7.x surely knows nothing about Vulkan 1.1 and 1.2 specs,

that's why I proposed the change, if you pencounter issues, 

but if you think your apps support Vulkan 1.1 or 1.2 you could even skip it.

..., if you actually encountered issues,

[youling 257]

please you submit this patch and push request to mesa upstream.

https://github.com/youling257/mesa/commit/8ddc9d04e8f24669835ac8dc1e8841041ce1662e

在 2020年7月21日星期二 UTC+8上午5:54:36，Mauro Rossi写道：

[Mauro Rossi]

On Tuesday, July 21, 2020 at 5:41:50 AM UTC+2, youling 257 wrote:

please you submit this patch and push request to mesa upstream.

https://github.com/youling257/mesa/commit/8ddc9d04e8f24669835ac8dc1e8841041ce1662e

Hi youling,

could you please provide commit message including the building error you get with Android 7

and your Signed-off-by tag?

It will be easier to ask the Merge Request acceptance

Thanks

Mauro

[Mauro Rossi]

On Tuesday, July 21, 2020 at 11:24:39 PM UTC+2, Mauro Rossi wrote:

On Tuesday, July 21, 2020 at 5:41:50 AM UTC+2, youling 257 wrote:

please you submit this patch and push request to mesa upstream.

https://github.com/youling257/mesa/commit/8ddc9d04e8f24669835ac8dc1e8841041ce1662e

Hi youling,

could you please provide commit message including the building error you get with Android 7

and your Signed-off-by tag?

And also the following tag, which will highlight which commit introduced the issue with Android 7

Fixes: commit hash ("commit title") 

[youli...@gmail.com]

is there a way to solve it? please revert "Don't use YV12 color format for video decoding"

[Fabio de Oliveira Alves]

Android x86 is still under development so vlc is not working very well I use Web Video Caster Receive on Android x86 and Web Video Caster on Mobile to stream videos. 

Em ter, 6 de jul de 2021 12:11, youli...@gmail.com <youli...@gmail.com> escreveu:

is there a way to solve it? please revert "Don't use YV12 color format for video decoding"

--
You received this message because you are subscribed to the Google Groups "Android-x86" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-x86...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-x86/2da3017f-40ba-4697-a890-a6f8e9b32ea2n%40googlegroups.com.