GL_EXT_YUV_target extension support with Vulkan backend on Android

[July Mk]

Hi,

Seems that GL_EXT_YUV_target  extension is not support with vulkan backend currently.

Any plan to support it?

It's very import, 

because sampling texture mapping from ahardwarebuffer with OES_EGL_image_external can't customize the yuv to rgb conversion matrix.

At the same time, Android CTS tests have not covered the color conversion correctness when sampling with OES_EGL_image_external extension. 

So if we decode full range yuv format compressed video  to ahardwarebuffer(creating with AImageReader, private format),, use OES_EGL_image_external to sample the pixel rgb value from the texture creating from ahardwarebuffer, some device treat the full range yuv data as video range, which cause wrong color.

Beside hardware decoding, sometimes we may use ahardwarebuffer to upload yuv data to texture instead of glTexImage2D because it's more efficient, we have no way to tell the OES_EGL_image_external  sampler whether our data is full range or video range.

With GL_EXT_YUV_target , we can use our own color conversion matrix, so we can get the right rgb color data.

So, if angle doesn't support GL_EXT_YUV_target extension, we have no way to resolve this problem. 

Any plan to support this extension? Android is planing to switch to angle as the OpenGL ES driver, and samsung s22 ultra have already done. It's too bad this thing can't be resolved.

[Geoff Lang]

Hey,

We're working on it but it may be a while until we have a portable implementation of this extension. There is currently no functionality in Vulkan to do this so we're in the process of specifying a Vulkan extension.

Geoff

--
You received this message because you are subscribed to the Google Groups "angleproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angleproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/f9e7acd3-f8d8-47a3-9331-4530584a82b0n%40googlegroups.com.

[July Mk]

Could a new OpenGL ES Extension can be implemented? Just for sampling the yuv plane data, no need to convert to rgba. I think it's too long to wait specifying a Vulkan extension and then implement the ext_yuv_target extension.I know VK_SAMPLER_YCBCR_MODEL_CONVERSION_YCBCR_IDENTITY_KHR can do this with vulkan.

It's urgent because Samsung S22 ultra (GPU: Samsung Xclipse 920, SOC: Exynos) have chosen the angle as the default opengl es driver. Seems they forked the angle, and implement the GL_EXT_YUV_TARGET, But have many bugs, such as two  __samplerExternal2DY2YEXT sampler in shader will cause glCompileShader crash in angle. Two months have gone by, and they have not fixed those bugs.

What we developers can do is disable the use of AHardwarebuffer for decoder and renderer, or refactor our whole project to use vulkan. Many users complain about the pool performance for our app, but we can just explain this is a Samsung and Android bug again and again.

Maybe more and more android phones will use angle as OpenGL ES driver before GL_EXT_YUV_TARGET implemented, this is simply a disaster.

[Geoff Lang]

Could you link me to the bug or post a reproduction case? I can help forward it to them.

Adding a new extension is probably not the solution because it won't replace the driver on their devices without them explicitly updating ANGLE. You *may* be able to import a YUV texture into 2D_ARRAY texture in GL and sample the planes but I've never tried it, it may be against the spec somehow.

Geoff

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/3e7eecf1-8715-4f73-8b71-ac4c596b17a4n%40googlegroups.com.

[Shahbaz Youssefi]

> Just for sampling the yuv plane data, no need to convert to rgba.

As a workaround, how about sampling yuv _with_ conversion to rgb, then convert it back to yuv in the shader itself? That's done with a matrix multiplication, which if I'm not mistaken, is as follows:

- For ITU601: mat3(0.299, -0.1687, 0.5, 0.587, -0.3313, -0.4187, 0.114, 0.5, -0.0813) * rgb_color

- For ITU709: mat3(0.2126,  -0.1146, 0.5, 0.7152, -0.3854, -0.4542, 0.0722, 0.5, -0.0458) * rgb_color

[July Mk]

This is a simple project to reproduce the crash bug:  julee/s22-GL_EXT_YUV_target-crash: A simple project to reproduce the crash problem of Samsung S22 Ultra (SM-S908B/DS) when use GL_EXT_YUV_target (github.com).

shader source is in: gles3jni.cpp.

[July Mk]

This can work, but with OpenGL ES(angle), still no extension to sampling the texture creating from ahardwarebuffer. 

[Geoff Lang]

I forwarded this to my contact on the Samsung team.

Given that Samsung forked ANGLE for their driver on this device, the Samsung bug reporting systems are the best way to report this kind of issue. I'm guessing you already did that though.

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/d89e82bf-b4c8-4fe9-8238-e6dcdd308a48n%40googlegroups.com.