UNPACK_PREMULTIPLY_ALPHA_WEBGL

[Andrew Varga]

Hi,

I'm trying to understand what this flag means and the related best practices.

Seems that this flag specifies if the texture data that is about to be uploaded to the GPU via texImage2D is in premultiplied form or not.

How does that work? If the image I upload is un-premultiplied and I specify false for the flag, than the result of texture2D in glsl will be those un-premultiplied values (?), but if I specify it incorrectly as true, WebGL will un-premultiply the result of texture2D because it thinks the image is premultiplied?

Also as far as I know there is no way to know for sure if an image is premultiplied or not: it can only be known if it's not premultiplied in case there is a pixel in it which has a color value that is larger than the alpha value would allow it.
With that in mind, what is a good strategy for allowing users to upload arbitrary images? Is it to assume most of them are premultiplied and use the [ONE, ONE_MINUS_SRC_ALPHA] blend mode which means if the image is not premultiplied it will be rendered incorrectly, especially if non NEAREST texture filters are used?

Thank you!

[Ken Russell]

Hi Andrew,

This unpack parameter is mainly for use when uploading HTMLImageElements containing PNG images to WebGL textures. By default, when such images are decoded by the browser, the alpha channel is multiplied into the color channels, because that makes the browser's rendering and interaction with 2D canvas contexts more efficient. Setting UNPACK_PREMULTIPLY_ALPHA_WEBGL to false skips this multiplication step during decoding. In practice, this implies a synchronous re-decode of the image, at least in Chromium.

If you want to preserve the separate alpha channel, the recommended code path nowadays (Mozilla's engineers will dispute this) is to fetch your image into a Blob, and create an ImageBitmap from that blob, specifying "premultiplyAlpha" as "none" in the ImageBitmapOptions dictionary. See this article:

https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#imagebitmap_creation

This will cause the image data to be decoded exactly once, asynchronously, in the desired format. Then upload the ImageBitmap to a WebGL texture.

As far as I know, images containing alpha, such as transparent PNGs, aren't saved to disk in premultiplied format. They're saved with a separate alpha channel, so to do correct blending for arbitrary images you load, you actually do want to premultiply the alpha channel during WebGL texture upload, and use [ONE, ONE_MINUS_SRC_ALPHA] blend factors.

-Ken

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/f8e92154-caa6-47d8-976d-013aa56fe214n%40googlegroups.com.

[Ivan Popelyshev]

If the data that calculated IN SHADER is not premultiplied, blend should be 

Its possible to detect not-premultipied data by "R>A" or "G>A" or "B>A" condition, but its better to just give user options. 

In PixiJS we have three options, not two: 
===
https://pixijs.download/dev/docs/PIXI.html#ALPHA_MODES

NO_PREMULTIPLIED_ALPHA aka  NPM=0
Source is not premultiplied, leave it like that. Option for compressed and data textures that are created from typed arrays. 

REMULTIPLIED_ALPHA aka  PMA=1

Source is already premultiplied Example: spine atlases with _pma suffix.

PREMULTIPLY_ON_UPLOAD aka  UNPACK=2

Source is not premultiplied, premultiply on upload. Default option, used for all loaded images.

===

UNPACK is called only for 2. 
For both 1 and 2 we use regular pma blend [ONE, ONE_MINUS_SRC_ALPHA]
For 0 blend is  [SRC_ALPHA, ONE_MINUS_SRC_ALPHA, ONE, ONE_MINUS_SRC_ALPHA] , two extra alues for alpha itself <-- IMPORTANT!

Also sometimes we pass that thing to shader uniform so shader can change data received from texture and work with PMA if it needs it.
For example, this line was ported to ALL spine runtimes, not just pixi/webgl : https://github.com/pixijs/pixi-heaven/blob/fa5c059637060779f2b4915eadc36c3ec687dc91/src/twotint/sprites/HeavenBatchRenderer.ts#L48 . " vDark.a" is 0 for NPM and 1 for PMA, the rest of params are two tints - dark and light. Thus, shader produces PMA result for PMA texture and NPM for NPM.

Ivan.

среда, 7 декабря 2022 г. в 08:00:04 UTC+3, Kenneth Russell:

[Andrew Varga]

Thank you Ken and Ivan, this does help a lot.

Ken, you mentioned that by default those images get premultiplied by the browsers during decode but setting the flag to false skips that. Just want to verify "by default" here means what browsers usually do and this doesn't refer to the default value of the UNPACK_PREMULTIPLY_ALPHA_WEBGL flag, which is false.

So my understanding is that even though PNG images themselves are not premultiplied on disk, browsers internally typically premultiply them anyway, so it's better to set this flag to true to avoid a re-decode and also for correct blending.

I don't really need to preserve the separate alpha channel, the reason I looked into this was that some PNGs were rendering with a black border around their edges when scaled up and setting the magFilter to LINEAR (instead of NEAREST), so I think your suggestion works well for me.

Ivan, I think the blend was missing after the words "blend should be"?
Thanks for the tips, it sounds pretty flexible to have all these settings to cover all possibilities and that blend mode for case "0" is good to know!

Thank you,
Andrew

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/47c9d429-af14-4694-b3a5-2a000e561f9cn%40googlegroups.com.

[Ken Russell]

On Wed, Dec 7, 2022 at 2:10 PM Andrew Varga <griz...@gmail.com> wrote:

Thank you Ken and Ivan, this does help a lot.

Ken, you mentioned that by default those images get premultiplied by the browsers during decode but setting the flag to false skips that. Just want to verify "by default" here means what browsers usually do and this doesn't refer to the default value of the UNPACK_PREMULTIPLY_ALPHA_WEBGL flag, which is false.

So my understanding is that even though PNG images themselves are not premultiplied on disk, browsers internally typically premultiply them anyway, so it's better to set this flag to true to avoid a re-decode and also for correct blending.

I hadn't double-checked the default value of UNPACK_PREMULTIPLY_ALPHA_WEBGL before sending my earlier email - so yes, you're correct. In your scenario it's better to set the flag to true. People do sometimes use PNGs to store non-image data where it would be incorrect to premultiply the alpha channel.

-Ken

 

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CABNv9Xx4nU60KLkg9YMqFvn%2B%2Br5rAt3t89znjXaCBvooydg2fA%40mail.gmail.com.

[Andrew Varga]

Thank you, that makes sense!

After a bit of testing I found something I'm still not sure about: rendering a texture (with a PNG image source) zoomed in and using TEXTURE_MAG_FILTER = LINEAR, I would expect the following 2 options to be equivalent:

Option A
Texture with: UNPACK_PREMULTIPLY_ALPHA_WEBGL = true

Not premultiplying in the shader:

gl_FragColor = texture2D(u_texture, v_uv);

Option B

Texture with: UNPACK_PREMULTIPLY_ALPHA_WEBGL = false

Premultiplying in the shader:
gl_FragColor = texture2D(u_texture, v_uv);
gl_FragColor.rgb *= gl_FragColor.a;

Blend mode for both is [ONE, ONE_MINUS_SRC_ALPHA]

I would expect the same result as in both cases the color gets premultiplied, either manually or via UNPACK but for Option B, I'm getting the dark silhouette around the semi-transparent texels of the texture. I wonder why that is?

My only guess for what's different would be that premultiplication is lossy so doing that and reversing it can lead to some loss but it doesn't sound like that would be the cause?

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CAMYvS2c%2BtJUhve9kmaNTyvgcezzTF_tan5%2BsefCjtbsYvZ7qZw%40mail.gmail.com.

[Ivan Popelyshev]

Because, as you mentioned before, LINEAR over not-premultiplied data gives different result because if there's neighbouring pixel with A=0 , whatever trash data is in that pixel affects interpolation, resulting in dark silhouette. That's why people use "extrude" feature of texture packer ( i dont remember how is it named in photoshop) , it takes colors from pixels A>0 and applies to their A=0 neibs.

четверг, 8 декабря 2022 г. в 02:29:26 UTC+3, griz...@gmail.com:

[Ivan Popelyshev]

Sorry for double-post. Here's an article: http://www.adriancourreges.com/blog/2017/05/09/beware-of-transparent-pixels/

четверг, 8 декабря 2022 г. в 10:08:43 UTC+3, Ivan Popelyshev:

[Andrew Varga]

I guess what I didn't take into consideration is that the fragment shader runs only once for a fragment but there are still multiple texture samples for that fragment when a texture has LINEAR filtering, so if the texture has non premultiplied colors at that point the interpolation will still be wrong.

Thanks for the link too!

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/327b7a8a-b9e0-4d40-8125-05feafb0bcdcn%40googlegroups.com.