WebGL2 render to texture with MSAA anti-aliasing and transparent background

[Martin]

Hi fellow WebGL developers, 

I'm trying to render a set of triangles with multi-sample-anti-aliasing MSAA enabled in WebGL2. Therefore, I'm setting up rendering pipeline with a multisample renderbuffer to render to a target texture. Anti-aliasing seems to work, however if I try to render the scene to a transparent renderbuffer, the anti-aliasing seems gradually blend to the opaque background color despite it being fully transparent.

In the example image below, a set of green rgb(0,1,0,1) triangles is drawn: first with background clear color set to gl.clearColor(0, 0, 0, 0) - second with clear color set to gl.clearColor(1, 0, 0, 0) - (The resulting texture is blended on a white background to show the results).

How can I render the scene to a transparent texture with anti-aliasing going gradually from rgba(0,0,255,1) to rgba(0,0,0,0)?

//initialization code
gl.frameBufferAA = gl.createFramebuffer();

//render code
let renderBufferAA = gl.createRenderbuffer();
gl.bindRenderbuffer(gl.RENDERBUFFER, renderBufferAA);
gl.renderbufferStorageMultisample(gl.RENDERBUFFER, gl.getParameter(gl.MAX_SAMPLES), gl.RGBA8, texDst.width, texDst.height);

//attach renderBufferAA to frameBufferRenderBuffer
gl.bindFramebuffer(gl.FRAMEBUFFER, gl.frameBufferAA);
gl.framebufferRenderbuffer(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.RENDERBUFFER, renderBufferAA);
gl.clearColor(0, 0, 0, 0);  //<--- transparent color affects anti-aliasing 
gl.colorMask(true, true, true, true);
gl.clear(gl.COLOR_BUFFER_BIT);

twgl.drawBufferInfo(gl, gl.TRIANGLES, bufferInfo);

//blit renderBuffe
gl.bindFramebuffer(gl.READ_FRAMEBUFFER, gl.frameBufferAA);
gl.bindFramebuffer(gl.DRAW_FRAMEBUFFER, gl.frameBuffer1);
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texDst, 0);
gl.blitFramebuffer(
  0, 0, texDst.width, texDst.height,
  0, 0, texDst.width, texDst.height,
  gl.COLOR_BUFFER_BIT, gl.NEAREST
);

gl.deleteRenderbuffer(renderBufferAA);

I've created a jsfiddle to isolate the problem. The fiddle draws an anti-aliased red circle. The pixels created by anti-aliasing are fading to green which is the clear-color of the multisample renderbuffer. The problem seems to be related to alpha=false creation parameter of the webgl2 context.

[Kai Ninomiya]

If I'm understanding correctly, your program:

- Into the MS framebuffer, clears to transparent-green

- Into the MS framebuffer, draws an opaque-red multisampled circle from lines, so it now contains some 100% transparent-green (0,1,0,0) pixels and some 100% opaque-red (1, 0, 0, 1) pixels

- Blits the MS framebuffer into the non-MS framebuffer, so it resolves multiple samples in the same pixel into a single pixel, so it now contains pixels which are (IIRC) averaged between transparent-green and opaque-red (e.g. (0.5, 0.5, 0, 0.5))

- Draws a quad textured with the non-MS render target, with blending, onto the canvas

If that's correct, the third step is where the green would be getting introduced. I don't think there's a way to configure how the "blending" in that resolve step occurs.

In this case you should be able to fix it by just clearing to transparent-black instead. However I'm not sure if that will help with your original use case. Hope this helps anyway!

-Kai

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/35f91292-459c-4143-a395-cbda626d6d43%40googlegroups.com.

[Shrek Shao]

Would the blendFuncSeparate be somewhat useful to workaround the issue if you only want to get the alpha channel blended?

https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/blendFuncSeparate 

[Ken Russell]

Martin: it's not clear to me whether the artifacts are showing up because of blending of the WebGL-rendered canvas with the content behind it, or entirely within the WebGL rendering, but note that correct source-over blending requires the use of premultiplied alpha colors. This can often be the cause of darkening around antialiased edges. See Tom Forsyth's blog entries "Premultiplied Alpha" and "Premultiplied Alpha part 2":

https://tomforsyth1000.github.io/blog.wiki.html

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/92a3af4f-eff7-4fea-87da-4a784d83cf2c%40googlegroups.com.

[Martin]

Hello Kai,

Thanks for your help and analysis - your explanation of the code is correct.

Clearing the MS framebuffer to (0,0,0,0) creates semi-transparent black pixels instead of semi-transparent green pixels which looks a bit less ugly but still not right (see fork of above fiddle: https://jsfiddle.net/j8k03zxs/)

My goal is to render anti-aliased triangles to a transparent texture which can in turn be blended on any background inside the webgl canvas.

kind regards,

Martin

To unsubscribe from this group and stop receiving emails from it, send an email to webgl-d...@googlegroups.com.

[Martin]

Hi Shrek,

I've tried several blend functions that made any sense to me to blend the resulting anti-aliased texture to the white background, no luck yet. At what point could

the blending be introduced to avoid the artifacts?

kind regards,

Martin 

[Martin]

Hi Ken,

thanks for the pointer to the blog post. For the question whether the background elements in the dom could have an effect on the artifacts: if I understand correctly setting alpha=false on the rendering context should prevent any interaction between blending within the canvas and outside. 

Do you say premultiplication required for 

gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);

to work correctly inside the webgl canvas? Should it be executed in the fragment shader before rendering to the MSAA buffer? I've played with several configurations, but couldn't find

a solution to this problem yet.

Thanks for your help,

Martin

To unsubscribe from this group and stop receiving emails from it, send an email to webgl-d...@googlegroups.com.

[Ken Russell]

On Tue, Apr 7, 2020 at 1:16 PM 'Martin' via WebGL Dev List <webgl-d...@googlegroups.com> wrote:

Hi Ken,

thanks for the pointer to the blog post. For the question whether the background elements in the dom could have an effect on the artifacts: if I understand correctly setting alpha=false on the rendering context should prevent any interaction between blending within the canvas and outside. 

Hi Martin,

That's true, but setting alpha:false on the canvas element is strongly discouraged, as it has performance implications on several platforms. See the WebGL Best Practices:

https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#Textures

See "Avoid alpha:false for perf reasons".

Do you say premultiplication required for 

gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);

to work correctly inside the webgl canvas? Should it be executed in the fragment shader before rendering to the MSAA buffer? I've played with several configurations, but couldn't find

a solution to this problem yet.

Yes, the blend functions all work correctly inside the WebGL context itself.

I think what you want to do is:

1) Set up a multisampled renderbuffer and framebuffer.

2) Do your rendering into it.

3) Blit that framebuffer into a single-sampled texture attached to another framebuffer.

Now that single-sampled texture has *separate* alpha. You need to premultiply the alpha channel into the color channel inside your fragment shader when sampling from it, and use glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA), per Tom Forsyth's blog post.

The fragments you write can have alpha=1.0 to avoid the need to use alpha:false. Note that you're not relying on the destination alpha value anywhere.

Does that help?

-Ken

 

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/0a5d7c3a-bc55-43d4-bffe-6abdcd2add55%40googlegroups.com.

[Kai Ninomiya]

> Clearing the MS framebuffer to (0,0,0,0) creates semi-transparent black pixels instead of semi-transparent green pixels which looks a bit less ugly but still not right (see fork of above fiddle: https://jsfiddle.net/j8k03zxs/)

Ah right, I should have realized that. (That's a classic blending/MSAA bug....)

When you do this (specifically with clearing to (0,0,0,0), the resolve step creates an image with premultiplied alpha - that is, red channel is stored as R/A, etc.

Since it's premultiplied, you don't want to multiply it *again* on the final blend. So before that drawArrays:

        gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);
        gl.drawArrays(gl.TRIANGLES, 0, 6);

And to clarify about blendFunc, it can be used to control blending, but it cannot be used to control multisample resolve (during blit from MS to non-MS), which is where the issue is creeping in.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CAMYvS2djQxUWEOcxAao_V%3DrtPdjp_4WOZDMOi%2BXYskN%3DRNxjhg%40mail.gmail.com.

[Ken Russell]

Thanks Kai for clarifying - especially if my analysis and recommendation was incorrect.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CANxMeyAjH%3DdszrFeSojHjOjGEaXB9oddLdHQnyQ%2BH7pu7Pgj8A%40mail.gmail.com.

[Martin]

Hello Kai,

this seems to do the job - https://jsfiddle.net/3amwb7y4/

I'll need to review for myself, but I think this is what I want. Thanks a lot!

I dont't quite understand which resolve step creates the premultiplied values...

kind regards, Martin

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/0a5d7c3a-bc55-43d4-bffe-6abdcd2add55%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.

To unsubscribe from this group and stop receiving emails from it, send an email to webgl-d...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CAMYvS2djQxUWEOcxAao_V%3DrtPdjp_4WOZDMOi%2BXYskN%3DRNxjhg%40mail.gmail.com.

[Martin]

Hi Ken,

 

That's true, but setting alpha:false on the canvas element is strongly discouraged, as it has performance implications on several platforms. See the WebGL Best Practices:

https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices#Textures

I went over your recommendations before and wondered about this single point because with my latest tests I had rather significant performance boost (10fps)

using alpha=false. So I was just wondering. 

 

Thanks again for your help. 

[Kai Ninomiya]

When rendering into a MS framebuffer, one pixel may have 4 color values, e.g.:

(1,0,0,1), (0,0,0,0), (0,0,0,0), (0,0,0,0)

if just one of the samples was covered by the rendered line.

When resolving an MS framebuffer into a non-MS framebuffer, each pixel's 4 sample values get combined into one value by averaging:

(0.25, 0, 0, 0.25)

. In this case, you want that value to represent "full red at 25% opacity." Unpremultiplied, that would be (1, 0, 0, 0.25).

--

You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/87e76050-b504-4dcf-b62c-5760a07e20fc%40googlegroups.com.

[Ken Russell]

On what platform and browser did you observe this? Do you have a test case demonstrating it?

We've seen significant speedups of various content on multiple browsers (Chrome, Firefox, Oculus Browser) using the default alpha:true. The emulation the browser must do internally to set the alpha channel to 1.0 at the end of the frame usually amounts to another full-screen blit, which consumes fill rate, most noticeably on integrated GPUs. Chrome bug http://crbug.com/1045643 tracks this issue.

-Ken

 

Thanks again for your help. 

--

You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/87e76050-b504-4dcf-b62c-5760a07e20fc%40googlegroups.com.

[Jeff Gilbert]

(Firefox currently masks out alpha channel channel writes for alpha:false, but this is expected to have some perf impact too)

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CAMYvS2eEOvBuhGEbqr0XfGfazCKQDNAV7abS146WUHx5G5kUYw%40mail.gmail.com.

[Martin]

Hi Ken

 

On what platform and browser did you observe this? Do you have a test case demonstrating it?

Thats my win10 development machine on chrome using an NVIDIA Geforce GTX 560.

Unfortunately I don't have a benchmark demonstrating this - I'm developing an webgl enabled image editor application. I've used the fps meter in chrome dev tools and noticed

the speedup when switching alpha to false.  I haven't tested recently (that must be almost a year), I'll give it another try with the latest chrome version and post my results.

kind regards,

Martin

[Ken Russell]

This is what Chrome does, too, and it has a significant performance impact on some hardware.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/CAOztSJ1XREE00WJmP_B%3DAu%2BfBp4wCBV2-Cofet1wisbcoHUnGw%40mail.gmail.com.

[Ken Russell]

Please do and let us know what you find. We may also be able to find other performance improvement opportunities.

--

You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/ff1ded84-d329-426f-91e0-d5ce64125c8c%40googlegroups.com.