Chromium on Mac OS X + Transparency

[Trevor Linton]

Hi All,

I'm having a bizarre problem.  I've created a new UI (not using chrome/ui) in a browser window on Mac OS X Mountain Lion (10.8.2).  I'm trying to render a webpage with transparent background, I've done this by:

1) Implemented a delegate between RenderViewHost and RenderView to execute WebView::setIsTransparent to true.

2) Created an NSWindow that IsOpaque is set to false, and drawRect paints a full clearColor to the background

3) All backgrounds of NSView's drawRect paints a full clearColor to the background

It worked, well, sort of, the results are in the video:

https://vimeo.com/55688094

The video shows the transparent window working perfectly, until you interact with it.  It appears as if the graphics buffer isn't cleared, but rather just drawn on top of the old buffer.  Does anyone have any recommendations on how to fix this?

Here's some more information:

    prefs->frame_flattening_enabled = false;

    prefs->accelerated_2d_canvas_enabled = true;

    prefs->accelerated_animation_enabled = true;

    prefs->accelerated_compositing_enabled = true;

    prefs->gl_multisampling_enabled = true;

    //prefs->force_compositing_mode = true;

    prefs->experimental_webgl_enabled = true;

If I set force_compositing_mode to true the entire frame is painted black.

In addition, the window has the style:

  NSUInteger style_mask = NSTitledWindowMask | NSClosableWindowMask |

            NSMiniaturizableWindowMask | NSResizableWindowMask |

            NSTexturedBackgroundWindowMask;

The WebView is originally created without a transparency, then its added later.

[Avi Drissman]

There's two completely different drawing modes in Chromium, the original path and the accelerated one. In the original one (I think it's what you're getting when you comment out the "force compositing mode" line) we're blitting rectangles that come from WebKit. I'd imagine that should work but it's never anything we supported and I don't know why it isn't working.

In the compositing path, the entire render view is covered by a CompositingIOSurfaceMac which uses an NSOpenGLContext to do some magic. I have no clue how it works or even if it can do transparency at all.

Not sure what advice to give.

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Kenneth Russell]

The compositing path draws the web page to the screen using OpenGL.
While it is possible to make an on-screen OpenGL context translucent
or transparent on Mac OS using the NSOpenGLCPSurfaceOpacity property,
mouse events will always be received by the OpenGL content as though
it were opaque. This means it is not possible to make non-rectangular
windows that contain OpenGL content without reading back the
framebuffer and drawing it to the screen using Core Graphics, which
will be very slow. However, depending on your use case, this might not
be a limitation.

-Ken

[Trevor Linton]

This isn't a limitation, i'm fine with hit tests failing, whats more a concern is the rendering isn't working (as you can see in the video).  Is there any area to start investigating?  I've tried tracing down what the setIsTransparent on WebView is doing but i've gone pretty far down the rabbit hole and i'm continually getting lost.. :/  Once again it seems as if the next frame is writing on top of the previous frame without clearing the dirty part. Thus it results in effectively a black screen pretty quickly when you begin to interact with it.

[Avi Drissman]

I'm looking at the RenderWidgetHostViewCocoa's -drawRect:; there are so many ways that drawing can happen. Does the backing store (if you're using it) even support transparency?

That has gone through two generations of work since I last dealt with it, and I have no suggestions.

Avi

[Trevor Linton]

Well, part of the story is coming together. 

Macbook Pro's with Retina displays do not render chromium correctly when using a non-compositing mode. Problem occurs in backing_store_mac.mm, in the PaintToBackStore.  When it attempts to scale a blit to correct for the higher DPI (CGImageCreate func) it doesn't interpolate.  Passing true into the method fixes the issue.

The second part seems to be that the wrong portions of the screen are being updated on higher DPI screens.  I'd imagine because its using pixels as dimensions.  More to come.

-t

[Avi Drissman]

+nico

While it's unlikely we'll change code to enable transparency unless it's otherwise harmless, if you're finding actual issues with our existing code, please fix them. Adding Nico; he worked on HiDPI.

Avi

[Trevor Linton]

I have a fix for both MacOSX blit/quartz2d non-compositing and compositing/opengl paths.  I'm not sure if this is of any interest to chromium.

[Avi Drissman]

If the patches fix bugs that might show up in normal (i.e. non-transparent) use, we'd like to see them. If all they do is enable transparency, probably not. Nico? Kenneth?

[Trevor Linton]

I have a commit with the diff for chromium here:

https://github.com/trevorlinton/chromium/commit/794af4b3f3b66945fd2e99272157b069c7f61100

This just adds support for transparency in Mac OS X (Leopard, Snow Leopard, Mountain Lion and HiDPI screens) while in accelerated mode.  Non-accelerated mode still has some blit issues i'm trying to figure out but they're fairly peculiar and seem to be isolated to HiDPI screens. I should note this does not solve hit test problems, those have to be taken care of by the hosting window delegate.

I'm also working on supporting Windows transparency, however that's more complicated due to Direct3D/Layered Windows… 

- Trevor

On Dec 31, 2012, at 11:16 AM, Kenneth Russell <k...@google.com> wrote:

Trevor, I'd certainly like to see your patches to understand whether
they are of general interest. Could you upload them?

Thanks,

-Ken

[Kenneth Russell]

Trevor, thanks for posting your patch. It's good that it was only a
few lines to enable your use case.

I'm not inclined to incorporate these changes for two reasons. First,
as you point out, there are hit testing issues in general with OpenGL
applications on OS X. Apps using Cocoa and Core Graphics won't receive
mouse events when a transparent portion of the top-level window is
hit, but apps using OpenGL do receive those mouse events. This is a
longstanding problem with the window server on OS X and I don't
foresee the behavior changing. It's potentially a security issue in
the context of the web, because untrusted content could make the
window completely transparent, but still receive mouse clicks and
keyboard events. The window server prohibits this for pure Cocoa
applications -- I think the rule is that the app doesn't receive a
mouse event if the pixel it drew underneath the mouse pointer is less
than 10% opaque.

Second, the changes to compositing_iosurface_mac.mm aren't gated on
whether the view is transparent. In ordinary Chromium we would not
want to pay any potential performance penalty from making the OpenGL
context transparent.

If maintaining these diffs in your own tree is a significant burden,
let me know and maybe we can figure out a way to upstream them.

-Ken

[Trevor Linton]

Thanks Kenneth,

I agree that the changes aren't helpful at all for Chromium/Chrome, but more to those who want to use the rendering/v8 platform for other purposes. On top of the additional performance penalty it costs its probably not worth incorporating.  I can mark the build once linux/aura/windows support is added and other developers can refer to it if they wish to integrate the code (its looking like it won't be too hard on the accelerated paths). 

There is one recommendation, i've spoken to an OpenGL expert who believes the texture generated code moving from .rgb to rgba actually should improve compositing performance.   Doing a vec4(texture.rgb,1), or effectively wiping out the alpha layer on purpose, actually costs more than a simple copy of the texture. I'm not an expert in opengl but its something to consider that might help performance.  However the other line for setting the opacity of the backing buffer to blend with the texture behind it should not be added as it will hurt performance considerably.

[Kenneth Russell]

On Wed, Jan 2, 2013 at 6:18 PM, Trevor Linton <trevor...@gmail.com> wrote:
> Thanks Kenneth,
>
> I agree that the changes aren't helpful at all for Chromium/Chrome, but more
> to those who want to use the rendering/v8 platform for other purposes. On
> top of the additional performance penalty it costs its probably not worth
> incorporating. I can mark the build once linux/aura/windows support is
> added and other developers can refer to it if they wish to integrate the
> code (its looking like it won't be too hard on the accelerated paths).
>
> There is one recommendation, i've spoken to an OpenGL expert who believes
> the texture generated code moving from .rgb to rgba actually should improve
> compositing performance. Doing a vec4(texture.rgb,1), or effectively
> wiping out the alpha layer on purpose, actually costs more than a simple
> copy of the texture. I'm not an expert in opengl but its something to
> consider that might help performance.

Thanks for the recommendation. From recollection, I am pretty sure the
setting of the alpha channel to 1.0 there is needed in order to make
the rendering correct in all cases, but I'll keep the possibility of
an optimization in mind.

-Ken

[se...@adifferentengine.com]

Hi Trevor,

I came across this when looking for the same exact thing for linux.  I was wondering if your work had proceeded to other platforms yet?  BTW, did you ever manage to get a sense of the performance implications? 

Thanks,

[Trevor Linton]

Yes, i've made some progress on it.

Using the rendering benchmark tests I haven't seen performance penalties, infact in narrower circumstances there's been performance improvements from not having to knock out the alpha layer for root views.  However as a previous developer had mentioned it does introduce potential security problems.  As a breakdown here's the support matrix:

MacOSX  - Supports both accelerated compositing and non-compositing modes (although threaded compositing still has some gotchas)

Windows - Supports accelerated compositing and non-compositing modes but has one major draw back, if the user does not have a DWM compositing (or Aero) enabled it fails if the user tries to view anything requiring accelerating compositing such as css animations or webgl. Currently i'm trying to discover how to implement this with WS_EX_LAYERED but both Windows and Chromium's rendering systems are extremely complicated and its taking time to fix all of the issues involved. 

I should say mouse hit testing is fine in all circumstances, I found a novel solution to prevent windows from "Trapping" mouse clicks if they're transparent (e.g., if the mouse is over an area less than 10% transparent all mouse events are ignored by the process through the window manager on both Win/Mac).  This hopefully negates certain security concerns of a web page creating a transparent view and trapping/logging mouse or keyboard clicks. 

You can find most of the patches at:

http://github.com/trevorlinton/chromium 

Albeit, i'm still at R29 and they're not necessarily very clear.. In addition I have a custom shell to demo it at: 

http://github.com/trevorlinton/tint