As of late Wednesday the 10th, plugins were at the following state: on
GTK2/GNU/Linux systems, we can load a single instance of a single plugin
in windowed mode out of process. We haven't yet implemented the
NPRuntime (scripting wrapper for plugins), but theoretically
non-scripted plugins should be able to load and draw to the screen.
Unfortunately we haven't observed this yet, because both flash and our
"test" plugin want the NPRuntime :S. But most importantly, our IPC
architecture is starting to come together, and adding NPRuntime support
should not be a massive burden. We are ready to start implementing the
IPDL ("I piddle"), inter-*-communication protocol definition language,
compiler described somewhat here
(https://wiki.mozilla.org/IPC_Protocols), although this page is
outdated, inaccurate, and incomplete. This language should hopefully
make writing IPC code more productive and less error prone.
We have a complete short-term story for plugin graphics on X11.
Windowed plugins are very easy to support, as X windows are identified
with cross-process XID's; we need only to create the window in the Gecko
process and ship the ID across to the plugin. Windowless plugins are
more difficult to support, but we can reuse som work done for Hildon;
essentially, we create a window for the windowless plugins and create a
surface within that window for the plugin to draw to, but use Xdamage
and Xcomposite to composite the Gecko window with the plugin's window.
(Graphics folks, if this is a bad summary of this technique, please
correct me.) Ben Turner will likely be working on plugin graphics for
Windows, but we're not yet sure of our approach to plugin graphics for OS X.
In other Electrolysis news, our graphics team has a solid plan of attack
for multi-process Gecko rendering, both for the short and long terms.
They will probably document this plan publicly, soon. Ben Turner is
also working on creating our "content" process, in which web pages will
be rendered (as opposed to the "chrome" process in which the Firefox UI
will reside). We expect to have an initial prototype of this up and
Shouldn't windowless plugins be easier to support then windowed ones?
Is there any reason you can't just create a surface in a region of
memory shared between the plugin process and the Gecko process? It's
not clear to me what problems are solved by using Xdamage and
This shared memory approach should also work fine on OS X.
On 15-Jun-09, at 3:17 PM, Chris Jones wrote:
> It's just disgustingly easy to ship XID's from the content to plugin
> process. Also, all of the Xdamage/Xcomposite code is written and
> tested. Again, this is the short-term (i.e. something shippable as
> soon as possible) and not long-term (i.e. something we're entirely
> happy with) plan.
If the Xdamage/Xcomposite code you're talking about is the Fennec/
Hildon code, you won't be able to use that code out of the box because
it's designed for making all windowed plugins act like windowless
plugins and does scary and limited event forwarding. I've also only
ever tested it with flash and a test plugin.
Currently we have code paths for:
- Windowed plugins
- Windowless plugins
and an optional one (#ifdef'd to replace the windowed plugins path) for:
- Windowed plugins that act like windowless plugins
with this approach we'd add a another path for:
- Windowless plugins that act like windowed plugins that act like
and that scares me a little bit.
My guess is that modifying Xdamage/Xcomposite code to fit your needs
won't be any easier then just using a shared memory surface. Further,
if we're not going to use that approach in the end it seems better to
just do it right the first time. However, if you've already got
windowless plugins working using Xdamage/Xcomposite, I guess we can
always move to a saner approach later on.
For the record, here's my rough understanding of what WebKit does for
out of process plugins on Snow Leopard.
The implementation is in WebKit/mac/Plugins/Hosted. Unfortunately, the
WebKit tree does not seem to include the client code???
Overview and background
- All of the IPC is done using mach ports
- They have a client (presumably the plugin process) and a host
- CoreAnimation (CA) is a new UI API introduced in Leopard and on the
- A CALayer is a high level abstraction of a backing store. The
backing store is usually in video ram and CALayers are usually drawn
using a textured quads
- CARenderer is an object responsible for drawing a hierarchy of layers.
On host initialization
- The host manager (NetscapePluginHostManager.mm) creates a render
server using WKInitializeRenderServer() which returns a renderServerPort
- This renderServerPort is passed into _WKPHCheckInWithPluginHost(),
however I'm not sure what that does or where it's implemented.
On plugin initialization (WebHostedNetscapePluginView.mm:createPlugin)
- instantiatePlugin() is called which returns a renderContextID and a
- the renderContextID is presumably associated with the RenderServer
- if (useSoftwareRenderer) then
- the softwareRenderer draws in drawRect() using
- I'm not sure what the advantage of using CoreAnimation here is,
because I assume the client process wouldn't have more then one layer...
- otherwise we call WKMakeRenderLayer(renderContextID)
- WKMakeRenderLayer presumably creates one side of a CALayer that
is shared between the host and the client
- This shared layer is added to the existing layer hierarchy in the
browser and presumably drawn as though it was a regular layer
- Printing is done using either the softwareRenderer or shuttling a
bitmap between the plugin and the host.
Overall this seems like a good way to solve the cross-process hardware
acceleration problem. The shared CALayer is simple and flexible
(though it's unclear how they deal with synchronization).
Unfortunately, this method is implemented using private webkit symbols
(WKInitializeRenderServer, WKMakeRenderLayer) so it is not easy for us
to use unless Apple publicly exports the interface that these
functions use. I think it would be worth while to put pressure on
Apple to do so.
Yes, let's get radar on file, and we can ask some of our Apple
contacts for help getting those things exposed.
Yes, there is a reason. Windowless plugins require an X drawable to be
passed into them.
On 16-Jun-09, at 2:55 PM, Robert O'Callahan <rob...@ocallahan.org>
Oh. Well in that case things should be even easier. Just ship a
drawable id to the plugin process and have it draw to that just like
you would a window id.
Do you mean "whenever we paint our content window, ship the drawable ID
to the plugin process and have it synchronously draw to that"? That
would work, but it would lose some of the benefits of out-of-process
plugins; a slow or hung plugin would retard drawing of the window. The
overhead of the synchronous IPC would also impact painting.
Or do you mean "have the plugin process paint to an X pixmap
asynchronously, and copy that pixmap whenever we paint our content
window"? That would work, but for transparent plugins you actually need
two pixmaps, one with a white background and one with a black
background, so you can compute the plugin's alpha values.
We discussed this at length on IRC a while back. I guess no-one took
I saved the raw IRC conversation, but didn't extract clean notes from
it. Here's the raw conversation:
These notes will probably materialize near
http://wiki.mozilla.org/Content_Processes, but they're not my top
priority at the moment.