X11 windowless plugin API - rfc
Karl Tomlinson
windowless plugins on X11, together with code to add the
plugin-side support to a couple of example plugins:
https://bugzilla.mozilla.org/show_bug.cgi?id=137189
The API that I'm using is based on the existing NPAPIs for
windowless plugins on MS Windows and windowed plugins on X11.
I would be interested to hear what other features plugin
developers are needing.
Drawing
For drawing, a GraphicsExpose XEvent is sent through
NPP_HandleEvent. This contains the Xlib Drawable, its Display,
and the dirty rectangle (optional clip rectangle). The Drawable
is an offscreen Pixmap.
The plugin should draw to the Drawable with the offset specified
in the x and y members of the NPWindow structure, with the
clipping rectangle specified in the clipRect member, and with the
Visual and Colormap specified in the ws_info member. (A
NPP_SetWindow call notifies the plugin of any changes to these.)
"The clipRect member defines the clipping rectangle of the plug-in
in a coordinate system where the origin is the top-left corner of
the drawable" according to existing documentation. The dirty
rectangle is specified similarly.
Input events
Mouse, key, focus, and enter/leave events are provided in XEvents
through NPP_HandleEvent. The mouse events have x and y relative
to the plugin rectangle origin. This seemed the most useful
information to me, but is not the same as what is provided on
Windows. (I think on Windows the mouse events are relative to the
containing window.)
As the XEvents sent to the plugin are synthesized and there is not
a native window corresponding to the plugin rectangle, some of the
members of the XEvent structures are not set to their normal
Xserver values. e.g. Key events don't have a position. However
the information should be complete enough.
Discussion.
A notable difference to the NPAPI for MS Windows is that the
Drawable is not specified in the window member of the NPWindow
structure. The Drawable is different for each GraphicsExpose
event so passing this information through the NPWindow would
require a NPP_SetWindow for each expose event. It is tempting to
use the window member for something else such as the containing
native window, but I fear that if this were done some plugin
developers may misunderstand and (incorrectly) start drawing to
that window.
A piece of information that is currently provided for MS Windows
but not yet for X11 is the location of the plugin port relative to
the containing native window. This could be provided through a
ConfigureNotify XEvent, if needed. This was needed on MS Windows
because the mouse events were reported relative to the containing
window (which is not the case on X11). Does anyone need this
information for another purpose?
The NPNVnetscapeWindow on MS Windows sometimes returns the
containing native window and sometimes the document window. The
documentation suggests that it should be the containing native
window ("the native window in which plug-in drawing occurs"), and
this is what is currently implemented for X11, although the name
"NPNVnetscapeWindow" sounds more like a document window or a
application toplevel window.
If the ConfigureNotify XEvent is sent then it would be possible to
use it to send the containing window (through the event Window
member of the XConfigureEvent, which would be a little ugly), and
to use NPNVnetscapeWindow for a different window. What
information do plugin developers want?
(BTW, I did look into using XEmbed to pass the input events to the
plugin, but the GTK implementation of XEmbed at least seems to use
a GDK_INPUT_OUTPUT window which gets in the way of drawing. It
seems simpler to use the proven NPAPI interface rather than doing
our own special input only XEmbed implementation.)
otte.b...@googlemail.com
It's great that someone finally tackles the windowless mode. It's been
one of the features Swfdec lacked that weren't Swfdec's fault. ;)
)
The proposal looks good to me, I've added some (mostly syntactical)
comments below.
There's also two things that you don't mention in your mail and that I
think are important:
1) Please keep in mind when deciding on the events to pass to the
plugin that Handle_Event() returns a boolean and ideally this should
be handled. (Most requested: mouse scrolling) It's just something to
keep in mind, I think it works fine in this proposal.
2) Detecting that Mozilla really supports windowless mode. Current
Linux Mozilla returns TRUE for NPPVpluginTransparentBool, even though
it doesn't support it. So it's impossible to decide on the correct
mode.
On Jun 12, 4:29 am, Karl Tomlinson <mozn...@karlt.net> wrote:
> Mouse, key, focus, and enter/leave events are provided in XEvents
> through NPP_HandleEvent. The mouse events have x and y relative
> to the plugin rectangle origin. This seemed the most useful
> information to me, but is not the same as what is provided on
> Windows. (I think on Windows the mouse events are relative to the
> containing window.)
>
There's two reasons why I'd prefer the coordinates to be relative to
the drawable:
1) On Gtk (no clue about Qt or other toolkits) all X and Y coordinates
are relative to the window they belong to.
2) Drawing operations are relative to the window. I think it's
confusing if drawing is different from mouse events.
> A notable difference to the NPAPI for MS Windows is that the
> Drawable is not specified in the window member of the NPWindow
> structure. The Drawable is different for each GraphicsExpose
> event so passing this information through the NPWindow would
> require a NPP_SetWindow for each expose event. It is tempting to
> use the window member for something else such as the containing
> native window, but I fear that if this were done some plugin
> developers may misunderstand and (incorrectly) start drawing to
> that window.
>
I don't think this will cause issues. For one, it should be pretty
obvious that it doesn't work when you try it and it's properly
documented that you should call to the Drawable in the Handle_Event()
function and call Invalidate_Rect().
But the important thing for me as a plugin developer is having access
to the Window in some way. I can imagine me wanting to popup dialogs
or right-click menus, and those should be transient.
> A piece of information that is currently provided for MS Windows
> but not yet for X11 is the location of the plugin port relative to
> the containing native window. This could be provided through a
> ConfigureNotify XEvent, if needed. This was needed on MS Windows
> because the mouse events were reported relative to the containing
> window (which is not the case on X11). Does anyone need this
> information for another purpose?
>
As said above, right-click menus might be a use-case.
Cheers,
Benjamin
Karl Tomlinson
otte.benjamin writes:
> There's also two things that you don't mention in your mail and that I
> think are important:
> 1) Please keep in mind when deciding on the events to pass to the
> plugin that Handle_Event() returns a boolean and ideally this should
> be handled. (Most requested: mouse scrolling) It's just something to
> keep in mind, I think it works fine in this proposal.
Yes, the plugin should return false in Handle_Event if it doesn't
handle the evetn or if it wants the browser to act on the event.
For some reason Mozilla currently doesn't send mouse scroll events
to (windowless) plugins, and it always handles the event itself.
I don't know why this is, but it seems inconsistent. Perhaps this
is because some plugins were not correctly returning false,
perhaps because Mozilla doesn't handle the false return correctly,
or perhaps someone didn't think plugins would want mouse scroll
events...
> 2) Detecting that Mozilla really supports windowless mode. Current
> Linux Mozilla returns TRUE for NPPVpluginTransparentBool, even though
> it doesn't support it. So it's impossible to decide on the correct
> mode.
Yes. Thanks for the reminder. From what I've seen Mozilla
currently pretends to support windowless mode, so it could be fun
trying to detect this. I'll check this more thoroughly. I hope
we don't need to do something stupid like detecting a
NULL window->window.
>
> On Jun 12, 4:29 am, Karl Tomlinson <mozn...@karlt.net> wrote:
>> Mouse, key, focus, and enter/leave events are provided in XEvents
>> through NPP_HandleEvent. The mouse events have x and y relative
>> to the plugin rectangle origin. This seemed the most useful
>> information to me, but is not the same as what is provided on
>> Windows. (I think on Windows the mouse events are relative to the
>> containing window.)
>>
> There's two reasons why I'd prefer the coordinates to be relative to
> the drawable:
> 1) On Gtk (no clue about Qt or other toolkits) all X and Y coordinates
> are relative to the window they belong to.
Do toolkit widgets try to interpret coordinates of events
on windows of parent widgets?
> 2) Drawing operations are relative to the window. I think it's
> confusing if drawing is different from mouse events.
I should clarify that the drawable is often not aligned with the
window, so I'm afraid that (2) can't be addressed and it is going to
be "confusing". On expose events, Mozilla (trunk) creates an
offscreen pixmap only large enough to handle its dirty region, so
drawable offsets will change often.
The way I'd been looking at the events was that it was like
pretending that the plugin rectangle was a window, but it would of
course be possible (and probably easier for Mozilla) to provide
the events relative to the real window.
Would it actually save the plugin writers any effort by making the
coordinates relative to the window? I can only imagine it making
more work because the position of the plugin rectangle relative to
the containing window would need to be involved.
> ...
> But the important thing for me as a plugin developer is having access
> to the Window in some way. I can imagine me wanting to popup dialogs
> or right-click menus, and those should be transient.
That seems a good reason for NPNVnetscapeWindow to provide the
application toplevel window rather than the containing window.
Plugins could walk the window tree from the containing window up
to the root to determine the toplevel window, but it would be
easier for them not to.
But if NPNVnetscapeWindow is the toplevel window then the
containing window should probably be provided elsewhere
(particularly if mouse events are relative to this containing
window).
>
>> A piece of information that is currently provided for MS Windows
>> but not yet for X11 is the location of the plugin port relative to
>> the containing native window. This could be provided through a
>> ConfigureNotify XEvent, if needed.
which would essentially pretend "that the plugin rectangle was a
window".
Karl Tomlinson
> otte.benjamin writes:
>
>> 2) Detecting that Mozilla really supports windowless mode. Current
>> Linux Mozilla returns TRUE for NPPVpluginTransparentBool, even though
>> it doesn't support it. So it's impossible to decide on the correct
>> mode.
>
> Yes. Thanks for the reminder. From what I've seen Mozilla
> currently pretends to support windowless mode, so it could be fun
> trying to detect this.
Suggestion at https://bugzilla.mozilla.org/show_bug.cgi?id=386537
Karl Tomlinson
It should soon be available in a7pre linux trunk nightlies dated
2007-07-03 or later.
Karl Tomlinson
>
>> otte.benjamin writes:
>>
>>> 2) Detecting that Mozilla really supports windowless mode. Current
>>> Linux Mozilla returns TRUE for NPPVpluginTransparentBool, even though
>>> it doesn't support it. So it's impossible to decide on the correct
>>> mode.
>>
>> Yes. Thanks for the reminder. From what I've seen Mozilla
>> currently pretends to support windowless mode, so it could be fun
>> trying to detect this.
We've added a new NPN_GetValue variable NPNVSupportsWindowless so
that X11 plugins can reliably detect support: