Proxy Events, Proxy Objects, and UI.
Doug Turner
Please reply only to the netscape.public.mozilla.xpfe newsgroup.
I am extending Alecf's initial ProxyEvent code to allow anyone to call
any object across thread boundaries. The main reason that I needed this
is that XPInstall runs a long Javascript on a non-ui thread and it needs
to access UI elements such as a progress meter and so on. I believe
that other people working on Seamonkey need similar solutions. In this
email, I will try to explain how proxy events are working today and
hopefully inspire feedback.
Proxy events work on Proxy objects. A proxy object is a simple class
that implements the interface of the "real" object. This implementation
does some data marshalling and then post events to a given event queue.
The event handler, when popped off the event queue, calls the "real"
object.
For a small example, I make a proxy object of the nsAppShellService.
(This allows me to create a window with any URL). Here is a look at
RegisterTopLevelWindow().
NS_IMETHODIMP nsAppShellProxy::RegisterTopLevelWindow(nsIWebShellWindow*
aWindow)
{
nsXPTCVariant var[1];
var[0].flags = nsXPTCVariant::PTR_IS_DATA;
var[0].ptr = aWindow;
nsProxyEvent *event = nsProxyEventCreate( m_eventQueue,
m_realObject,
9, // call meathod
number
1, // number of
params
var); // parameters
nsresult rv = nsProxyEventPostSync(event);
nsProxyEventDelete(event);
return rv;
}
So we stuff aWindow into a struct nsXPTCVariant. Then we call
nsProxyEventCreate(). The first two parameters are always the same.
Since we are using XPConnect's XPTC_InvokeByIndex(), the next parameter
is the index in the vtbl for this object. By using
XPTC_InvokeByIndex(), it reduces the proxy code size as well as
simplifies data marshalling. The call to nsProxyEventPostSync() blocks
and when it returns, any changed out values will be in nsXPTCVariant.
(In this example, on exit we should store var[0].ptr in aWindow )
As you can see, it is pretty trivial to create on of these. In fact, we
could generate these proxies from the xpidl compiler. (More on that
later)
Once you have a proxy object, you can call it from any thread. The two
piece of information of a proxy object are (1) the event queue that you
want to post events to, and (2) the instance of the class that this is a
proxy of. Eventually, (may there is already) there should be a way via
the Service Manager to get the UI event queue. This would allow a
default value for the event queue. We should be able to get rid of the
instance parameter and replace it with the CID of the "real" object.
Then the proxy object will take care of creating and releasing this real
object.
Once you create a proxy object, it essentially like the real object.
This is a snippet of sample code, which will create a window and load a
url from a non UI-thread:
nsIAppShellService* appShell;
rv = nsServiceManager::GetService(kAppShellServiceCID,
kIAppShellServiceIID,
(nsISupports**)&appShell);
if (NS_FAILED(rv))
{
return rv;
}
nsAppShellProxy *appShellProxy = new nsAppShellProxy(
installInfo->GetUIEventQueue(),
appShell);
nsString *aCID = new nsString("00000000-dead-beef-0000-000000000000");
nsIWebShellWindow* newWindow;
nsIURL* url;
char* urlstr = "resource:/res/samples/WhereIsWaldo.xul";
NS_NewURL(&url, urlstr);
appShellProxy->CreateDialogWindow(nsnull,
url,
*aCID,
newWindow,
nsnull,
nsnull,
250,
125);
The call installInfo was passed into this thread via the
PR_CreateThread() call. All GetUIEventQueue does is returns a pointer
to the event queue that was stored before this thread was started. So
there you have it, a window that displays content even if your thread
goes to sleep. (If you remove the proxy code and simply call the
AppShellService directly, you will find out why we have to go to this
much trouble. The window will never be drawn.)
The next problem is how do you get any information to or from this
dialog window. You can not just walk the DOM tree directly, but
theoretically we can apply the same method to the DOM classes. This
would allow us to do anything ui from a non-ui window. We can either
hand write the classes that we need and hope that the interface never
changes, or we can write a xpidl compiler extension that will generate
these proxy objects. Like the above example, we can hand write the
proxy objects that we need. The problem with this is that we are
limited to what proxy events are available and are face with the threat
that when an interface changes will not just stop working, but crash.
On the other hand, the problem with generating proxies for all the DOM
and other commonly used UI related class is that there will be some code
bloat (How much?).
Any feedback or help would be appreciated.
--
Doug Turner
do...@netscape.com
Alec Flett
>
> This message has been cross-posted:
> Please reply only to the netscape.public.mozilla.xpfe newsgroup.
>
> I am extending Alecf's initial ProxyEvent code to allow anyone to call
> any object across thread boundaries. The main reason that I needed this
> is that XPInstall runs a long Javascript on a non-ui thread and it needs
> to access UI elements such as a progress meter and so on. I believe
> that other people working on Seamonkey need similar solutions. In this
> email, I will try to explain how proxy events are working today and
> hopefully inspire feedback.
>
Doug, this stuff looks good...
I like what you've done with the varargs thing too....are you using the
XPCall stuff to actually make the call instead of using the
nsProxyEventHandler callback? That's very cool.
I'm not sure I understand how the auto-creation and destruction thing is
useful because I don't know how it would get any kind of contextual
information in the UI thread if it was created through events.... but I
assume you have a use for this for XPInstall...
I've been trying to come up with some way of making a generic object
that could automatically marshal any interface using information stored
in the typelib, but I can't figure out a way to actually create that
generic object in memory... I wonder if the
very least we could do would be to rig XPCOM to say "Give me a proxy
event for this object with this interface" and have XPCOM keep track of
proxy-implementations of certain interfaces:
nsIMyInterface *obj;
nsComponentManager::CreateInstance("component://netscape/something",
nsnull,
nsIMyInterface::GetIID(),
(void **)&obj);
// Now create a proxy object to pass to other threads..
nsIMyInterface *proxyobj;
nsComponentManager::CreateInstanceProxy(obj,
nsIMyInteface::GetIID(),
(void **)&proxyobj);
This wouldn't be too difficult for XPCOM, actually. It would just need
to maintain a mapping in the registry from IID->CID so that given an
IID, it knew the how to create a proxy object for it.
You could have some sort of call in the component manager:
nsComponentManager::RegisterComponentProxy(const nsIID &aIID,
const nsCID &aClass,
etc..);
that establishes this mapping...
Alec
John Bandhauer
As usual I disagree more than I agree :)
My questions and suggestions are in the following areas:
- do you really really need this?
- does anyone else need it?
- do you really need a transparent/generic system?
- what additional complexities are lurking?
- if you really do need to do this, then let's not generate any
code.
So, why do you need this? As I understand it you have scripts
that you want to have run from start to finish while a download
is going on. The script may or may not show a UI. For this you
need to be on another thread, but have the ability to run some
code on the UI thread. Right?
Does the script really need to run straight through to
completion? What about an event driven model? In this scheme the
script would (optionally) define event handlers for things like:
init, block read, failure, completion, etc. The script could also
have the standard handlers for buttons clicked (etc.) for
whatever UI it shows. You could supply the script an api through
which it could start an asynch download, check on
progress/status, abort, etc. But the bulk of the user code in the
script only runs during event handlers. Isn't this really how
normal client JavaScript code works? Would this not work?
Couldn't it then just run on the UI thread?
Has anyone replied to say that this proxy event technology is
needed for anything else?
Even if you must cross thread boundaries, is it not possible that
you have a limited set of interfaces that need to be 'remoted'?
Perhaps custom wrappers could be created that make the specific
necessary functionality available. Those wrappers might not just
reflect existing interfaces, but rather define a new set of
interfaces to satisfy specific requirements.
If there really is no other viable scheme by which your
objectives can be reached, I urge you to be careful in your
assumptions about just how transparent and generic the system can
be. My experience with corba is that code sometimes need to be
restructured and take into account the idiosyncrasies of using
this thread crossing layer. I would bet that there will be more
problems and oddities than you might at first expect.
At one point you wrote of also supporting non-blocking calls.
Blocking calls will be hard enough. Don't underestimate how big a
mess you'd have with non-blocking calls.
A very substantial set of xpcom interfaces in our tree are for
one reason or another not very compliant with xpcom rules. You
will likely butt up against that fact as you try to generically
reflect them.
You will also need to have these proxies work for calling in both
directions, no? Won't you need to route spontaneous UI (and
other) events from the UI thread back to your 'main' thread?
As you know, xptcall's invoke code supplies zero type safety.
Handwritten calls into it are just plain dangerous. You mention
using the xpidl compiler to generate code. This is quite doable,
but is exactly the route we decided *not* to go down in doing
xpconnect. If you must build this technology, I suggest that you
use and adapt more of the code in xpconnect and xptcall to do
your bidding. xptcall supports a 'stubs' system that allows for
the implementation of a class whose instances can dynamically
impersonate *any* xpcom interface that has been expressed in
xpidl. With not too much work you can build a class which can do
all of the proxy work for you to dynamically remote an interface
with no generated code.
The similarities between the tread boundary bridging that you
want to do and the language boundary bridging that xpconnect does
are just too close to ignore.
Still, I strongly urge you to look for possible solutions (like
the event-driven model I suggested above) that make this thread
crossing code unnecessary.
John.
--
Opinions are my own. Any similarity to those of
Netscape Communications is purely coincidental.
Brendan Eich
blocking-calls-only, thread-crossing proxy to do...@netscape.com a while
ago, then disappeared on vacation and provided no help or advice.
The problem with event-handler (aka callback or pseudo-threaded) scripts
is "just" programming complexity. We are not the only authors of
xpinstall scripts. Many consumers of mozilla.org code, even "end
users", will be writing these things. Should they be forced to write
callback style code around every lengthy operation (not just file i/o --
don't forget decompression)?
It may be that Netscape's 4.x-era customers have written SmartUpdate(tm)
scripts in a blocking style rather than a callback style. Enough such
old scripts and it would be bad ownership for mozilla.org to switch to
an incompatible, as well as a harder-to-program, model. Doug and
dveditz, please comment.
I think jband's point about restricted or limited proxying is right on.
The proxy code could be very DOM-specific; there might not be a
one-for-one relation between proxy classes and DOM classes, perhaps even
one proxy class that could be used for any DOM class. Doug, can you
fill us in on what you've prototyped?
/be
Doug Turner
Thank you for your response. My comments are below.
John Bandhauer wrote:
> Doug,
>
> As usual I disagree more than I agree :)
I am glad: groupthink is bad.
> My questions and suggestions are in the following areas:
> - do you really really need this?
I am pretty sure. There has been no other solution to solve the
problems that XPInstall has run into. I am sure that we have explain
the problem via our status reports and through jim. XPInstall
installation scripts differ from the majority of Javascripts. Most
Javascripts that are run are small and can be quickly run. On the other
hand:
(1) XPInstall installation scripts are sometime very complex and can
require long execution time due to unzipping or native file system
actions.
(2) XPInstall also needs to be able to control windows.
So we do not hang the entire browser while XPInstall is working on a
script, we moved to a separate thread. This solves the first problem,
but now we can not control windows. How do we find out if a user click
a cancel button to tell us to stop the installation? How do we set
information in our progress dialog? It seamed to us that we needed to
be able to pass messages to the UI thread.
Furthermore, as a result to moving to a new thread, the window.*
namespace is not reflected into our thread. This means that
conveniences like calling alert() to debug your script do not work in
our context. We spoke to Brendan about this, and need offer a solution
of writing a "proxy resolver" which would append the namespace which we
created. He too would need some way of getting messages into the the UI
event queue.
> - does anyone else need it?
Other than xpinstall, I believe Alecf needed something similar for
imap. He may be able to fill in the details. Alec?
> - do you really need a transparent/generic system?
Given that we "really really need" this system, it would be great if it
was generic. I would hate to implement each interface that I need to be
proxied differently. I think the work the current state is very generic
and can be generated without too much trouble.
I do have my doubts that it can or should be transparent. ProxyObjects
should not be cumbersome to use, but users should know that they are
using them.
> - what additional complexities are lurking?
I am not sure what you are asking here.
- if you really do need to do this, then let's not generate any code.
That would be fine with me, but it is a bit easier to say it than
implement it.
> So, why do you need this? As I understand it you have scripts
> that you want to have run from start to finish while a download
> is going on. The script may or may not show a UI. For this you
> need to be on another thread, but have the ability to run some
> code on the UI thread. Right?
Yes, as stated above.
> Even if you must cross thread boundaries, is it not possible that
> you have a limited set of interfaces that need to be 'remoted'?
> Perhaps custom wrappers could be created that make the specific
> necessary functionality available. Those wrappers might not just
> reflect existing interfaces, but rather define a new set of
> interfaces to satisfy specific requirements.
Sure. I could agree that we should only make remote objects for things
that we need. We should first look at how big these proxied object will
be. It may be diminishing returns if we invest tons of time to figure
out which objects we need and which objects we do not. It may be better
just to say heck with it, and generate a proxy object for every dom
object. We will see.
I would disagree with us taking on the burden of having a new set of
interfaces for a solution. This just complicates things more than they
are.
> If there really is no other viable scheme by which your
> objectives can be reached, I urge you to be careful in your
> assumptions about just how transparent and generic the system can
> be. My experience with corba is that code sometimes need to be
> restructured and take into account the idiosyncrasies of using
> this thread crossing layer. I would bet that there will be more
> problems and oddities than you might at first expect.
I hope that you can offer some of your experience both in the design and
development of our solution.
> At one point you wrote of also supporting non-blocking calls.
> Blocking calls will be hard enough. Don't underestimate how big a
> mess you'd have with non-blocking calls.
I rewrote the code in C++ and have done away with non-blocking class.
It was just getting too hairy.
> You will also need to have these proxies work for calling in both
> directions, no? Won't you need to route spontaneous UI (and
> other) events from the UI thread back to your 'main' thread?
>
> As you know, xptcall's invoke code supplies zero type safety.
> Handwritten calls into it are just plain dangerous. You mention
> using the xpidl compiler to generate code. This is quite doable,
> but is exactly the route we decided *not* to go down in doing
> xpconnect. If you must build this technology, I suggest that you
> use and adapt more of the code in xpconnect and xptcall to do
> your bidding. xptcall supports a 'stubs' system that allows for
> the implementation of a class whose instances can dynamically
> impersonate *any* xpcom interface that has been expressed in
> xpidl. With not too much work you can build a class which can do
> all of the proxy work for you to dynamically remote an interface
> with no generated code.
I will take a look at this. I was looking for something that could
impersonate another class. How does this work? Can you point me at an
example of this in action?
It would be great to get rid of all the proposed generated code. If we
could impersonate any object and have the ability to implement the
functionality of each method, I think that this generated code will go
away.
--
Doug Turner
do...@netscape.com
John Bandhauer
>
> I will take a look at this. I was looking for something that could
> impersonate another class. How does this work? Can you point me at an
> example of this in action?
This is declared in
mozilla/xpcom/libxpt/xptcall/public/xptcall.h. Have the class
inherit from nsXPTCStubBase and implement the 'GetInterfaceInfo'
and 'CallMethod' methods.
The only example at present is in
mozilla/js/src/xpconnect/xpcwrappedjs.c. It is somewhat
compilcated. A wrapped xpcom object may have multiple interfaces
and xpcom rules dictate that they have specific QI sematics to
link them together. A linked list of these wrappers for any given
object is maintained. I have the concept of 'class' objects that
represent the types of interfaces that are wrapped. They factor
data and funtionality common to all wrappers for a given type of
interface. In this code I am wrapping a JS object and making it
appear as an xpcom object with one or more interfaces. The scheme
is highly adaptable to the thread proxy situation, but this will
entail some work.