best practices for asynchronous processing

[Alex Iskold]

Hi all,

I am trying to figure out what is the best way of
doing background processing of data from my extension.

The use case is that I synchronize with bunch of web services
and get updates (addes,deletes,updates) from them when my
extension starts.

While currently it takes 10-15 seconds, this might be an issue
in the future, if ther amount of data or services increases. I'd like to
implement it the right way in any case. So here are my questions:

1) I'd like to have UI responsive at all times, is this possible?

2) Is there any advatange to using nsIThread? I read that JS interpreter is
single threaded, so it sounds like the answer is 'no', but I want to
confirm.
If the answer is 'yes', please point to a good example of how it should
be used.

3) Is there a definitive way to avoid 'Stop script' prompt besides
implementing
all algorithms in terms of chunking/callLater combinations. I mean in
general all algs would
have to look like this:

alg( startIndex, endIndex )
while ( !done ) {
processData( startIndex, endIndex );
updateIndexes;
callLater( sleepTime, newStartIndex, newEndIndex );
}

4) Does explicit progress dialog avoid the Stop Script alert?

Thanks in advance for your help!

Alex

[James Ross]

Alex Iskold wrote:
> 1) I'd like to have UI responsive at all times, is this possible?

Depends what you define "responsive" as. All JS code attached to XUL
windows will run on the main thread, which will block any interaction
with the GUI.

> 2) Is there any advatange to using nsIThread? I read that JS interpreter is
> single threaded, so it sounds like the answer is 'no', but I want to
> confirm.
> If the answer is 'yes', please point to a good example of how it should
> be used.

I'm not sure "single threaded" is the phrase you wanted here. The JS
engine doesn't have any of its own ways to start code on a new thread,
or even any synchronisation objects, but it sure as heck can run JS on
threads other than the main one (certain existing features of the
browser will run JS on background threads, but they don't involve code
found on webpages or in XUL windows - see catches below).

For example, you can run a block of JS on its own background thread
using nsIThread, but there are some catches (all due to flawed design on
the part of the existing systems, IMHO).

First catch: the DOM is completely and utterly NOT thread-safe. Using
any part of the DOM anywhere apart from the main thread will cause
warnings and potentially anything can go wrong.

Second catch: the global scope object ('window') for XUL and
webpage-hosted JS is a DOM object. This means that it is not safe to
access global variables (including functions) from other threads (you
can use code layouts to work around this, but the third catch really
nails the coffin down well).

Third catch: any XPConnect calls made from JS use the global scope
object. This means that, even if your code doesn't touch the global
scope itself, if you call a method or access a property on an XPCOM
object you will be caught out if you're not on the main thread.

Basically, the result is that you can't put any of the code in the
window. The solution, therefore, is to use a component. The JS component
global scope object is thread-safe, so you can put code in there and
then launch it in its own thread via nsIThread. I know of no examples of
this, however, so it is all a bit theoretical (but I'm told it'll work).

> 3) Is there a definitive way to avoid 'Stop script' prompt besides
> implementing
> all algorithms in terms of chunking/callLater combinations.

No. There is an enhancement request to not show the slow script dialog
for chrome-based JS code, but frankly that is just hiding from the
problem. If you see the dialog, it means the UI has been hung for 5
seconds or more - that's a really bad experience for the user.

> 4) Does explicit progress dialog avoid the Stop Script alert?

The only way to stop the counter for the dialog is to stop the script,
which is usually done via setTimeout as I expect you've seen.

--
James Ross <sil...@warwickcompsoc.co.uk>
ChatZilla Developer

[Alex Iskold]

James,

Thanks a lot for your detail answer!

My take away is:

1) Try to find an example of running JS code in a separate Thread,
started from a component instead of a window.
See the what I tried below....

2) No matter what, even if it runs in a separate thread, the Stop scrip
dialog will appear,
therefore ALL algorithms whatsoever need to be abstracted using
basically iterator pattern.
Something like this:
while ( hasMoreData ) {
process( nextDataSlice );
}
and the programmer needs to ensure that process never takes > 5
seconds.

Alex

Here is the example of the thread starting that did not work:
(I started it from a window though, maybe this is the issue...)

Utils.prototype.asyncCall = function( func ) {
var threadEvent = new ThreadEvent();
threadEvent.func = func;
var Cct = Components.classes["@mozilla.org/thread;1"];
var Cit = Components.interfaces.nsIThread;
var mgr = Cct.createInstance(Cit);
mgr.init( threadEvent, 0, Cit.PRIORITY_NORMAL, Cit.SCOPE_GLOBAL,
Cit.STATE_JOINABLE);
mgr.join();
}

function ThreadEvent() {}

ThreadEvent.prototype.Run = function() {
alert( "asyncCall RUN CALLED: " + this.func );
this.func.call();
}

This does not work, it hangs forever, so more digging to do.

"James Ross" <sil...@warwickcompsoc.co.uk> wrote in message
news:V5OdnScFfb5PHaXZ...@mozilla.org...

[Nickolay Ponomarev]

On 4/9/06, Alex Iskold <alex....@adaptiveblue.com> wrote:
> 1) Try to find an example of running JS code in a separate Thread,
> started from a component instead of a window.
> See the what I tried below....

There is http://skrul.com/blog/projects/threaddemo , although I
haven't tried it myself yet.

> 2) No matter what, even if it runs in a separate thread, the Stop scrip
> dialog will appear,
> therefore ALL algorithms whatsoever need to be abstracted using
> basically iterator pattern.

I've been told that the stop script dialog appears only for code
running on UI thread, but I can't say for sure.

Nickolay

[James Ross]

Alex Iskold wrote:
> var mgr = Cct.createInstance(Cit);
> mgr.init( threadEvent, 0, Cit.PRIORITY_NORMAL, Cit.SCOPE_GLOBAL,
> Cit.STATE_JOINABLE);
> mgr.join();

I have to say, I don't actually know what the scope and state flags do
(and the documentation in nsIThread.idl is pathetic in this regard), but
I'm not sure you want to be joining the thread here - at least not when
testing. I believe I got a basic thread going OK some time ago using
just 0 for each parameter.

I believe there's also a neat trick with JavaScript and XPConnect's
handling of nsIRunable - you can pass a JS function into a nsIRunnable
parameter, and XPConnect will handle all the details.

It would almost certainly be advisable to use a debug build, if
possible, and use the -console command-line option to aid debugging
threading. In particular, that will be where any warnings, errors and
other mysterious goings on are displayed.

> ThreadEvent.prototype.Run = function() {

The IDL for nsIRunnable calls it "run", not "Run".

After I get the Venkman-on-Thunderbird patch burned into CVS, I think
I'll have a poke about with this code. It *ought* to be so simple, but
then I know how many times I've said that about the "Mozilla platform"
before... ;)