Using GWT modules as source for Gears workers

[Jason Essington]

Hi All

Sorry, this is going to be a long post with some code shown, but the short attention span version is:

When trying to get GWT generated code to work as a Gears Worker, I am ending up not ever getting any response back from the worker. It is behaving as if the onmessage function is never registered.

I'm looking for some cleaver hack that'll allow me to directly use GWT generated code as a worker, and ultimately I think there should be some clean way to use a gwt module directly as a worker.

So the long version goes something like:

I'm starting to try to figure out how to stuff GWT code into Gears Workers.

So as a simple test, I made a very simple GWT module that I'd like to use as a worker:

package com.grcs.worker.client;

import com.google.gwt.core.client.EntryPoint;

public class Worker implements EntryPoint {

  public void onModuleLoad() {

     init();

  }

  

  public native void init() /*-{

    gearsWorkerPool.onmessage = new function(msg, sender){ return this.@com.grcs.worker.client.Worker::onMessage(Ljava/lang/String;I)(msg, sender); };

  }-*/;

  

  public void onMessage(String msg, int sender)

  {

     sendMessage("Worker has been called.", sender);

  }

  

  private native void sendMessage(String msg, int sender) /*-{

     gearsWorkerPool.sendMessage(msg, sender);

  }-*/;

}

This gwt code is the equivalent of the following javascript (which works as a worker just fine):

function init(){

    gearsWorkerPool.onmessage = workerOnMessage;

}

function workerOnMessage(msg, sender){

  gearsWorkerPool.sendMessage("JS Worker has been called", sender);

}

init();

to get the code into a worker, I load the .js file (I was using the cache.js) that GWT generated, and take that string and stuff it into a worker:

   public void onResponseReceived(Request request, Response response)

   {

      String javascript = response.getText();

      try

      {

         worker1 = wp.createWorkerFromString(javascript);

      }

      catch (GearsException e)

      {

         showError(e);

      }

   }

Now, when I load the simple javascript, the worker initializes and works perfectly, but when I load the GWT compiled code, the worker acts like it didn't register the onmessage function. My parent's message handler is never called. Of course, the generated code is all contained in an anonymous function, so it is likely that it isn't executing any code (I'm skipping the bootstrap nocache.js file).

The generated code is attached.

So, my immediate question is what would I have to prepend or append to the gwt generated code to get it to properly initialize as a worker, and second, there's gotta be a better way to get gwt code into a worker, anyone got any great ideas?

-jason

[Ray Cromwell]

I haven't used gears, but am I missing something?

Where is gearsWorkerPool defined? If it is a global from an included <SCRIPT> element, you need to write it as $wnd.gearsWorkerPool

-Ray

  gearsWorkerPool.sendMessage ( "JS Worker has been called" , sender);

[Jason Essington]

Actually, this is a little tweaky ...

Workers don't actually have access to $wnd as far as I know (or
anything DOM for that matter)

the code should actually be executing in its own virgin context really.

As a reference, I'm using gwt-google-apis gears module as well

On Jun 15, 2007, at 4:46 PM, Ray Cromwell wrote:

>
> I haven't used gears, but am I missing something?
>
> Where is gearsWorkerPool defined? If it is a global from an
> included <SCRIPT> element, you need to write it as
> $wnd.gearsWorkerPool
>
> -Ray

I don't know if the actual js attached with the original post, so
here it is:

(function(){
var $wnd = window;
var $doc = $wnd.document;
var $moduleName, $moduleBase;
var _, package_com_google_gwt_core_client_ =
'com.google.gwt.core.client.', package_com_google_gwt_lang_ =
'com.google.gwt.lang.', package_com_grcs_worker_client_ =
'com.grcs.worker.client.', package_java_lang_ = 'java.lang.';
function nullMethod(){
}

function java_lang_Object(){
}

_ = java_lang_Object.prototype = {};
_.java_lang_Object_typeName = package_java_lang_ + 'Object';
_.java_lang_Object_typeId = 0;
function com_google_gwt_core_client_JavaScriptObject(){
}

_ = com_google_gwt_core_client_JavaScriptObject.prototype = new
java_lang_Object();
_.java_lang_Object_typeName = package_com_google_gwt_core_client_ +
'JavaScriptObject';
_.java_lang_Object_typeId = 5;
function com_grcs_worker_client_Worker_
$onModuleLoad__Lcom_grcs_worker_client_Worker_2(this$static){
this$static.init__();
}

function com_grcs_worker_client_Worker_init__(){

gearsWorkerPool.onmessage = new function(msg, sender){

return this.onMessage__Ljava_lang_String_2I(msg, sender);
}
();
}

function com_grcs_worker_client_Worker_onMessage__Ljava_lang_String_2I
(msg, sender){
this.sendMessage__Ljava_lang_String_2I('Worker has been called.',
sender);
}

function
com_grcs_worker_client_Worker_sendMessage__Ljava_lang_String_2I(msg,
sender){
gearsWorkerPool.sendMessage(msg, sender);
}

function com_grcs_worker_client_Worker(){
}

_ = com_grcs_worker_client_Worker.prototype = new java_lang_Object();
_.init__ = com_grcs_worker_client_Worker_init__;
_.onMessage__Ljava_lang_String_2I =
com_grcs_worker_client_Worker_onMessage__Ljava_lang_String_2I;
_.sendMessage__Ljava_lang_String_2I =
com_grcs_worker_client_Worker_sendMessage__Ljava_lang_String_2I;
_.java_lang_Object_typeName = package_com_grcs_worker_client_ +
'Worker';
_.java_lang_Object_typeId = 0;
function java_lang_String_$clinit__(){
java_lang_String_$clinit__ = nullMethod;
{
java_lang_String__1_1initHashCache__();
}
}

function java_lang_String__1_1initHashCache__(){
java_lang_String_$clinit__();
java_lang_String_hashCache = {};
}

_ = String.prototype;
_.java_lang_Object_typeName = package_java_lang_ + 'String';
_.java_lang_Object_typeId = 9;
var java_lang_String_hashCache = null;
function init(){
com_grcs_worker_client_Worker_
$onModuleLoad__Lcom_grcs_worker_client_Worker_2(new
com_grcs_worker_client_Worker());
}

function gwtOnLoad(errFn, modName, modBase){
$moduleName = modName;
$moduleBase = modBase;
if (errFn)
try {
init();
}
catch (e) {
errFn(modName);
}
else {
init();
}
}

var com_google_gwt_lang_Cast_typeIdArray = [{}, {2:1}, {2:1}, {2:1},
{2:1}, {1:1}, {2:1}, {2:1}, {2:1}, {3:1}];

if (com_grcs_worker_Worker) {
var __gwt_initHandlers =
com_grcs_worker_Worker.__gwt_initHandlers;
com_grcs_worker_Worker.onScriptLoad(gwtOnLoad);
}
})();

[Jason Essington]

O.K. hopefully my previous posts weren't completely incoherent babble.

But basically what I was trying to do was to create a GWT module that could be directly used as a GoogleGears worker, I was using the gwt-google-apis project for the GWT/Gears integration part.

So, the first trick is getting a compiled GWT module to actually kick off and do something, the second thing is to register an onMessage handler when it does finally kick off.

So, the first thing I did was created a new type of entry point called a Worker ... this worker makes the onModuleLoad() method final, and handles initialization and registration of the required onMessage method:

import com.google.gwt.core.client.EntryPoint;

public abstract class Worker implements EntryPoint

{

   protected static native void sendMessage(String msg, int sender) /*-{

       gearsWorkerPool.sendMessage(msg, sender);

   }-*/;

   

   /**

    * init() is provided as an empty method. Override this method if there is some initialization

    * other than registering the onMessage handler required.

    */

   public void init() {

   }

   /**

    * Implement this method to handle messages from other workers.

    * @param msg Message from other worker

    * @param sender Sending worker's ID

    */

   public abstract void onMessage(String msg, int sender);

   /**

    * made final so unwitting developers don't break worker initialization

    */

   public final void onModuleLoad()

   {

      init();

      registerOnMessage();

   }

   private native void registerOnMessage() /*-{

      var foo = this; // persist this object past the initialization stage.

      gearsWorkerPool.onmessage = function(msg, sender){ return foo.@com.grcs.worker.client.Worker::onMessage(Ljava/lang/String;I)(msg, sender); };

   }-*/;

}

any one extending this class only need to implement the onMessage() method, and use the subclass as the entrypoint for the module. So far, it is necessary to avoid any DOM methods, or really anything that uses deferred binding since we are going to want to have to make a decision about what file to load depending upon platform (right yet).

then once GWTCompile has its way with this code, there should only be one *.cache.js file.

your main GWT application can load this file using an XHR.

Then, there's the issue that the *.cache.js expects to find window, and document, so we'll have to prepend a couple of lines of code to pretend like those objects exist.

function window(){};

window.document = "Fake Document";

and of course there's the issue of actually kicking this module off, that's two more lines of code to prepend:

function my_module_path_ModuleName(){};

my_module_path_ModuleName.onScriptLoad = function(gwtOnLoad){ gwtOnLoad(null, "", "");}; 

the last thing to do is simply to feed the resulting string to the worker pool:

            worker1 = workerpool.createWorkerFromString(javascript);

From there, you can just begin sending messages to the newly created worker!

-jason

[Rob Jellinghaus]

On Jun 18, 10:42 am, Jason Essington <jason.essing...@gmail.com>
wrote:

> From there, you can just begin sending messages to the newly created
> worker!

So you're saying you got it working? :-)

Having been through some muckery with the 1.4 startup changes breaking
the GWTJSF code, I'm a bit concerned that your technique -- though
cool -- is a bit exposed to the implementation details of the GWT
startup sequence. But your success with mocking window and
window.document indicates that maybe it's not too bad. As long as the
GWT compiler doesn't expect or require any preexisting variables
beyond those two, it's probably fine, and what else could it want,
really?

Cheers!
Rob

[Jason Essington]

Yes, I did get it working ... even managed to get Safari to perform
an XHR from a worker using GWT code! (that is naughty by the way, and
will likely be disabled in gears very soon)

Actually at this moment startup is a bit tweaky ...

The startup sequence is solid, no problem there, since the code is
loaded via a XHR, there is no ambiguity about startup.

The remaining issues are:
1) workers really need to stay away from anything in the DOM class,
and also the GWT class would require a more robust mockup of window
and document. It would be awesome if the compiler could whine if a
Worker entry point used code that attempted to do any DOM modification.

2) I need a new kind of selection script. one that can be fed to eval
() and return the strongname for the particular platform. If none of
the used code performs deferred binding, or internationalization,
then there should only be one permutation, but if something causes
additional permutations there's no way to get ahold of them.

I'm currently working towards a WorkerUtility that allows creating a
worker simply by doing something like:

int workerId = WorkerUtil.loadModule("com.my.module.MyModule");
Actually, this may not work so well since the process of loading the
module is really asynchronous, and trying to shoehorn it into a
synchronous call is ugly at best.

But that'd require the new selection script ... Scott any ideas on
this one?

-jason

[Scott Blum]

I think we really need a comprehensive strategy to target Gears worker threads, honestly.  It would probably involve compiler changes to generate a new selection script as well as perhaps a sibling to gwt-user.jar that contains only the APIs that are safe to call from within a worker (no DOM).

Scott

[Miguel Méndez]

Yes, this sounds like the ideal approach.

--
Miguel

[Jason Essington]

Here's what I've come up with so far for using GWT code directly in Gears workers ...

first a code snippet:

   wp.createWorkerFromModule(WORKER_MODULE, new PrimeWorkerCallback());

This is all it takes to load a GWT module into a Gears Worker once a worker pool has been created. 

So, how would it work?

First create the module that you want to use as a worker, but instead of implementing an EntryPoint, implement a subclass of EntryPoint called Worker.

Worker would expose init() and onMessage(String msg, int sender) methods that could be implemented to give the worker functionality.

Then in the main module, the worker pool could be created, and a new worker loaded based upon the module name.

So, would it actually work?

yup: http://jason.essington.googlepages.com/GWT-GearsWorker.html

What did it really take to do that?

1) I first needed to create a new type of entry point that would properly register onmessage handlers and such, so I created the "Worker" entry point. it finalizes the onModuleLoad() method, and exposes two new methods ( init() and onMessage() ) that could be used by the worker.

2) I needed to extend WorkerPool so that it could load files using XHR. (Initially I just loaded the javascript payload directly, but that was a bit of a hassle, so I needed a way to create a selection script that could identify which javascript file the particular browser should be loading. see #3). So, the WorkerPool looks for a file named [MODULE_NAME]-gg.js, and tries to load it using XHR. once the file is loaded, it evaluates the script which hopefully returns the strong name of the actual javascript payload. with this URL, WorkerPool performs a second XHR to fetch the worker script (which is simply one of the *.cache.js files). then it wraps that script in a try/catch block (for more verbose error handling), mocks up a window and document object, and  creates a [MODULE_NAME] object (usually created by the selection script). the [MODULE_NAME] object has a method that will actually initialize the module when it is fed to the worker. Finally the whole thing is fed off to the WorkerPool's regular createWorkerFromString(); method. Once the WorkerPool returns with a worker ID, a callback is fired to notify the original caller that the worker is ready. Since loading a worker from a module involves 2 XHRs, it is not possible or even desirable to perform this step as a synchronous operation.

3) The selection script is really the most hackish part of the process at this point. Since I was after the instant gratification, I simply modified the SelectionScriptTemplat-xs.js file (Thanks for pointing that out to me Scott). basically I stripped it down, and made it just evaluate to a string. This is the part that could really use some more polishing up on, it is ugly, and fragile at this point since there is no error checking about attempting to access the window or document objects.

4) once both projects (the parent module, and the worker module) have been compiled, the *.cache.js, and -xs.nocache.js (renamed to -gg.nocache.js) files need to be copied from the worker module's output directory to the final deployment location. all of the regular files required for deployment are copied from the parent's output directory to the deployment location.

Now what?

First anyone interested should have a look at the code that I tossed together and decide if this is even the correct way to solve the problem. (I'm attempting to attach the files to this mail, if the forum strips the files, feel free to contact me directly)

If it is, then it might be nice to have a much cleaner selection script generator (maybe a compiler switch could be used to tell it that the module is going to be a gears worker). It might also be nice to continue to support resource injection for workers (at least for javascript libraries), so perhaps the selection script could return an array of URLs (this is a topic that could certainly use some discussion)?

Then the actual WorkerPool and associated interfaces could be cleaned up, and submitted back to gwt-google-apis.

-jason

[Ray Cromwell]

I actually think this type of thing should be extended as a general purpose separate-compilation system. Today, GWT takes all reachable inherited modules + JRE layer and boils them down to a monolithic source for each permutation.  But sometimes, you want to create reusable JS libraries outside of this process, libraries which are not bootstrapped in the standard way.  The need to run workers is one example, generating ActionScript compatible EcmaScript would be another. The various widgets and gadget APIs are still another, some of which don't expose a complete browser environment.

-Ray

On 6/20/07, Jason Essington <jason.e...@gmail.com> wrote:

Here's what I've come up with so far for using GWT code directly in Gears workers ...

first a code snippet:

   wp.createWorkerFromModule( WORKER_MODULE , new PrimeWorkerCallback());

This is all it takes to load a GWT module into a Gears Worker once a worker pool has been created. 

So, how would it work?

First create the module that you want to use as a worker, but instead of implementing an EntryPoint, implement a subclass of EntryPoint called Worker.

Worker would expose init() and onMessage(String msg, int sender) methods that could be implemented to give the worker functionality.

Then in the main module, the worker pool could be created, and a new worker loaded based upon the module name.

So, would it actually work?

yup:  http://jason.essington.googlepages.com/GWT-GearsWorker.html

What did it really take to do that?

1) I first needed to create a new type of entry point that would properly register onmessage handlers and such, so I created the "Worker" entry point. it finalizes the onModuleLoad() method, and exposes two new methods ( init() and onMessage() ) that could be used by the worker.

2) I needed to extend WorkerPool so that it could load files using XHR. (Initially I just loaded the javascript payload directly, but that was a bit of a hassle, so I needed a way to create a selection script that could identify which javascript file the particular browser should be loading. see #3). So, the WorkerPool looks for a file named [MODULE_NAME]- gg.js, and tries to load it using XHR. once the file is loaded, it evaluates the script which hopefully returns the strong name of the actual javascript payload. with this URL, WorkerPool performs a second XHR to fetch the worker script (which is simply one of the *.cache.js files). then it wraps that script in a try/catch block (for more verbose error handling), mocks up a window and document object, and  creates a [MODULE_NAME] object (usually created by the selection script). the [MODULE_NAME] object has a method that will actually initialize the module when it is fed to the worker. Finally the whole thing is fed off to the WorkerPool's regular  createWorkerFromString(); method. Once the WorkerPool returns with a worker ID, a callback is fired to notify the original caller that the worker is ready. Since loading a worker from a module involves 2 XHRs, it is not possible or even desirable to perform this step as a synchronous operation.

3) The selection script is really the most hackish part of the process at this point. Since I was after the instant gratification, I simply modified the SelectionScriptTemplat-xs.js file (Thanks for pointing that out to me Scott). basically I stripped it down, and made it just evaluate to a string. This is the part that could really use some more polishing up on, it is ugly, and fragile at this point since there is no error checking about attempting to access the window or document objects.

4) once both projects (the parent module, and the worker module) have been compiled, the *.cache.js, and -xs.nocache.js (renamed to -gg.nocache.js) files need to be copied from the worker module's output directory to the final deployment location. all of the regular files required for deployment are copied from the parent's output directory to the deployment location.

Now what?

First anyone interested should have a look at the code that I tossed together and decide if this is even the correct way to solve the problem. (I'm attempting to attach the files to this mail, if the forum strips the files, feel free to contact me directly)

If it is, then it might be nice to have a much cleaner selection script generator (maybe a compiler switch could be used to tell it that the module is going to be a gears worker). It might also be nice to continue to support resource injection for workers (at least for javascript libraries), so perhaps the selection script could return an array of URLs (this is a topic that could certainly use some discussion)?

Then the actual WorkerPool and associated interfaces could be cleaned up, and submitted back to gwt-google-apis.

-jason