Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
[RFC] Modules for workers
87 views
Skip to first unread message
David Rajchenbach-Teller
May 18, 2013, 6:09:39 AM5/18/13
to dev-platform
Hi everyone,
As part of the ongoing effort to make (Chrome) Workers useful for
platform refactorings, we have been working on a lightweight module
loader for workers (bug 872421). This loader implements a minimal
version of CommonJS modules, aka require.js.
Example:
// Setup the loader. We need this once per worker.
importScripts("resource://gre/modules/workers/loader.js");
// Import a few modules
let Logger = require("resource://gre/modules/workers/logger.js");
let Storage = require("resource://gre/modules/workers/storage.js");
// ...
// All values that are not exported are private to the module
// ...
exports.foo = function() { ... } // Export a value |foo|
exports.bar = 5; // Export a value |bar|
Once this loader lands, we will need some convention for where to place
modules for workers. Unfortunately, main thread modules (both .jsm and
Jetpack) can generally not be used by worker, due to different module
semantics, and more importantly due to the fact that most main thread
modules depend indirectly on XPCOM/XPConnect.
Given that main thread modules are rooted in
resource://gre/modules/
and Jetpack modules are rooted in
resource://gre/modules/commonjs/
I would like to place worker modules in
resource://gre/modules/workers/
Any comments?
Cheers,
David
--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla
As part of the ongoing effort to make (Chrome) Workers useful for
platform refactorings, we have been working on a lightweight module
loader for workers (bug 872421). This loader implements a minimal
version of CommonJS modules, aka require.js.
Example:
// Setup the loader. We need this once per worker.
importScripts("resource://gre/modules/workers/loader.js");
// Import a few modules
let Logger = require("resource://gre/modules/workers/logger.js");
let Storage = require("resource://gre/modules/workers/storage.js");
// ...
// All values that are not exported are private to the module
// ...
exports.foo = function() { ... } // Export a value |foo|
exports.bar = 5; // Export a value |bar|
Once this loader lands, we will need some convention for where to place
modules for workers. Unfortunately, main thread modules (both .jsm and
Jetpack) can generally not be used by worker, due to different module
semantics, and more importantly due to the fact that most main thread
modules depend indirectly on XPCOM/XPConnect.
Given that main thread modules are rooted in
resource://gre/modules/
and Jetpack modules are rooted in
resource://gre/modules/commonjs/
I would like to place worker modules in
resource://gre/modules/workers/
Any comments?
Cheers,
David
--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla
Dave Townsend
May 20, 2013, 2:53:10 PM5/20/13
to
On 5/18/2013 3:09 AM, David Rajchenbach-Teller wrote:
> Hi everyone,
>
> As part of the ongoing effort to make (Chrome) Workers useful for
> platform refactorings, we have been working on a lightweight module
> loader for workers (bug 872421). This loader implements a minimal
> version of CommonJS modules, aka require.js.
>
>
> Example:
>
> // Setup the loader. We need this once per worker.
> importScripts("resource://gre/modules/workers/loader.js");
>
> // Import a few modules
> let Logger = require("resource://gre/modules/workers/logger.js");
> let Storage = require("resource://gre/modules/workers/storage.js");
>
> // ...
> // All values that are not exported are private to the module
> // ...
>
> exports.foo = function() { ... } // Export a value |foo|
> exports.bar = 5; // Export a value |bar|
>
>
>
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
On the face of it it looks like it should be possible for Jetpack's
> Hi everyone,
>
> As part of the ongoing effort to make (Chrome) Workers useful for
> platform refactorings, we have been working on a lightweight module
> loader for workers (bug 872421). This loader implements a minimal
> version of CommonJS modules, aka require.js.
>
>
> Example:
>
> // Setup the loader. We need this once per worker.
> importScripts("resource://gre/modules/workers/loader.js");
>
> // Import a few modules
> let Logger = require("resource://gre/modules/workers/logger.js");
> let Storage = require("resource://gre/modules/workers/storage.js");
>
> // ...
> // All values that are not exported are private to the module
> // ...
>
> exports.foo = function() { ... } // Export a value |foo|
> exports.bar = 5; // Export a value |bar|
>
>
>
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
module loader to load these worker modules. Is that something that seems
desirable or are these modules not useful outside of workers?
David Rajchenbach-Teller
May 22, 2013, 2:39:05 PM5/22/13
to Dave Townsend, dev-pl...@lists.mozilla.org
It should be possible to share some modules between Jetpack and Workers,
for Jetpack modules that do not depend on DOM or XPCOM and Worker
modules that do not depend on Worker-only code. This is not an immediate
goal, but it is considered a-would-be-nice-to-have.
Cheers,
David
On 5/20/13 8:53 PM, Dave Townsend wrote:
> On the face of it it looks like it should be possible for Jetpack's
> module loader to load these worker modules. Is that something that seems
> desirable or are these modules not useful outside of workers?
for Jetpack modules that do not depend on DOM or XPCOM and Worker
modules that do not depend on Worker-only code. This is not an immediate
goal, but it is considered a-would-be-nice-to-have.
Cheers,
David
On 5/20/13 8:53 PM, Dave Townsend wrote:
> On the face of it it looks like it should be possible for Jetpack's
> module loader to load these worker modules. Is that something that seems
> desirable or are these modules not useful outside of workers?
Jonas Sicking
May 23, 2013, 7:12:31 PM5/23/13
to David Rajchenbach-Teller, Justin Lebar, dev-platform
On Sat, May 18, 2013 at 3:09 AM, David Rajchenbach-Teller
<dte...@mozilla.com> wrote:
> Hi everyone,
>
> As part of the ongoing effort to make (Chrome) Workers useful for
> platform refactorings, we have been working on a lightweight module
> loader for workers (bug 872421). This loader implements a minimal
> version of CommonJS modules, aka require.js.
>
>
> Example:
>
> // Setup the loader. We need this once per worker.
> importScripts("resource://gre/modules/workers/loader.js");
>
> // Import a few modules
> let Logger = require("resource://gre/modules/workers/logger.js");
> let Storage = require("resource://gre/modules/workers/storage.js");
>
> // ...
> // All values that are not exported are private to the module
> // ...
>
> exports.foo = function() { ... } // Export a value |foo|
> exports.bar = 5; // Export a value |bar|
>
>
>
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
>
<dte...@mozilla.com> wrote:
> Hi everyone,
>
> As part of the ongoing effort to make (Chrome) Workers useful for
> platform refactorings, we have been working on a lightweight module
> loader for workers (bug 872421). This loader implements a minimal
> version of CommonJS modules, aka require.js.
>
>
> Example:
>
> // Setup the loader. We need this once per worker.
> importScripts("resource://gre/modules/workers/loader.js");
>
> // Import a few modules
> let Logger = require("resource://gre/modules/workers/logger.js");
> let Storage = require("resource://gre/modules/workers/storage.js");
>
> // ...
> // All values that are not exported are private to the module
> // ...
>
> exports.foo = function() { ... } // Export a value |foo|
> exports.bar = 5; // Export a value |bar|
>
>
>
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
>
> Given that main thread modules are rooted in
> resource://gre/modules/
> and Jetpack modules are rooted in
> resource://gre/modules/commonjs/
> I would like to place worker modules in
> resource://gre/modules/workers/
>
> Any comments?
My main concern is that Workers created by Gecko are really expensive
> resource://gre/modules/
> and Jetpack modules are rooted in
> resource://gre/modules/commonjs/
> I would like to place worker modules in
> resource://gre/modules/workers/
>
> Any comments?
memory-wise. See the thread started by Justin Lebar titled "Rethinking
the amount of system JS we use in Gecko on B2G".
The short of it is that each Worker requires a separate JS Runtime and
we simply haven't optimize runtimes for having lots of them. This is
especially a problem for B2G where we are very short on memory and
where we are running multiple copies of Gecko.
I would expect the same thing to be an issue on Firefox for Android,
though maybe less so since we're generally running on higher-end
hardware with more memory.
So creating Workers from frontend desktop-only code seems fine. But
it's something that would worry me if we start doing in cross platform
Gecko code.
/ Jonas
David Rajchenbach-Teller
May 24, 2013, 3:39:24 AM5/24/13
to Jonas Sicking, Justin Lebar, dev-platform
Well, if we do not want the main thread to collapse under its weight, we
have to move code off the main thread and to encourage add-ons to do
likewise.
I'm not sure I see an alternative here.
Cheers,
David
On 5/24/13 1:12 AM, Jonas Sicking wrote:
> My main concern is that Workers created by Gecko are really expensive
> memory-wise. See the thread started by Justin Lebar titled "Rethinking
> the amount of system JS we use in Gecko on B2G".
>
> The short of it is that each Worker requires a separate JS Runtime and
> we simply haven't optimize runtimes for having lots of them. This is
> especially a problem for B2G where we are very short on memory and
> where we are running multiple copies of Gecko.
>
> I would expect the same thing to be an issue on Firefox for Android,
> though maybe less so since we're generally running on higher-end
> hardware with more memory.
>
> So creating Workers from frontend desktop-only code seems fine. But
> it's something that would worry me if we start doing in cross platform
> Gecko code.
>
> / Jonas
>
have to move code off the main thread and to encourage add-ons to do
likewise.
I'm not sure I see an alternative here.
Cheers,
David
On 5/24/13 1:12 AM, Jonas Sicking wrote:
> My main concern is that Workers created by Gecko are really expensive
> memory-wise. See the thread started by Justin Lebar titled "Rethinking
> the amount of system JS we use in Gecko on B2G".
>
> The short of it is that each Worker requires a separate JS Runtime and
> we simply haven't optimize runtimes for having lots of them. This is
> especially a problem for B2G where we are very short on memory and
> where we are running multiple copies of Gecko.
>
> I would expect the same thing to be an issue on Firefox for Android,
> though maybe less so since we're generally running on higher-end
> hardware with more memory.
>
> So creating Workers from frontend desktop-only code seems fine. But
> it's something that would worry me if we start doing in cross platform
> Gecko code.
>
> / Jonas
>
Jonas Sicking
May 27, 2013, 1:34:49 PM5/27/13
to David Teller, Justin Lebar, dev-platform
The alternative is to use C++ workers. This doesn't work for addons
obviously, but those aren't yet a concern for B2G.
Weren't we moving addons into separate processes anyway?
/ Jonas
On May 24, 2013 12:39 AM, "David Rajchenbach-Teller" <dte...@mozilla.com>
wrote:
obviously, but those aren't yet a concern for B2G.
Weren't we moving addons into separate processes anyway?
/ Jonas
On May 24, 2013 12:39 AM, "David Rajchenbach-Teller" <dte...@mozilla.com>
wrote:
David Rajchenbach-Teller
May 28, 2013, 8:08:28 AM5/28/13
to Jonas Sicking, Justin Lebar, dev-platform
On 5/27/13 7:34 PM, Jonas Sicking wrote:
> The alternative is to use C++ workers. This doesn't work for addons
> obviously, but those aren't yet a concern for B2G.
Well, my main concern is front-end- and add-on-accessible code.
> The alternative is to use C++ workers. This doesn't work for addons
> obviously, but those aren't yet a concern for B2G.
Normally, it shouldn't influence B2G.
> Weren't we moving addons into separate processes anyway?
Dave Townsend
May 28, 2013, 1:29:34 PM5/28/13
to
On 5/28/2013 5:08 AM, David Rajchenbach-Teller wrote:
> On 5/27/13 7:34 PM, Jonas Sicking wrote:
>> The alternative is to use C++ workers. This doesn't work for addons
>> obviously, but those aren't yet a concern for B2G.
>
> Well, my main concern is front-end- and add-on-accessible code.
> Normally, it shouldn't influence B2G.
>
>> Weren't we moving addons into separate processes anyway?
>
> This has been discussed, but I haven't heard from this since in ages.
It relied on e10s support and so was deprioritised when e10s was.
> On 5/27/13 7:34 PM, Jonas Sicking wrote:
>> The alternative is to use C++ workers. This doesn't work for addons
>> obviously, but those aren't yet a concern for B2G.
>
> Well, my main concern is front-end- and add-on-accessible code.
> Normally, it shouldn't influence B2G.
>
>> Weren't we moving addons into separate processes anyway?
>
> This has been discussed, but I haven't heard from this since in ages.
Rob Campbell
Jul 11, 2013, 9:13:20 AM7/11/13
to dev-platform
On 2013-05-18, at 06:09 , David Rajchenbach-Teller <dte...@mozilla.com> wrote:
> Hi everyone,
>
> As part of the ongoing effort to make (Chrome) Workers useful for
> platform refactorings, we have been working on a lightweight module
> loader for workers (bug 872421). This loader implements a minimal
> version of CommonJS modules, aka require.js.
>
> Example:
>
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
This might kill my dream of having a debuggable worker via require'ing a debug-server.js module.
> Once this loader lands, we will need some convention for where to place
> modules for workers. Unfortunately, main thread modules (both .jsm and
> Jetpack) can generally not be used by worker, due to different module
> semantics, and more importantly due to the fact that most main thread
> modules depend indirectly on XPCOM/XPConnect.
It's certainly feasible to recreate a basic debug-server that conforms to the restrictions of workers though. We'd need a WebSocket-only transport anyway.
> Given that main thread modules are rooted in
> resource://gre/modules/
> and Jetpack modules are rooted in
> resource://gre/modules/commonjs/
> I would like to place worker modules in
> resource://gre/modules/workers/
~ robcee
0 new messages