[modules] Do you think require() should be implemented in JS?
ondras
I would like to hear your opinions and ideas relevant to the fact that
the "require" function (https://wiki.mozilla.org/ServerJS/Modules/
SecurableModules) can be implemented in two ways: as a pure-JS
function (we have already seen first successful implementations), or
in a low-level code (C, C++). Both of these approaches have their
strengths and weaknesses.
Pure-JS solution: we can easily take advantage of the closure-based
system to "remember" module's path. However, to be able to access the
module file, a File() (or similar) is necessary. Also, if we
acknowledge the need for binary datatype, File() will surely depend on
this Binary(). Last, if we want to provide means of loading binary
modules (.so, .dll), our require() function will need some kind of
"dlopen" and "dlsym" exposed into JavaScript world.
To sum this up - JS require cannot be *the* only "extra" thing
available at JS runtime.
Low-level solution: creating require() might be tricky in C, due to
the closure trick used. Also, we won't be able to use one require() in
all JS interpreters, each will have to use its own, low-level require.
However, this way really adds only one foreign object - the require()
function - to the JS world.
Thanks for responses,
Ondrej
Peter Michaux
> I would like to hear your opinions and ideas relevant to the fact that
> the "require" function (https://wiki.mozilla.org/ServerJS/Modules/
> SecurableModules) can be implemented in two ways: as a pure-JS
> function (we have already seen first successful implementations), or
> in a low-level code (C, C++).
It is up to the implementation to decide.
Implementing in C (or Java for Rhino) allows for direct manipulation
of the scope chain and can protect against missing "var".
Peter
Wes Garland
> the closure trick used. Also, we won't be able to use one require() in
> all JS interpreters, each will have to use its own, low-level require.
> However, this way really adds only one foreign object - the require()
> function - to the JS world.
There is no trickiness at all. C variables don't leak into JavaScript
by accident. And if a native module purposefully modifies global (or
anything else outside its scope), there is no way in hell ANY closure
trick will help you.
Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Ondrej Zara
>> the closure trick used. Also, we won't be able to use one require() in
>> all JS interpreters, each will have to use its own, low-level require.
>> However, this way really adds only one foreign object - the require()
>> function - to the JS world.
>
> There is no trickiness at all. C variables don't leak into JavaScript
> by accident. And if a native module purposefully modifies global (or
> anything else outside its scope), there is no way in hell ANY closure
> trick will help you.
creates a special version of "require" for every module loaded. This
way we can persist the module's path to be used in subsequent
(recursive) includes. See
http://github.com/tlrobinson/jack/blob/22b333abc2a9376f2a3527102dad9633bcbd9341/core.js
for more detail.
I believe that *this* particular thing might be tricky in C. I have
never said anything about "leaking C variable into JS" :)))
O.
Wes Garland
> creates a special version of "require" for every module loaded.
Clearly I'm missing something (yes, I looked at your link). Can you
explain in more detail?
Ondrej Zara
>> creates a special version of "require" for every module loaded.
>
> Clearly I'm missing something (yes, I looked at your link). Can you
> explain in more detail?
proper require() method to be passed into every module being loaded
(executed). This way, every module receives its *own* version of
require(), which "remembers" (using a closure) module's path. The
purpose of this solution is that module may recursively call require()
in its code with a relative path - which will be expanded with respect
to parent module's path.
Sample code (moduleA and moduleB are located in the same directory):
--- moduleA.js
exports.moduleA = function() {
require("moduleB").moduleB();
}
--- moduleB.js
exports.moduleB = function() { doStuff(...); }
--- main.js
var a = require("/some/path/to/moduleA");
a();
Tricky stuff happens in moduleA, where the require() call looks for
moduleB relatively to where moduleA is located.
Ondrej
Kris Kowal
> There is no trickiness at all. C variables don't leak into JavaScript
> by accident. And if a native module purposefully modifies global (or
> anything else outside its scope), there is no way in hell ANY closure
> trick will help you.
This is mostly tangential to serverjs, but the thesis Ihab's and my
proposal to TC39 is that all ES really needs (deferring the issue of
syntactic sugar for modules) is a hermetic evaluator so that a
"require" function can create module instances in an air-tight room.
This room would have its own, frozen, tamper-free global object tree,
so module factory functions can be evaluated with this "global" in its
scope chain without risk of communicating malice through mutation of
their common scope. The same can be effectively achieved, as a
stopgap, on the server side by a validator that the client module
loader trusts. The reason we call our proposal to serverjs
"securable" instead of "secure" is that its our hope that "serverjs"
creates a population of modules that are written in a fashion that
would be compatible with a compliant loader implementation that uses
this secured, hermetic evaluator to achieve security on the client
side. It's something we'd encourage prototyping, but is summarily
off-topic for your endeavour.
It's also worth mentioning that a compliant module loader does not
actually need to add anything to the global scope. There's been
mention that "require" would be added to "global", which is not
strictly necessary. A module loader implementation can be constructed
entirely in the privacy of a closure and use its private "require"
sandbox object to execute the initial program module.
Kris Kowal
Kris Kowal
> Sample code (moduleA and moduleB are located in the same directory):
>
> --- moduleA.js
>
> exports.moduleA = function() {
> require("moduleB").moduleB();
> }
>
> --- moduleB.js
>
> exports.moduleB = function() { doStuff(...); }
>
> --- main.js
>
> var a = require("/some/path/to/moduleA");
> a();
>
>
> Tricky stuff happens in moduleA, where the require() call looks for
> moduleB relatively to where moduleA is located.
Excellent explanation. But, a word on the PATH: while a module loader
implementation may or may not use a list of paths to look for modules,
there should definitely be a module root path off which "require"
finds modules by default. I find that its far more common for a
library to import standard libraries off this "root module path" than
to instantiate neighbor modules relative to its own module path. I
recommend that "require" resolve module identifiers off the "root
module path" by default, and only resolve the path relative to its own
module identifier when the first term of the required module is "." or
"..". This is analogous to the new convention supported by Python 2.6
and 3 where modules are "absolute" by default and "relative" if they
begin with a ".". The problem Pythonists ran into here was that you
could not effectively import a standard library from any module that
contained a module by the same name as the standard library module in
the same directory as the importing module. For example:
/usr/lib/python/site-packages/csv.py
/www/csv.py: import csv
/www/foo.py: import csv
In this example, /www/foo.py will get /www/csv.py, and /www/csv.py
will get itself. This is resolved in the new Python:
/usr/lib/python/site-packages/csv.py
/www/csv.py: import csv
/www/foo.py: import .csv
Such that /www/foo.py receives /www/csv.py and /www/csv.py in turn
receives the standard library.
Kris Kowal