[modules] Are we agreed on require, in general?
Kevin Dangoor
freenode) regarding modules and scoping. The big question for module
loading was between these two choices:
1. loading the code in global scope (more or less like a <script> tag)
2. loading the code in a module-specific scope
Peter Michaux had originally been advocating the first style, and put
up the proposals on the modules page for that style:
https://developer.mozilla.org/ServerJS/Modules
He has been convinced that a variation on the "Pythonic Modules"
proposal (just having require) is the best way to go. For reference:
https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
The biggest advantage of #2 is that accidentally leaving off a "var"
will not pollute the global namespace, which is a bigger deal on the
server than on the client because of how much code you may be loading.
And, Peter was convinced that the style did not actually lose any
useful features over the global loading style.
Ignoring syntax details for the moment, can I get a rough idea of
whether people are happy with module-scoped require()?
Kevin
--
Kevin Dangoor
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com
Wes Garland
i.e., what does it do, exactly? Does it attach a specific module to a pre-determined name of the current 'this'? Does that property potentially become a class constructor?
Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Kris Zyp
(And as I mentioned on IRC, this doesn't preclude impls from copying to
the global where appropriate, while still providing the disambiguation
that #2 allows).
Kris
Peter Michaux
> Kevin, can you provide an example of the "require" statement you're talking
> about?
Three libraries and one program...
// one.js
var foo = function(){};
// two.js
var asdf = require('one');
var bar = function() {
asdf.foo(); // works
};
// three.js
var baz = = function() {
asdf.foo(); // error
};
// four.js -- this is the main program started from the command line
// $ js four.js
var a = require('two');
a.bar();
var b = require('three');
b.baz();
Note that even though one.js is loaded by the interpreter before b.baz
is called, asdf.foo() is not available inside three.js. Each file that
wants to use a library function must import from that library.
> i.e., what does it do, exactly? Does it attach a specific module to a
> pre-determined name of the current 'this'?
No. It returns the otherwise unbound "this" object in the file being
imported. That is the "module scope" object.
> Does that property potentially
> become a class constructor?
No. It might have a constructor function as one of it's properties, however.
Peter
Wes Garland
No. It returns the otherwise unbound "this" object in the file being
imported. That is the "module scope" object.
Ah, I see what you're talking about, and it makes perfect sense. That's similar to the implementation I'm working on, but you're thinking more about modules written in JavaScript, whereas I have spent most of my time thinking about modules written in C.
Have you worked out the syntax for providing a class? i.e. do you intend to snag Crockfords "beget()" method for making objects? Or are you going to have something like 'new myModule.constructor()'?
(when I say "you" in this context, I meant he group, not just Peter. :) )
My personal preference (which I admit has caused some implementation snags) is to allow
var xyz = loadModule('moduleName');
var thing = new xyz();
It's particularly nice that way because it allows for easy building of trees containing modules objects and constructors as leaf nodes:
var file = new ca.page.system.File("myfile");
var loadavg = ca.page.system.os.loadavg;
Also, how do you plan to handle the case where many modules require the same other module? This is relatively painless with JavaScript, but can be quite a bit more .. interesting when talking about native-language modules, either those loaded through DSO mechanisms or simple compiled into the embedding.
My current solution is to allow the module to be "loaded" multiple times, but after the first time, instead of loading/initializing the module, I simply return the same moduleObject that was created for the first load. This means that it's possible that the same module will exist by more than one name for a given scope chain, but I don't really see any drawbacks to that.
Wes
Mario Valente
> Today, there were productive discussions on IRC (#serverjs on
> freenode) regarding modules and scoping. The big question for module
> loading was between these two choices:
>
> 1. loading the code in global scope (more or less like a <script> tag)
> 2. loading the code in a module-specific scope
>
> Ignoring syntax details for the moment, can I get a rough idea of
> whether people are happy with module-scoped require()?
>
/me +1 on #2
-- MV
Peter Michaux
> On Sun, Feb 1, 2009 at 6:09 PM, Peter Michaux <peterm...@gmail.com>
> wrote:
>>
>> No. It returns the otherwise unbound "this" object in the file being
>> imported. That is the "module scope" object.
>
> Ah, I see what you're talking about, and it makes perfect sense. That's
> similar to the implementation I'm working on, but you're thinking more about
> modules written in JavaScript, whereas I have spent most of my time thinking
> about modules written in C.
There should be no difference between loading a module written in
C/Java or loading a JavaScript module.
> Have you worked out the syntax for providing a class? i.e. do you intend to
> snag Crockfords "beget()" method for making objects? Or are you going to
> have something like 'new myModule.constructor()'?
>
> (when I say "you" in this context, I meant he group, not just Peter. :) )
I think OOP style is unrelated to module loading in the Pythonic style.
> My personal preference (which I admit has caused some implementation snags)
> is to allow
>
> var xyz = loadModule('moduleName');
> var thing = new xyz();
It would be
// moduleName.js
function xyz() {}
// anotherFile.js
var xyz = require('moduleName').xyz;
var thing = new xyz();
> It's particularly nice that way because it allows for easy building of trees
> containing modules objects and constructors as leaf nodes:
> var file = new ca.page.system.File("myfile");
> var loadavg = ca.page.system.os.loadavg;
>
> Also, how do you plan to handle the case where many modules require the same
> other module?
Every file that wants to use the xyz constructor function would need
to have the following at the top.
var xyz = require('moduleName').xyz;
If you have a common set of modules you like to use together, you
could bundle them up to reduce boiler plate.
// common.js
var abc = require('abc').abc;
var xyz = require('xyz').xyz;
// anotherFile.js
var c = require('common');
var foo = new c.abc();
var bar = new c.xyz();
> My current solution is to allow the module to be "loaded" multiple times,
> but after the first time, instead of loading/initializing the module, I
> simply return the same moduleObject that was created for the first load.
That is how the Helma/Pythonic system works. Modules are "singletons".
> This means that it's possible that the same module will exist by more than
> one name for a given scope chain, but I don't really see any drawbacks to
> that.
I think it could only make things confusing to read.
Peter
Hannes Wallnoefer
> Today, there were productive discussions on IRC (#serverjs on
> freenode) regarding modules and scoping. The big question for module
> loading was between these two choices:
>
> 1. loading the code in global scope (more or less like a <script> tag)
> 2. loading the code in a module-specific scope
>
> Peter Michaux had originally been advocating the first style, and put
> up the proposals on the modules page for that style:
>
> https://developer.mozilla.org/ServerJS/Modules
>
> He has been convinced that a variation on the "Pythonic Modules"
> proposal (just having require) is the best way to go. For reference:
> https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
>
> The biggest advantage of #2 is that accidentally leaving off a "var"
> will not pollute the global namespace, which is a bigger deal on the
> server than on the client because of how much code you may be loading.
> And, Peter was convinced that the style did not actually lose any
> useful features over the global loading style.
projects with more lines of code and developers. The missing var is
just one of many things that can go wrong in such an environment.
As a proof of concept, I implemented require for spidermonkey-based
jslibs (http://code.google.com/p/jslibs/). It's very rough and my
C-skills are lousy, but it seems to work so I'm attaching the patch to
this message. There's also a little demo included, consisting of
main.js and module.js.
Hey, getting broad support for this kind of loading would be
absolutely fantastic! I can really see great stuff happening on top of
it.
Hannes
Chris Zumbrunn
On Feb 1, 2009, at 23:29 , Kevin Dangoor wrote:
> 1. loading the code in global scope (more or less like a <script> tag)
> 2. loading the code in a module-specific scope
+1 for #2, scoped modules
As mentioned on IRC, my preference would be using "modules.foo" as the
syntax instead of "require('foo')" but +1 for scoped modules either way.
Chris
Wes Garland
As a proof of concept, I implemented require for spidermonkey-based
jslibs (http://code.google.com/p/jslibs/). It's very rough and my
C-skills are lousy, but it seems to work so I'm attaching the patch to
this message. There's also a little demo included, consisting of
main.js and module.js.
Any chance you'd be willing to publish your code in its entirety, and under a less restrictive license? I won't work on/with jslibs becase it's GPL only.
Here's a code-inclusion function I wrote last year which wasn't intended for module loading, but has most of the elements required for what you're describing. License is Version: MPL 1.1/GPL 2.0/LGPL 2.1.
Oh, in this example IIRC, the second argument to the function is an object which becomes "this" for the loaded code.
/** Load and interpreter a script in the caller's context */
static JSBool system_include(JSContext *cx, JSObject *obj, uintN argc, jsval *argv, jsval *rval)
{
JSScript *script;
JSObject *scrobj;
JSObject *thisObj;
jsval *fnArg;
const char *scriptFilename;
JSString *scriptFilename_jsstr;
JSBool retval;
switch(argc)
{
default:
return jseng_throw(cx, MODULE_ID ".include.arguments.count");
break;
case 1:
thisObj = ((jseng_interpreter_t *)(JS_GetRuntimePrivate(JS_GetRuntime(cx))))->globalObj;
fnArg = argv + 0;
break;
case 2:
thisObj = JSVAL_TO_OBJECT(argv[0]);
fnArg = argv + 1;
if (JSVAL_IS_OBJECT(argv[0]) != JS_TRUE)
return jseng_throw(cx, MODULE_ID ".include.arguments.0.notObject");
if (JSVAL_IS_NULL(thisObj) == JS_TRUE)
return jseng_throw(cx, MODULE_ID ".include.arguments.0.isNull");
break;
}
scriptFilename_jsstr = JS_ValueToString(cx, *fnArg);
scriptFilename = JS_GetStringBytes(scriptFilename_jsstr);
if (!scriptFilename[0])
return jseng_throw(cx, MODULE_ID ".include.filename: Unable to determine script filename");
errno = 0;
if (access(scriptFilename, F_OK))
return jseng_throw(cx, MODULE_ID ".include.file: %s - %s", scriptFilename, strerror(errno));
JS_AddNamedRoot(cx, &scriptFilename_jsstr, "System.include.scriptFilename_jsstr");
errno = 0;
script = JS_CompileFile(cx, thisObj, scriptFilename);
if (!script)
{
JS_RemoveRoot(cx, &scriptFilename_jsstr);
return jseng_throw(cx, MODULE_ID ".include.compile: Error compiling %s (OS reports %s)", scriptFilename, strerror(errno));
}
JS_RemoveRoot(cx, &scriptFilename_jsstr);
scrobj = JS_NewScriptObject(cx, script);
JS_AddNamedRoot(cx, &scrobj, "include_scrobj");
retval = JS_ExecuteScript(cx, thisObj, script, rval);
JS_RemoveRoot(cx, &scrobj);
return retval;
}
Wes
Peter Michaux
>
>> As a proof of concept, I implemented require for spidermonkey-based
>> jslibs (http://code.google.com/p/jslibs/). It's very rough and my
>> C-skills are lousy, but it seems to work so I'm attaching the patch to
>> this message. There's also a little demo included, consisting of
>> main.js and module.js.
>
> Any chance you'd be willing to publish your code in its entirety, and under
> a less restrictive license? I won't work on/with jslibs becase it's GPL
> only.
>
> Here's a code-inclusion function I wrote
Two examples of the Pythonic style. Could that be a strong indicator
that implementors are behind the Pythonic style?
Peter
Wes Garland
There should be no difference between loading a module written in
C/Java or loading a JavaScript module.
Agreed. I don't think the module user should even have to know how it was implemented.
> Have you worked out the syntax for providing a class? i.e. do you intend toI think OOP style is unrelated to module loading in the Pythonic style.
> snag Crockfords "beget()" method for making objects? Or are you going to
> have something like 'new myModule.constructor()'?
>
> (when I say "you" in this context, I meant he group, not just Peter. :) )
I think so, too, except that it causes implementation differences. It's actually easier to .beget() an object than it is to create non-global classes that are properly recognized by spidermonkey. That said, I am not a fan of beget. And that is an understatement.
> My personal preference (which I admit has caused some implementation snags)It would be
> is to allow
>
> var xyz = loadModule('moduleName');
> var thing = new xyz();
// moduleName.js
function xyz() {}
// anotherFile.js
var xyz = require('moduleName').xyz;
var thing = new xyz();
.....thank you for that example. As long as that is reasonably implementable in Spidermonkey in C, you've sold me. I'm going to try just that this week.
I think it could only make things confusing to read.
You're right. The syntax above is much better.
Wes
Kris Zyp
(pythonic/helma require) is that it shouldn't be specified in terms of
how it should be implemented, but rather in terms of the required
behavior (as any spec should be written). Currently, the proposal states
(perhaps not intentionally) that the module scope should have it's
[[Parent]] set to null, and it's [[Prototype]] set to the global scope.
I would prefer to be able to implement module scopes differently. I
think the spec should state that the module scope object that is
returned from the require should be an object where the properties
correspond to the top-level variables/function created by the evaluation
of the module. It should also state that the global scope should be in
the lookup chain for the evaluation of the module (so that all the
global functions/constructors are available to the modules).
Kris
Wes Garland
I think the spec should state that the module scope object that is
returned from the require should be an object where the properties
correspond to the top-level variables/function created by the evaluation
of the module.
Agreed.
It should also state that the global scope should be in
the lookup chain for the evaluation of the module (so that all the
global functions/constructors are available to the modules).
It's actually not possible to implement that any other way for native modules in Spidermonkey; formalizing that is not a bad idea.
Wes
Hannes Wallnoefer
> One minor recommendation/change I would like to see with #2
> (pythonic/helma require) is that it shouldn't be specified in terms of
> how it should be implemented, but rather in terms of the required
> behavior (as any spec should be written). Currently, the proposal states
> (perhaps not intentionally) that the module scope should have it's
> [[Parent]] set to null, and it's [[Prototype]] set to the global scope.
> I would prefer to be able to implement module scopes differently. I
> think the spec should state that the module scope object that is
> returned from the require should be an object where the properties
> correspond to the top-level variables/function created by the evaluation
> of the module. It should also state that the global scope should be in
> the lookup chain for the evaluation of the module (so that all the
> global functions/constructors are available to the modules).
compatible between implementations, and I'm not sure this is the case
unless module scopes are top level scopes (i.e. their parent scope is
set to null).
I'll update the proposal this evening (central european time) to drop
the include() and import() functions and a few other things that have
been suggested. I'll ping the list when I'm done.
Hannes
mob
collisions?
Is it a reverse domain name? com.corp....?
Just having the module name be a file name doesn't scale -- Or am I
missing something?
Michael
On Feb 2, 8:56 am, Hannes Wallnoefer <hann...@gmail.com> wrote:
> 2009/2/2 Kris Zyp <kris...@gmail.com>:
Peter Michaux
>
> What is the definition of the Module Name? How do you prevent
> collisions?
>
> Is it a reverse domain name? com.corp....?
>
> Just having the module name be a file name doesn't scale -- Or am I
> missing something?
The reason domain names don't collide is because there is a regulating
body. The same is true for Perl modules on CPAN due to Pause.
If a central repository is established for JavaScript modules then the
same system as CPAN/Pause could be used where a regulating body
approves module names. Mozilla has already stated they would host a
repository and it seems like a natural place for a central JavaScript
repository.
Your personal or company modules could be namespaced away from those
on the central repository by using your name or company name or domain
name as part of the module name.
Peter
Stuart Langridge
> on the central repository by using your name or company name or domain
> name as part of the module name.
Or by having them on a separate path. It would be deeply handy to be
able to add folders to the path that require() looks in; I find myself
doing the equivalent in Python (sys.path manipulation) all the time in
a webserver environment. At that point you can do
file=require('file','../mycompany') and get your company's file object
rather than the one on the "standard" require path.
Actually, that's a good point. Probably all of the following could do
with being thought about for require():
1. where does require() look? can it be tweaked from inside code
(rather than from the environment)?
2. are submodules allowed? (require('foo.bar'))?
3. what's the "recommended way" to add my own personal library of
modules? Put them in a separate folder and add that folder to the
require() path? Pass the folder as a second parameter to require()?
Put my whole library of modules in the same folder as the including
file?
I suspect the way to handle my "company library" is to have one
mycompanylibrary.js file which requires all the bits of the library,
put the whole company library in mycompanylibrary/, add
mycompanylibrary/ to the require path, and then
require('mycompanylibrary'), thus giving me
mycompanylibrary.whatever() available.
sil
--
New Year's Day --
everything is in blossom!
I feel about average.
-- Kobayashi Issa
Hannes Wallnoefer
> On Mon, Feb 2, 2009 at 11:59 AM, mob <m...@embedthis.com> wrote:
>
> > What is the definition of the Module Name? How do you prevent
> > collisions?
>
> > Is it a reverse domain name? com.corp....?
>
> > Just having the module name be a file name doesn't scale -- Or am I
> > missing something?
>
> The reason domain names don't collide is because there is a regulating
> body. The same is true for Perl modules on CPAN due to Pause.
leeds to incredibly deep package structures, like
com.company.project.module.package.Class. I think this is overkill,
even for the amount of Java code out there.
I think the sane thing is just to use the project, product or company
name as primary namespace, and then grow deeper as your project
requires. In Helma NG modules are called helma.file, helma.httpclient,
helma.webapp, helma.webapp.response etc.
> If a central repository is established for JavaScript modules then the
> same system as CPAN/Pause could be used where a regulating body
> approves module names. Mozilla has already stated they would host a
> repository and it seems like a natural place for a central JavaScript
> repository.
>
> Your personal or company modules could be namespaced away from those
> on the central repository by using your name or company name or domain
> name as part of the module name.
I've done any Perl coding). They usually consist of 2-3 tokens, and
you get a good idea what they're about.
Hannes
> Peter
mob
different but the design goals seem very similar.
We support a dot notation for module names. Each module can depend on
others and thus forms a recursive dependency tree. Requiring one
module will import it and all dependent modules are loaded but
dependent modules are not introduced into scope implicitly.
To search for modules, we use an EJSPATH env var that specifies a set
of search directories for modules.
Our search strategy is then:
* Given a name "a.b.c", scan for:
*
* 1. File named a.b.c
* 2. File named a/b/c
* 3. File named a.b.c in EJSPATH
* 4. File named a/b/c in EJSPATH
* 4. File named c in EJSPATH
We don't (yet) support versioning in require directive.
mob
If the module name is defined such that it must be unique and be
either centrally registered or a reverse domain name. That allows a
nice namespace for short, memorable names, but still allows internal
use that is guaranteed not to clash.
Michael
Ondrej Zara
> *
> * 1. File named a.b.c
> * 2. File named a/b/c
> * 3. File named a.b.c in EJSPATH
> * 4. File named a/b/c in EJSPATH
> * 4. File named c in EJSPATH
>
> We don't (yet) support versioning in require directive.
I generally agree with the idea of search path, but this seems just a
little bit too much "automagic" to my likings. GNU make comes to mind
with a billion of automatic features which make life sooo
complicated...
Ondrej
Peter Michaux
> 1. where does require() look?
I think how the library path is set should be implementation
dependent. For some shell a environment variable might be good. For
some application like a mod_js, it should be set in the apache config
file.
> can it be tweaked from inside code
> (rather than from the environment)?
I like this idea but I don't think it is critical for the first
version of the spec. It can be added in a backwards compatible way at
a later time.
I've though that "require.path" would be an array of directories that
could be modified during runtime. This doesn't introduce any new
globals and makes sense to be a property of the function doing the
requiring. No unfortunately named $" global variable floating around.
Unfortunately Java requires that the classpath be set when the JVM is
started so it isn't possible to modify the lookup path successfully
for modules which need to load jar files outside the initial
classpath. A custom class loader could be written but that is a big
extra step.
I don't think that the ultimate spec should be driven by what is
difficult to do in a particular ECMAScript implementation. I do think
that the first spec should be easy to implement in all the main
ECMAScript implementations so that conforming projects (shell, web
server, etc) can appear sooner rather than later. This will help with
marketing and momentum.
I also think a "required.loaded" array of loaded module scopes could
be available but as it is not critical it can wait.
Standardizing the minimum at first is a good idea, in my opinion.
> 2. are submodules allowed? (require('foo.bar'))?
Yes but the following should be false
require('foo').bar === require('foo.bar')
The left is the bar property in the foo.js module scope. The right is
the foo/bar.js file's module scope.
It may be better to use something other than '.' as the separator for
this reason. '/' would be fine by me.
> 3. what's the "recommended way" to add my own personal library of
> modules? Put them in a separate folder and add that folder to the
> require() path?
That is my preference.
Eventually your company could set up its own repository and install
the modules with a package management client by specifying multiple
repositories to be searched.
Peter
Hannes Wallnoefer
> > Your personal or company modules could be namespaced away from those
> > on the central repository by using your name or company name or domain
> > name as part of the module name.
>
> Or by having them on a separate path. It would be deeply handy to be
> able to add folders to the path that require() looks in; I find myself
> doing the equivalent in Python (sys.path manipulation) all the time in
> a webserver environment. At that point you can do
> file=require('file','../mycompany') and get your company's file object
> rather than the one on the "standard" require path.
file = require('mycompany.file')
> Actually, that's a good point. Probably all of the following could do
> with being thought about for require():
>
> 1. where does require() look? can it be tweaked from inside code
> (rather than from the environment)?
directories. If you just start the interactive shell without running a
script it's the current working directory and Helma NG's modules
directory. If you run a script, the first element is the directory
parent directory of the script instead of the current working dir.
You can add additional directories to the module search path either
via command line options or from within the application using the
helma.system.addRepository method. So in the case of a company
library, you'd probably put that into a separate directory and add
that to the module path in one way or the other.
> 2. are submodules allowed? (require('foo.bar'))?
your module search path.
> 3. what's the "recommended way" to add my own personal library of
> modules? Put them in a separate folder and add that folder to the
> require() path? Pass the folder as a second parameter to require()?
> Put my whole library of modules in the same folder as the including
> file?
>
> I suspect the way to handle my "company library" is to have one
> mycompanylibrary.js file which requires all the bits of the library,
> put the whole company library in mycompanylibrary/, add
> mycompanylibrary/ to the require path, and then
> require('mycompanylibrary'), thus giving me
> mycompanylibrary.whatever() available.
usually you'd just have a collections of modules, and require/import
just the ones you actually need.
Hannes
Srdjan Pejic
Wes Garland
On a more serious note, it is probably worth considering the model used by the UNIX runtime linker for resolving DSOs. It will
- Check the system default paths
- Check your LD_LIBRARY_PATH (JS_LIBRARY_PATH?)
- Check paths that were compiled into it with -R linker option (that would be an embedding compile time option in our case)
Something to note: It is unlikely that different implementations will have binary compatible native modules, certainly not at first.
Suggestion: module paths include embedding technology at head for implementation-specific modules. This allows more than one ssjs environment to be installed on a given host, and they can share javascript-language modules.
i.e.
/usr/local/ssjs/libexec/spidermonkey/system/file.so
/usr/local/ssjs/libexec/squirrelfish/system/file.so
/usr/local/ssjs/libexec/portable/parser.js
In this setup, two ssjs implementations are using /usr/local/ssjs/libexec in their lib path; parser.js can use the system.file.File object for either squirrelfish or spidermonkey, depending on which environment was using it.
Path searching is particularly important, BTW, when root installs the ssjs environment and plebes want to write their own modules.
Wes
--
Peter Michaux
> We don't (yet) support versioning in require directive.
I think versioning should not be supported in the standard. The idea
is used in Ruby gem and it is a broken idea. Gem has been criticized
for this.
A requires B and C
B requires version 2 of D
C requires version 3 of D
Sometimes this won't matter because the Pythonic system avoids global
namespace collision; however, if module D is dealing with file locks
or database transactions, and truely needs to be a singleton, then all
hell breaks loose and you don't even know it because you didn't even
know that your module A ultimately even used module D. Other folks
wrote B and C.
Peter
Wes Garland
We should be doing "capability testing" whenever possible.
Wes
Srdjan Pejic
Peter Michaux
> You can add additional directories to the module search path either
> via command line options or from within the application using the
> helma.system.addRepository method. So in the case of a company
> library, you'd probably put that into a separate directory and add
> that to the module path in one way or the other.
Does Helma have a custom class loader so the files in the new module
path can use the jar's upon which they depend?
Peter
Wes Garland
Wes
Hannes Wallnoefer
> 2009/2/2 Kris Zyp <kris...@gmail.com>:
> > One minor recommendation/change I would like to see with #2
> > (pythonic/helma require) is that it shouldn't be specified in terms of
> > how it should be implemented, but rather in terms of the required
> > behavior (as any spec should be written). Currently, the proposal states
> > (perhaps not intentionally) that the module scope should have it's
> > [[Parent]] set to null, and it's [[Prototype]] set to the global scope.
> > I would prefer to be able to implement module scopes differently. I
> > think the spec should state that the module scope object that is
> > returned from the require should be an object where the properties
> > correspond to the top-level variables/function created by the evaluation
> > of the module. It should also state that the global scope should be in
> > the lookup chain for the evaluation of the module (so that all the
> > global functions/constructors are available to the modules).
>
> This sounds reasonable. We shoud just make sure the actual outcome is
> compatible between implementations, and I'm not sure this is the case
> unless module scopes are top level scopes (i.e. their parent scope is
> set to null).
>
> I'll update the proposal this evening (central european time) to drop
> the include() and import() functions and a few other things that have
> been suggested. I'll ping the list when I'm done.
leaving just require(). I also rephrased a few other parts, nothing
radical. I think it now better represents what most people can agree
on. Let me know what you think.
https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
Hannes
Peter Michaux
> Ok, I removed the description of include(), import(), and export(),
> leaving just require(). I also rephrased a few other parts, nothing
> radical. I think it now better represents what most people can agree
> on. Let me know what you think.
>
> https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
Can remove the following line at the bottom?
"¹) Helma NG currently only consults the exported properties for
include(), since this is the feature that is most prone to
unintentional scope pollution."
Peter
Hannes Wallnoefer
> On Mon, Feb 2, 2009 at 2:18 PM, Hannes Wallnoefer <han...@gmail.com> wrote:
>
>> Ok, I removed the description of include(), import(), and export(),
>> leaving just require(). I also rephrased a few other parts, nothing
>> radical. I think it now better represents what most people can agree
>> on. Let me know what you think.
>>
>> https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
>
> Can remove the following line at the bottom?
Hannes
Peter Michaux
> Ok, I removed the description of include(), import(), and export(),
> leaving just require(). I also rephrased a few other parts, nothing
> radical. I think it now better represents what most people can agree
> on. Let me know what you think.
>
> https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
I added a list of open issues I've seen arise on the mailing list and irc
https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules#Open_issues
Peter
ihab...@gmail.com
On Mon, Feb 2, 2009 at 2:18 PM, Hannes Wallnoefer <han...@gmail.com> wrote:
> https://developer.mozilla.org/ServerJS/Modules/Pythonic_Modules
I am a member of the Caja group. Caja is an Open Source,
Apache-licensed Google project building a Javascript sanitizer.
As my colleague Mike Samuel pointed out, Kris Kowal (to introduce
himself) and I are working on a proposal to ECMA TC39 (ECMAScript) to
standardize an approach to module systems. Our presentation is here:
http://docs.google.com/Presentation?id=dcd8d5dk_0cs639jg8
and a more extensive document, which is in a bit of flux but goes into
more detail, is:
https://docs.google.com/Edit?tab=view&docid=dfgxb7gk_34gpk37z9v
I have a question about you folks' proposal. One thing we are
concerned about in our ECMA work is the lexical scoping of "imported"
variables. In your proposal, these are simply free variables of the
module code, and are obtained from the prototype chain of the module
scope. In our proposal, the imported variables are available as:
require.env.<name>
so we narrow the module code's free variables to only the following set:
- The name "require";
- The name "exports"; and
- The primordials ("Array", "Object", ...).
This has the following advantages:
1. It makes syntax checking of modules for dangling free references
(e.g., by IDEs) much easier.
2. It allows us to specify a clean desugaring such that the value of
the module is a *function* simply by putting the module author's text
in a function --
(function(require, exports) { <text> })
then eval'ing and returning this. This function can now be
"instantiated" as many times as needed to yield independent copies of
the module. This permits two usages of modules:
2a. Each module name defines a singleton. require("foo") returns that singleton.
2b. Each module name can be resolved to a module function via a
"loader", then the client can instantiate the module as needed to
achieve strict isolation.
This is technically possible with the current serverjs proposal but it
would require some special native magic. We have tended towards
something that is programmable in userland as much as possible and
amenable to simple scope analysis.
What do you folks think?
Ihab
--
Ihab A.B. Awad, Palo Alto, CA
Peter Michaux
> I are working on a proposal to ECMA TC39 (ECMAScript) to
> standardize an approach to module systems. Our presentation is here:
>
> http://docs.google.com/Presentation?id=dcd8d5dk_0cs639jg8
Even that presentation is a lot to absorb in one shot.
> I have a question about you folks' proposal. One thing we are
> concerned about in our ECMA work is the lexical scoping of "imported"
> variables.
Can you explain more what this means exactly. The word "lexical" has
been overloaded and I don't follow exactly your concern in comparison
with the Helma/Pythonic proposal.
----
A concrete example might help me understand. With the Helma/Pythonic
system here are several libraries and a program.
//
// print.js
// An implementation dependent library.
// This example works for Rhino. May be a .so file.
//
var print = function(str) {
java.lang.System.out.println(str);
};
//
// greet-en.js
// A pure JavaScript library
//
var print = require('print').print;
var greet = function(name) {
print('hello, ' + name);
};
//
// greet-sp.js
// Another pure JavaScript library
//
var print = require('print').print;
var greet = function(nombre) {
print('hola, ' + nombre);
};
//
// program.js
//
var greeten = require('greet-en').greet;
var greetes = require('greet-es').greet;
greeten('Ihab'); // hello, Ihab
greetes('Kris'); // hola, Kris
Can you rewrite the above four files how they would be with your proposal?
Peter
ihab...@gmail.com
> Can you explain more what this means exactly. The word "lexical" has
> been overloaded and I don't follow exactly your concern in comparison
> with the Helma/Pythonic proposal.
/* code code code */
function foo() {
/* code code code */
function bar() {
/* code code code */
var z = x + 1;
}
}
Lexical scope analysis is to ask: is "x" a free variable of this block
or not? This is obvious to the extent that the "code code code"
sections are short, which they will typically not be (a module will
likely contain Lots Of Stuff). The alternative is below.
> A concrete example might help me understand. With the Helma/Pythonic
> system here are several libraries and a program.
Thank you, that is a great idea. Here goes.
//
// print.js
// An implementation dependent library.
// This example works for Rhino. May be a .so file.
//
export print = function(str) {
require.env.java.lang.System.out.println(str);
};
[[ Note that the only free variables in the above are "require" and
"export". The latter you don't see because we are using syntactic
sugar, but it's there. ]]
//
// greet-en.js
// A pure JavaScript library
//
var print = require('print').print; /* NOTE: this is *not* re-exported */
export greet = function(name) {
print('hello, ' + name);
};
//
// greet-sp.js
// Another pure JavaScript library
//
var print = require('print').print;
export greet = function(nombre) {
print('hola, ' + nombre);
};
//
// program.js
//
var greeten = require('greet-en').greet;
var greetes = require('greet-es').greet;
greeten('Ihab'); // hello, Ihab
greetes('Kris'); // hola, Kris
Ihab
Peter Michaux
>
> On Mon, Feb 2, 2009 at 5:30 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> Can you explain more what this means exactly. The word "lexical" has
>> been overloaded and I don't follow exactly your concern in comparison
>> with the Helma/Pythonic proposal.
>
> /* code code code */
> function foo() {
> /* code code code */
> function bar() {
> /* code code code */
> var z = x + 1;
> }
> }
>
> Lexical scope analysis is to ask: is "x" a free variable of this block
> or not? This is obvious to the extent that the "code code code"
> sections are short, which they will typically not be (a module will
> likely contain Lots Of Stuff). The alternative is below.
I get lexical analysis for closures and block scope etc. What I don't
get is how your proposal is somehow completely "lexical" or that it is
more lexical than the Helma/Pythonic proposal.
>> A concrete example might help me understand. With the Helma/Pythonic
>> system here are several libraries and a program.
>
> Thank you, that is a great idea. Here goes.
>
> //
> // print.js
> // An implementation dependent library.
> // This example works for Rhino. May be a .so file.
> //
> export print = function(str) {
We cannot do that because we won't be introducing new keywords.
> require.env.java.lang.System.out.println(str);
Why the "env" level in the hierarchy?
I'm not in favor of the dot being used like this for both module
hierarchy and regular property access.
The above line is a bit of an unnecessarily complex example but is
there a file "java/lang/System.js" available to the loader with the
following content?
export out = {
println: function(){}
};
OR is their a file "java/lang/System/out.js" available with the
following content?
export println = function() {};
This is a show stopper ambiguity. Some other way of importing is needed.
> };
>
> [[ Note that the only free variables in the above are "require" and
> "export". The latter you don't see because we are using syntactic
> sugar, but it's there. ]]
>
> //
> // greet-en.js
> // A pure JavaScript library
> //
> var print = require('print').print; /* NOTE: this is *not* re-exported */
> export greet = function(name) {
> print('hello, ' + name);
> };
With the Helma/Pythonic system, the module author can avoid
re-exporting like this...
var greet;
(function() {
var print = require('print').print;
greet = function(name) {
print('hello, ' + name);
};
})();
Peter
Kris Kowal
> > //
> > // print.js
> > // An implementation dependent library.
> > // This example works for Rhino. May be a .so file.
> > //
> > export print = function(str) {
>
> We cannot do that because we won't be introducing new keywords.
transitional syntax that would work right now, and in interpreters
with new uses for the "export" and "import" keywords. That is, we
would just use the syntax that this would desugar to:
exports.print = function (str) {
> > require.env.java.lang.System.out.println(str);
>
> Why the "env" level in the hierarchy?
"print" that are implicitly available in the loader's environment.
It's supposed to be orthogonal with the module name space since
they're conceptually distinct. Ihab meant simply to illustrate that
"java" would preferably not be another "free-variable". So, this
could be a bit of a red herring. Presumably your module loader would
integrate with Java's module's name space so "println" could be
acquired more reasonably:
var println = require("java.lang.System.out").println;
In that case, we DO presume that the corresponding module (perhaps
"java/lang/System/out.js") would look something like this, albeit
presumed auto-generated since it's a reflection of a Java module:
exports.println = function () {
return require.env.java.lang.System.out.println.apply(this,
arguments);
};
Would it be better for us to modify the spec to say that "env" is
another one of the names provided by the module function?
(function (require, env, exports) { <module text> })
- Kris Kowal
Patrick Mueller
> With the Helma/Pythonic system, the module author can avoid
> re-exporting like this...
>
> var greet;
> (function() {
> var print = require('print').print;
> greet = function(name) {
> print('hello, ' + name);
> };
> })();
Using an object inside the module to determine your exports makes this
kind of yuckiness go away, which is nice. For the playing I've done,
I ended up using a variable named "module" which is the module object
returned from (your version of) require(). Anything I want to export,
then, gets added as a property of the "module" variable within the
body of the module code. If there's stuff I don't exported, don't add
it to the "module" variable.
Not sure if I really like that, there is something icky about
'special' variables I tend to not like. But I also dislike just
blanket exporting everything, and requiring "hide it in a function
definition" if I want to hide it.
Other techniques could be used to "hide" otherwise exportable things,
like using a "__" prefix (just like Python).
Patrick Mueller - http://muellerware.org/
Peter Michaux
>
> Just a primer before I dive in for real:
>
>> > //
>> > // print.js
>> > // An implementation dependent library.
>> > // This example works for Rhino. May be a .so file.
>> > //
>> > export print = function(str) {
>>
>> We cannot do that because we won't be introducing new keywords.
>
> Right. And for legacy scripts, we can't either, so we propose a
> transitional syntax that would work right now, and in interpreters
> with new uses for the "export" and "import" keywords. That is, we
> would just use the syntax that this would desugar to:
>
> exports.print = function (str) {
I like the idea that module identifiers are private by default and
must be explicitly exported. Implementationally, this is a minor
deviation from the Helma/Pythonic proposal. Just put an empty
"exports" object on the module scope before evaluating the module in
the module scope. Then a require for the module returns that "exports"
object.
>> > require.env.java.lang.System.out.println(str);
>>
>> Why the "env" level in the hierarchy?
>
> "env" is meant to contain objects like "window", "document", or
> "print" that are implicitly available in the loader's environment.
> It's supposed to be orthogonal with the module name space since
> they're conceptually distinct. Ihab meant simply to illustrate that
> "java" would preferably not be another "free-variable". So, this
> could be a bit of a red herring. Presumably your module loader would
> integrate with Java's module's name space so "println" could be
> acquired more reasonably:
>
> var println = require("java.lang.System.out").println;
No I wouldn't do that. I would write just
"java.lang.System.out.println" as I did in the example and isolate
this platform-dependent code in it's own module and call it from a
platform-independent module. This sort of platform-dependent stuff is
mostly out of scope for this group anyway.
> In that case, we DO presume that the corresponding module (perhaps
> "java/lang/System/out.js") would look something like this, albeit
> presumed auto-generated since it's a reflection of a Java module:
I wouldn't presume auto-generated.
> exports.println = function () {
> return require.env.java.lang.System.out.println.apply(this,
> arguments);
The dot notation is problematic. A module "java.lang.System" with an
exported "out" is confused with a module "java.lang.System.out" with
an exported "println". This sort of ambiguity cannot exist.
This system of dots seems inspired by Java. In Java there is not the
same problem due to lowercase namespace convention. In a Java import
statement a.b.C cannot be confused with a.b.c.D because of
capitalization. In Java each file is a class and classes start with
capital letters. Everything works out ok. It won't workout in
JavaScript because a module will not have just a constructor function
(captialized by convention). A module will potentially just be a
collection of objects with lowercase names and another module file may
have one of those object names as its filename.
> };
>
> Would it be better for us to modify the spec to say that "env" is
> another one of the names provided by the module function?
>
> (function (require, env, exports) { <module text> })
I don't think so. That isn't the main problem. Less free variables is
better. The ambiguity above is a problem.
----
I do like the private-by-default with explicit export as a
modification to the Helma/Pythonic proposal.
Peter
Peter Michaux
>
> On Feb 2, 2009, at 8:59 PM, Peter Michaux wrote:
>
>> With the Helma/Pythonic system, the module author can avoid
>> re-exporting like this...
>>
>> var greet;
>> (function() {
>> var print = require('print').print;
>> greet = function(name) {
>> print('hello, ' + name);
>> };
>> })();
>
> Using an object inside the module to determine your exports makes this
> kind of yuckiness go away, which is nice. For the playing I've done,
> I ended up using a variable named "module" which is the module object
> returned from (your version of) require(). Anything I want to export,
> then, gets added as a property of the "module" variable within the
> body of the module code. If there's stuff I don't exported, don't add
> it to the "module" variable.
I like the name "module" as it is analogous to "global" for the global
scope. I usually have a property called "global" of the global object
(analogous to "window" in the browser) so that my JavaScript isn't
browser-only.
> Not sure if I really like that, there is something icky about
> 'special' variables I tend to not like.
I'm the same.
> But I also dislike just
> blanket exporting everything, and requiring "hide it in a function
> definition" if I want to hide it.
>
> Other techniques could be used to "hide" otherwise exportable things,
> like using a "__" prefix (just like Python).
Using a prefix convention makes sharing code with the browser more
difficult because browsers now wouldn't protect the __ prefixed ones.
Peter
ihab...@gmail.com
On Mon, Feb 2, 2009 at 7:30 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> var println = require("java.lang.System.out").println;
> No I wouldn't do that. I would write just
> "java.lang.System.out.println" ...
but then you said:
> The dot notation is problematic. A module "java.lang.System" with an
> exported "out" is confused with a module "java.lang.System.out" with
> an exported "println". This sort of ambiguity cannot exist.
so I'm confused. The way Kris wrote it, the stuff inside the require()
is a module ID, and the stuff outside is dereferencing the structure
of the instance of the module. It seems like your choice of:
"java.lang.System.out.println"
is what caused this problem. What am I missing?
Peter Michaux
>
> You said:
>
> On Mon, Feb 2, 2009 at 7:30 PM, Peter Michaux <peterm...@gmail.com> wrote:
>>> var println = require("java.lang.System.out").println;
>> No I wouldn't do that. I would write just
>> "java.lang.System.out.println" ...
>
> but then you said:
>
>> The dot notation is problematic. A module "java.lang.System" with an
>> exported "out" is confused with a module "java.lang.System.out" with
>> an exported "println". This sort of ambiguity cannot exist.
>
> so I'm confused.
Sorry. For the specific example of the java package on the Rhino
implementation, I would just do java.lang.System.out.println because
it is available in Rhino and there isn't confusion because of the Java
naming convention where namespaces have lowercase and classes start
uppercase. I wouldn't use require() to load a java package class in
Rhino. What I thought Kris was implying was to somehow make require()
work for both modules and for java packages. I wouldn't overload it
like that. I knew this Java example was going to be confusing.
The important part is the following has trouble
require.A.B.C
as the way to get the identifiers exported in A/B/C.js
It needs to be something else like
require('A.B.C')
so the dots are not confused.
Does that make sense?
Peter
ihab...@gmail.com
On Mon, Feb 2, 2009 at 8:20 PM, Peter Michaux <peterm...@gmail.com> wrote:
> The important part is the following has trouble
> require.A.B.C
> as the way to get the identifiers exported in A/B/C.js
> It needs to be something else like
> require('A.B.C')
> so the dots are not confused.
> Does that make sense?
Absolutely. That is definitely what we propose.
> For the specific example of the java package on the Rhino
> implementation, I would just do java.lang.System.out.println because
> it is available in Rhino and there isn't confusion because of the Java
> naming convention where namespaces have lowercase and classes start
> uppercase. I wouldn't use require() to load a java package class in
> Rhino.
For that specific case, that makes a lot of sense.
That all said, there is a sense in which the module system we propose
is very "hermetic": it assumes that there are no free variables. Like,
really *really* no free variables in a module, apart from the minimum
possible (we have chosen 'require' and 'exports'). In your case,
'java' is a free variable. I don't know how that would play out in our
proposal but, again, this is Rhino and it is clearly a binding to a
specific feature of the embedding in Java....
If I had to do it myself, I would:
require("packages.java.lang.System").println;
Peter Michaux
>
> Remarking in reverse order ;) --
>
> On Mon, Feb 2, 2009 at 8:20 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> The important part is the following has trouble
>> require.A.B.C
>> as the way to get the identifiers exported in A/B/C.js
>> It needs to be something else like
>> require('A.B.C')
>> so the dots are not confused.
>> Does that make sense?
>
> Absolutely. That is definitely what we propose.
Because of the require.env example, I thought you were proposing
require.A.B.C
----
I think the only difference between your proposal and the
Helma/Pythonic proposal is adding the "exports" object. That is a much
smaller difference than the difference between the global eval
proposal and the Helma/Pythonic proposal.
Peter
Kris Kowal
> I think the only difference between your proposal and the
> Helma/Pythonic proposal is adding the "exports" object. That is a much
> smaller difference than the difference between the global eval
> proposal and the Helma/Pythonic proposal.
So, speaking for at least Ihab and myself, I'd like to enumerate our
general objectives:
* interoperability: scripts should work in more environments
* client to server
* insecure to secure
* present to future
* lexical analyzability
* sovereignty: scripts should be able to determine the names that
they import, although adopting names is expected to be the common case
* scalability: no coordination with other developers required to
maintain a single, global API
* verifiability: static tools can identify reference errors
* more difficult to subvert through modifications to the global
scope
* produce smaller optimized forms (YUICompressor variable renaming)
* tells the IDE what names are in scope
So, at this point, I have several nice things to say about the
"exports" object if there are hold-outs for the approach of scraping
the module scope for exportable variables. I'm not sure how much of
this is in question, but I'll give it a go anyway.
One of the questions we've discussed is whether a module ought to
return its exports, produce its exports in its last evaluated
expression, or receive its exports object as a parameter (our current
proposal). The former are behaviorally identical. We went with the
'exports' parameter because it enables us to resolve cyclic
dependencies without infinite recursion and without banning cycles
outright. Consider a module A that depends on B, and a module B that
depends on A. B would require A before A returns its complete
exports. However, if B is willing to require a partially constructed
version of A, using the "exports" object, A can build some of its
exports, then import B. The exports object would be memoized by the
loader before calling the corresponding module factory function, so
infinite recursion does not occur.
The next discussion, which is unresolved (between Ihab and me, at
least), is whether "exports" should be in scope, or a member of the
unique "require" object provided to the module. The "pro" for the
latter strategy is that it reduces the number of messy, implicit
variables in the module scope. The "pro" for the former strategy,
which involves passing the exports object as an argument to the module
function, is that the "exports.foo = ..." syntax closely resembles the
speculated "export foo = ..." syntax that we hope to make available in
future versions of ECMAScript. This is also the reason why we're
favoring "exports" over "module" as the name of this parameter, but
that, like all other parts of our proposal, is still up for
discussion.
As for whether to use an "exports" object versus the local scope of
the module, there are several facets:
- Pro for using the local scope:
* strongly resembles global scope, so migration is easier
- Con for using local scope:
* not interoperable since there's no analogous strategy for
scraping locals in a cross-browser fashion with current JavaScript
implementations.
- Pro for using an "exports" object:
* does not require any masking strategy for "closed" or "private"
data. The module function is like a constructor function since locals
are hidden.
- Con for using an "exports" object:
* consumes "exports" name in module scope, making it implicitly
"special".
Speaking to the point about whether it's possible to scrape local
scope in client-side JavaScript in a cross-browser fashion, the only
strategy I know that can accomplish this takes advantage of a bug in
Safari 2, where "var" declarations inside a "with" block add
attributes to the "with" object. In standards-compliant
implementations of JavaScript, including Safari 3 at least to this
specification point, the declaration aspect of a "var" statement adds
a name to the next function block scope in the scope chain, skipping
all other blocks, including "with" blocks. The initialization aspect
occurs separately and on the original line of the "var" statement,
whether the name refers to the declared object or not. This
particularly eccentric aspect of the specification is the reason why
"let" is encouraged in future versions of JavaScript. I don't recall
how function declaration statements behave in this context.
Meanwhile, I'm told that privileged JavaScript in Gecko engines does
have the ability to scrape its local scope. In either case, the point
is moot. There isn't a known interoperable way to scrape the local
scope.
Kris Kowal
Hannes Wallnoefer
My impression is that your propsal shares one goal with the pythonic
modules proposal, which is script isolation. But in the pythonic scope
scheme, protection is more about accidental collisions and less the
evil attacker I think your hermetic setup is addressing. Also, for me,
two really important goas were simplicity and backwards compatibility.
Simplicity means that I wanted to be able to explain the whole loading
scheme in one or two sentences. So beyond the special setup of the
scope object, it's just an ordinary script being evaluated. At one
time, I considered wrapping a function closure around each module
source, or put all exported properties into a separate object, like
you do in the salty versions of your proposal. But it seemed and still
seems like too much magic going on behind the scenes. I think it's ok
to leave it to the programmer to use closures when public/private
variables are required.
Backwards compatibility means that I wanted to preserve the
functionality of JavaScript as we know it, meaning the global object
and all its standard objects such as Object, Array, Date etc should be
there. I put the global object in the prototype chain, so everything
is there like you expect, but application code is kept apart. So we
can run existing code, and there's no steep relearning curve for
developers. Again, different goals, different result.
One question: are you able to implement this proposal in today's
browsers? I'm aware of Kris' module.js project - is that implementing
it, or going to implement it?
Hannes
2009/2/3 <ihab...@gmail.com>:
Patrick Mueller
Great information in general, thanks.
> As for whether to use an "exports" object versus the local scope of
> the module, there are several facets:
>
> - Pro for using the local scope:
> * strongly resembles global scope, so migration is easier
> - Con for using local scope:
> * not interoperable since there's no analogous strategy for
> scraping locals in a cross-browser fashion with current JavaScript
> implementations.
> - Pro for using an "exports" object:
> * does not require any masking strategy for "closed" or "private"
> data. The module function is like a constructor function since locals
> are hidden.
> - Con for using an "exports" object:
> * consumes "exports" name in module scope, making it implicitly
> "special".
The "Cons" here are the kicker for me. The first, "locals not
scrapeable" is a big deal, so more or less forces an "exports" object
in my mind. The second, "special variable", I'm not terribly happy
about, but isn't a big deal to me either. So an "exports" object wins
in my mind.
Eugene Lazutkin
problem --- it allows to alias package names locally during the
import.
Thanks,
Eugene
On Feb 2, 2:34 pm, Hannes Wallnoefer <hann...@gmail.com> wrote:
> On Feb 2, 9:05 pm, Peter Michaux <petermich...@gmail.com> wrote:
>
> > On Mon, Feb 2, 2009 at 11:59 AM, mob <m...@embedthis.com> wrote:
>
> > > What is the definition of the Module Name? How do you prevent
> > > collisions?
>
> > > Is it a reverse domain name? com.corp....?
>
> > > Just having the module name be a file name doesn't scale -- Or am I
> > > missing something?
>
> > The reason domain names don't collide is because there is a regulating
> > body. The same is true for Perl modules on CPAN due to Pause.
>
> I really hate the reverse domain package naming convention in Java. It
> leeds to incredibly deep package structures, like
> com.company.project.module.package.Class. I think this is overkill,
> even for the amount of Java code out there.
>
> I think the sane thing is just to use the project, product or company
> name as primary namespace, and then grow deeper as your project
> requires. In Helma NG modules are called helma.file, helma.httpclient,
> helma.webapp, helma.webapp.response etc.
>
> > If a central repository is established for JavaScript modules then the
> > same system as CPAN/Pause could be used where a regulating body
> > approves module names. Mozilla has already stated they would host a
> > repository and it seems like a natural place for a central JavaScript
> > repository.
>
> > Your personal or company modules could be namespaced away from those
> > on the central repository by using your name or company name or domain
> > name as part of the module name.
>
> Agreed. I really like CPAN names (although it's a very long time since
> I've done any Perl coding). They usually consist of 2-3 tokens, and
> you get a good idea what they're about.
>
> Hannes
>
> > Peter
Peter Michaux
>
>
>> I think the only difference between your proposal and the
>> Helma/Pythonic proposal is adding the "exports" object. That is a much
>> smaller difference than the difference between the global eval
>> proposal and the Helma/Pythonic proposal.
>
> That's very, very awesome.
Your proposal has no global object, correct?
The Helma/Pythonic proposal has the shared global object on the module
scope's prototype chain.
That would be a big difference.
Peter
Kris Kowal
> Your proposal has no global object, correct?
>
> The Helma/Pythonic proposal has the shared global object on the module
> scope's prototype chain.
>
> That would be a big difference.
>
> Peter
"in the scope chain", or "in the prototype chain" are generalized to
mean "provides names to the lexical scope".
Our proposal says that a security sandbox would have a deeply frozen
global object that only contains standard primordials.
We also say that in order for a module to be interoperable with a
security sandbox, it must not have free variables (missing var's)
apart from names of primordials.
I'm presently suggesting that for the purposes of serverjs, it would
be acceptable for some modules to take advantage of eccentricities of
their platform provided that we acknowledge that they would not work
in a security sandbox.
That being said, Ihab and I are strong proponents of encouraging
modules to be interoperable. To that end, we recommend that
environments agree to place and take cross-platform capabilities with
the "require.env" object.
This is to say that we have several degrees of specification that
we're working on.
1. ask ECMA TC39 to add tools to the ECMAScript standard such that
native client-side security sandboxes can be created with pure
JavaScript.
2. define how module systems must be implemented in order to provide
security assertions.
3. define how modules ought to be written such that they are
interoperable, including secure, insecure, client-side, server-side,
current, and future modules.
4. ask ECMA TC39 to give us some sugar with the hope that ten years
out, certain import and export patterns are more expressive.
Our present discussion is entirely about 3, how to write modules,
which is modestly informed by 2, the constraints on module system
implementations in security environments.
So, the parts of our proposal that are relevant to this discussion are
the syntax we recommend for importing and exporting.
1. a "require" function that accepts absolute or relative module
identifiers and returns objects that contain module exports.
2. an "exports" object, (which we also propose be bound to "this"),
that the module augments as it's interpreted to expose its API.
3. the guarantee that "var" and "function" declarations are private
to the module file.
4. an "env" object, currently proposed to be "require.env", that
contains the limited set of voodoo powers bestowed upon the module and
it's dependencies.
5. that the global object is only guaranteed to contain primordials
and may be deeply frozen.
The fourth and fifth points address interoperability with secure
sandboxes. If a module intends to work inside a sandbox, it really
has to use such objects at least as if they were a deeply frozen in
the assumption that they would be in some platforms, eventually. We
strongly encourage this practice. We also make no bones about how
these requirements are supported by the module system, but we do know
that there's at least one way to do it. We also have some thoughts
about how a sandbox can instantiate a new sandbox with an unprivileged
or adapted environment, and how module factory functions can be shared
by multiple sandboxes without leaking.
Beyond our specification, you guys would still need to reach an accord
about:
1. what's in the interoperable environment
2. what is the grammar for module identifiers
We intend to leave these options open to the curators of libraries and
package managers.
Kris
Peter Michaux
which will help the process move along.
On Tue, Feb 3, 2009 at 12:06 PM, Kris Kowal <cowber...@gmail.com> wrote:
> That being said, Ihab and I are strong proponents of encouraging
> modules to be interoperable. To that end, we recommend that
> environments agree to place and take cross-platform capabilities with
> the "require.env" object.
I don't think it is necessary to agree on this. The platform-dependent
code is a layer below the standardized API that this project seeks.
> 1. a "require" function that accepts absolute or relative module
> identifiers and returns objects that contain module exports.
How a particular domain of applications interprets the string sent to
"require" would be application dependent. For example, serverjs may
standardize require('a.b.c') and those are resolved relative to the
path directories.
> 2. an "exports" object, (which we also propose be bound to "this"),
> that the module augments as it's interpreted to expose its API.
I don't think mandating its binding it to "this" is necessary.
> 3. the guarantee that "var" and "function" declarations are private
> to the module file.
yes
> 4. an "env" object, currently proposed to be "require.env", that
> contains the limited set of voodoo powers bestowed upon the module and
> it's dependencies.
I don't think that is necessary for this project to agree about this.
Everything can be loaded through "require" including "env".
var env = require('env');
> 5. that the global object is only guaranteed to contain primordials
> and may be deeply frozen.
The global object must also have the "require" function, or at least
it should appear to be global just like the "exports" object is
already there when the module starts executing. since it is not
possible to do
var require = require('require').require;
Whether or not "require" is actually global or just appears in the
module scope could be up to the implementation.
----
I think your proposal is a generalized version of the helma/pythonic
proposal with the addition of the exports object (and private by
default). Also a standard library should not be allowed to mutate the
global object.
Perhaps you can write up a proposal for the wiki? There would be no
need to describe all the various environments in which the code would
be able to run etc. Just the minimal subset of your grand proposal
that would matter for the writing of the serverjs standard library.
https://developer.mozilla.org/ServerJS/Modules
Peter
ihab...@gmail.com
>> 4. an "env" object, currently proposed to be "require.env", that
>> contains the limited set of voodoo powers bestowed upon the module and
>> it's dependencies.
>
> I don't think that is necessary for this project to agree about this.
> Everything can be loaded through "require" including "env".
>
> var env = require('env');
We considered this and, so far, we have rejected it. The reason is that:
a. We do not wish to conflate the namespace of modules identifiers
with the namespace of environment names provided to the module;
because
b. The specification of the environment and the specification of the
module identifier space are orthogonal concerns; and in practice
c. The module identifier resolution and the resolution of environment
names are properly the responsibility of separate objects in the
system, and this separation allows sandboxes to be constructed
piecewise (e.g., mixing and matching module identifier conventions
within the same sandbox), which is important for interoperability.
Peter Michaux
>
> On Tue, Feb 3, 2009 at 1:00 PM, Peter Michaux <peterm...@gmail.com> wrote:
>>> 4. an "env" object, currently proposed to be "require.env", that
>>> contains the limited set of voodoo powers bestowed upon the module and
>>> it's dependencies.
>>
>> I don't think that is necessary for this project to agree about this.
>> Everything can be loaded through "require" including "env".
>>
>> var env = require('env');
>
> We considered this and, so far, we have rejected it. The reason is that:
>
> a. We do not wish to conflate the namespace of modules identifiers
> with the namespace of environment names provided to the module;
> because
I don't think of the environment and modules as different things.
Anything that needs to be exposed by the host can be part of the env
module.
> b. The specification of the environment and the specification of the
> module identifier space are orthogonal concerns; and in practice
I don't see them as orthogonal. To the programmer, both expose
functionality, objects, etc.
> c. The module identifier resolution and the resolution of environment
> names are properly the responsibility of separate objects in the
> system, and this separation allows sandboxes to be constructed
> piecewise (e.g., mixing and matching module identifier conventions
> within the same sandbox), which is important for interoperability.
A sandbox's require could be overloaded and treat require('env') specially.
Peter
Kris Kowal
> On Tue, Feb 3, 2009 at 12:06 PM, Kris Kowal <cowber...@gmail.com> wrote:
> How a particular domain of applications interprets the string sent to
> "require" would be application dependent. For example, serverjs may
> standardize require('a.b.c') and those are resolved relative to the
> path directories.
Agreed.
>> 2. an "exports" object, (which we also propose be bound to "this"),
>> that the module augments as it's interpreted to expose its API.
>
> I don't think mandating its binding it to "this" is necessary.
Binding to "this" does provide another layer of opportunity for
transitional syntax. That is, by introducing a "this" variable, some
scripts can be written in such a way that they would work both as
global scripts and as modules conforming to our specification. My
canonical example is jQuery, which has about four references to
"window", two of which are used to grab window capabilities, but two
others are used to bind the jQuery to its effective exports. If we
provide "this" as a synonym for "exports", we provide an incremental
migration path. jQuery could change its two references to "window"
when it means "exports" to "this" and it would continue to work in
both a module system and global script. This is very likely to be a
messy but necessary migration step for the larger extant libraries.
> The global object must also have the "require" function, or at least
> it should appear to be global just like the "exports" object is
> already there when the module starts executing. since it is not
> possible to do
> Whether or not "require" is actually global or just appears in the
> module scope could be up to the implementation.
I agree that "require" does not have to be global, and disagree that
it must be global. It's provided to modules as an argument to the
module factory function. The natural question is how to import your
initial module. That is one of those things we don't have to agree
about, I think. In Chiron, the initial module is specified on the
<script src="modules.js?./initialModule.js"> tag URL, and the <script>
tag is removed from the DOM before its mention can leak to latter
loaded scripts. So, there's really no way to even detect its
presence, much less access it via the transitive globals. In V8, I
imagine that it would be an argument to a "js" shell (this is
something I have played with).
> Perhaps you can write up a proposal for the wiki? There would be no
> need to describe all the various environments in which the code would
> be able to run etc. Just the minimal subset of your grand proposal
> that would matter for the writing of the serverjs standard library.
>
> https://developer.mozilla.org/ServerJS/Modules
Sure. I'll let you guys know when it's up.
Kris Kowal
Kris Kowal
> Peter:
>> Perhaps you can write up a proposal for the wiki? There would be no
>> need to describe all the various environments in which the code would
>> be able to run etc. Just the minimal subset of your grand proposal
>> that would matter for the writing of the serverjs standard library.
>>
>> https://developer.mozilla.org/ServerJS/Modules
>
> Sure. I'll let you guys know when it's up.
>
It's up at https://developer.mozilla.org/ServerJS/Modules/Securable_Modules
Kris Kowal
Wes Garland
I submit that before a final decision is made, some sample code showing two sample inter-dependent modules, as well as a module which provides a class constructor, be made available.
The problem with the "require" syntax in my head is that modules can't module-globally depend on a module that depends on them, or there are problems loading -- X loads Y which loads X which is still loading -- big problem.
Wes
--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Kris Kowal
> module-globally depend on a module that depends on them, or there are
> problems loading -- X loads Y which loads X which is still loading -- big
> problem.
>
> Wes
You're right that mutual dependency can be a problem. The trick is
that module loaders are required to prememoize the exports object to
break cyclic recursion, which also means that partially constructed
exports are available to cyclic dependencies. The order of exports
and requires becomes important. I do run into this in Chiron.
module X:
exports.foo = function () {};
var {bar} = require(".Y");
bar();
module Y:
var {foo} = require(".Y");
exports.bar = function () {};
foo();
Kris
Kris Kowal
On Feb 3, 4:41 am, Hannes Wallnoefer <hann...@gmail.com> wrote:
> One question: are you able to implement this proposal in today's
> browsers? I'm aware of Kris' module.js project - is that implementing
> it, or going to implement it?
>
> Hannes
https://developer.mozilla.org/ServerJS/Modules/Securable_Modules
However, it is very similar. I do support the "require" and "exports"
objects. A lot of the modules I've ported to Chiron will require
modification since I've used "from module import *" semantics (the
"include" function we have both redacted from our requirements) pretty
extensively. I intend to revisit modules.js when the dust settles and
implement sandboxing. My type system will probably require extensions
to this module system to support foreign module currying and stuff
like that. TC39 brought up the point that it would be desirable to
check whether an object is an instance of an exported type from a
module that may not have been loaded, and ought not be loaded. In any
case, I do intend to upgrade modules.js to support transitional
syntax. It's my hope that platform agnostic modules (hashing,
encoding, encryption, base types, events and observers, and such) will
be usable by serverjs.
Kris
Peter Michaux
>
> Peter:
>> Perhaps you can write up a proposal for the wiki? There would be no
>> need to describe all the various environments in which the code would
>> be able to run etc. Just the minimal subset of your grand proposal
>> that would matter for the writing of the serverjs standard library.
> It's up at https://developer.mozilla.org/ServerJS/Modules/Securable_Modules
Thanks.
I like the idea that "require" is just there in the module somehow and
doesn't need to be specified as global or not. Does it need to be
specified that the actual require function object available to modules
is === the same object no matter which module? I think it should be
=== the same object but maybe it doesn't matter in the spec.
I like there is no need for a "global" identifier as good library code
shouldn't need access. There is probably still a way to get a
reference to it somehow.
I like that exported symbols have to be white listed explicitly by
augmenting the exports object. As we've seen in browser scripting,
there is a lot of boilerplate code to hide variables in closures. Many
developers want that kind of encapsulation for sanity sake. Surely
server-side JavaScript would go in the same direction.
Quotations below are from the above wiki page.
> The "require" function may be called with an absolute or relative module identifier.
I think "may" should be "is". There is no need to mention absolute or
relative as together they are all the possible identifiers. The
concepts of "absolute" and "relative" may not even be meaningful in
some environments. The argument must be a string? That would lead it
to be...
The "require" function is called with an string which identifies the module.
The official ES3 language in Date.parse for the string argument is
The parse function applies the ToString operator to its argument
> It will receive an object containing the exported API of the foreign module.
"receive" should be "return"?
----
> its transitive dependencies
Why transitive? Dependencies are dependencies, aren't they?
----
> the object returned by "require" must contain at least the exports
> that the foreign module has prepared before the call to require
> that led to the current module's execution.
What is the justification for this "must"? What does it guarantee the
programmer in a practical sense?
Could say something like "the exports object is live". The exports
object of a module might continue to change throughout the whole
program execution.
----
> A module receives an "exports" object, synonymous with "this"
> in the top scope of the module, that it may add its exported API
> to as it executes.
I don't find the argument of a transitional coding style for binding
exports with "this" very compelling. I don't think it is necessary to
have both and I wouldn't want to read code that augments to "this" as
its way of exporting. We know that "this" is a problematic part of
JavaScript. I'd rather avoid it and avoid any problems that might be
associated that no one has though of yet. The designers of ECMAScript
were smart folks and did not foresee the many problems with "this".
I'd rather avoid repeating that history.
----
> A module receives a "require.env" object that contains objects
> provided by the module "sandbox" for communication with
> host environment.
There is no need for the require.env object. What can be provided by
the require.env object that cannot be imported?
The environment is not orthogonal to modules as it can be made
available as a module:
// env.js
export.env = require.env;
// some lib
var env = require('env').env;
From the programmers point of view, what the require.env object
provides are some objects, functions, arrays, etc, just like a module
does. Uniformity should be present in how the programmer accesses to
those things.
Peter
Peter Michaux
On Feb 3, 3:03 pm, Wes Garland <w...@page.ca> wrote:
> Between Peter and Kris, I'm starting to think that the "require" syntax has
> problems that the "export" syntax doesn't, in particular, for the case when
> there are inter-dependent modules.
regard to the cyclic dependency issues.
The only real difference is in the need for explicit hiding or
explicit exporting.
> I submit that before a final decision is made, some sample code showing two
> sample inter-dependent modules, as well as a module which provides a class
> constructor, be made available.
proposal has been discussed enough.
I'd suggest that examples of sharing code with the browser are also
important.
This decision is fundamental and cannot be made casually.
Peter
Kris Zyp
Peter Michaux wrote:
> [snip]
> The only real difference is in the need for explicit hiding or
> explicit exporting.
>
>
>
If I understand properly the question is how to differentiate between
hidden variables and exported ones. Have we considered differentiating
bases simply on whether variables are scoped? Scoped variables would be
hidden and those written as free variables (parent-level scope from the
module view) would be exported. For example:
module:
var myPrivateFunction = function(){ ... }
myExportedFunction = function(){ ... }
using the module:
require("module").myExportedFunction()
require("module").myPrivateFunction -> undefined
This could be done/described by a variation on the Helma scheme:
require = function(){
moduleExportedScope = {__proto__:global};
modulePrivateScope = {__parent__:moduleExportedScope};
load using modulePrivateScope
return moduleExportedScope ;
}
And just for differentiation the current proposals:
Helma/Pythonic:
module:
(function(){
var myPrivateFunction = function(){ ... }
})();
myPublicFunction = function(){ ... }
Securable Module:
var myPrivateFunction = function(){ ... }
exports.myExportedFunction = function(){ ... }
None of this module formats differ in their ability to express hidden
and exported functions, they just differ in their syntax. I would argue
that this proposal is the easiest and most intuitive.
Kris
Kris Kowal
> On Tue, Feb 3, 2009 at 3:02 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> Peter:
>>> Perhaps you can write up a proposal for the wiki? There would be no
>>> need to describe all the various environments in which the code would
>>> be able to run etc. Just the minimal subset of your grand proposal
>>> that would matter for the writing of the serverjs standard library.
>
>> It's up at https://developer.mozilla.org/ServerJS/Modules/Securable_Modules
>
> doesn't need to be specified as global or not. Does it need to be
> specified that the actual require function object available to modules
> is === the same object no matter which module? I think it should be
> === the same object but maybe it doesn't matter in the spec.
If you can make === work, that's cool by the spec, but the spec
deliberately avoids providing that guarantee. If a module loader opts
to support relative module identifier resolution, at least in pure JS,
it will be hard to do this without constructing a unique "require"
object that closes on the current module's id.
> Quotations below are from the above wiki page.
>> The "require" function may be called with an absolute or relative module identifier.
>
> I think "may" should be "is". There is no need to mention absolute or
> relative as together they are all the possible identifiers. The
> concepts of "absolute" and "relative" may not even be meaningful in
> some environments.
I've removed the qualifier that absolute and relative module
identifiers must be accepted by require. I've left the "may be"
qualifier since a module may not have dependencies.
> The argument must be a string? That would lead it
> to be...
>
> The "require" function is called with an string which identifies the module.
Ihab and I don't intend to constrain the domain of module identifiers
for particular groups of modules. For the purposes of "serverjs", you
will obviously need to agree on a module identifier domain. We're not
interested in preventing particular loaders from, for example,
accepting object literals with version numbers in addition to strings,
if they can prove their system to be secure.
>> It will receive an object containing the exported API of the foreign module.
> "receive" should be "return"?
The caller receives, the callee returns.
> Why transitive? Dependencies are dependencies, aren't they?
A dependency is a module that you directly require. To be a
transitive dependency, a module must be one of:
basis.) required by the module
recur.) required by a transitive dependency
That is, you have immediate children, and then you have descendants.
>> the object returned by "require" must contain at least the exports
>> that the foreign module has prepared before the call to require
>> that led to the current module's execution.
>
> What is the justification for this "must"? What does it guarantee the
> programmer in a practical sense?
It guarantees the programmer that, if they have prepared their exports
and imports in a particular fashion, they are guaranteed that they can
resolve cyclically dependent modules. This requirement imposes the
restriction on module systems that they must pre-memo the module
exports object before they call the module factory function.
> Could say something like "the exports object is live". The exports
> object of a module might continue to change throughout the whole
> program execution.
The exports object may be "alive", but we're not prepared to impose
this requirement on the module system. In a secure system, the
exports returned by "require" have to be frozen, which means that they
must either be a snapshot or begotten. The "begotten" paradigm would
support the "live" behavior you're suggesting, which most closely
emulates the trivial case of promiscuously passing the exports object
around, but might also be problematic to secure. We don't know yet.
> I don't find the argument of a transitional coding style for binding
> exports with "this" very compelling. I don't think it is necessary to
> have both and I wouldn't want to read code that augments to "this" as
> its way of exporting. We know that "this" is a problematic part of
> JavaScript. I'd rather avoid it and avoid any problems that might be
> associated that no one has though of yet. The designers of ECMAScript
> were smart folks and did not foresee the many problems with "this".
> I'd rather avoid repeating that history.
It might be a good idea to impose the additional requirement that
"this" not be used in your particular situation, as long as we're
aware that not providing "this" as "exports" is to demand that all
major libraries provide forks to support the module system.
>> A module receives a "require.env" object that contains objects
>> provided by the module "sandbox" for communication with
>> host environment.
>
> There is no need for the require.env object. What can be provided by
> the require.env object that cannot be imported?
You're right that dependency injection can be done by priming the
module exports memo of a sandbox, and I've made this argument
extensively myself, also being a proponent of simplicity. However,
from the perspective of security programmers, separating these name
spaces makes it easier to reason about security. The Caja team's goal
is to make module functions completely capability neutral, so that
modules cannot be a vector through which vulnerability can propagate.
The only way to be vulnerable to a system of modules is to explicitly
white-list certain capabilities to its sandbox, through the "env"
variable.
Summarily: It's much harder to separate modules from the environment
than it is to combine them.
You've also pointed out that it's trivial to break the orthogonality
between a module and an environment. I agree, and for this reason,
I'm content to go along with the partitioning of the environment from
the module name spaces since it's outright easy to provide an adapter
module to host the environment. From the common programmer's
perspective, it is unlikely that you'll need to interact with the
environment directly. I do not consider this issue paramount.
Kris
Kris Kowal
that all objects that are exported are also implicitly in scope? This
would enable a certain additional degree of conciseness, perhaps at an
unmentionable price.
exports.foo = function () {};
exports.bar = function () {
return foo();
};
Kris Kowal
Kris Kowal
> On Tue, Feb 3, 2009 at 3:02 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> It will receive an object containing the exported API of the foreign module.
> "receive" should be "return"?
On a closer look, rephrasing this sentence in terms of "return" does
improve the readability. Thanks.
Kris Kowal
Peter Michaux
That seems like a good idea to add to your proposal.
Peter
Peter Michaux
>
> On Tue, Feb 3, 2009 at 4:46 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> On Tue, Feb 3, 2009 at 3:02 PM, Kris Kowal <cowber...@gmail.com> wrote:
>>> Peter:
>> Quotations below are from the above wiki page.
>>> The "require" function may be called with an absolute or relative module identifier.
>>
>> I think "may" should be "is". There is no need to mention absolute or
>> relative as together they are all the possible identifiers. The
>> concepts of "absolute" and "relative" may not even be meaningful in
>> some environments.
>
> I've removed the qualifier that absolute and relative module
> identifiers must be accepted by require. I've left the "may be"
> qualifier since a module may not have dependencies.
I read it as "require" can be called with no argument at all.
>> Why transitive? Dependencies are dependencies, aren't they?
>
> A dependency is a module that you directly require. To be a
> transitive dependency, a module must be one of:
> basis.) required by the module
> recur.) required by a transitive dependency
>
> That is, you have immediate children, and then you have descendants.
Oops. I read "transient".
>> I don't find the argument of a transitional coding style for binding
>> exports with "this" very compelling. I don't think it is necessary to
>> have both and I wouldn't want to read code that augments to "this" as
>> its way of exporting. We know that "this" is a problematic part of
>> JavaScript. I'd rather avoid it and avoid any problems that might be
>> associated that no one has though of yet. The designers of ECMAScript
>> were smart folks and did not foresee the many problems with "this".
>> I'd rather avoid repeating that history.
>
> It might be a good idea to impose the additional requirement that
> "this" not be used in your particular situation, as long as we're
> aware that not providing "this" as "exports" is to demand that all
> major libraries provide forks to support the module system.
I think "this" should be the module scope.
>>> A module receives a "require.env" object that contains objects
>>> provided by the module "sandbox" for communication with
>>> host environment.
>>
>> There is no need for the require.env object. What can be provided by
>> the require.env object that cannot be imported?
>
> You're right that dependency injection can be done by priming the
> module exports memo of a sandbox, and I've made this argument
> extensively myself, also being a proponent of simplicity. However,
> from the perspective of security programmers, separating these name
> spaces makes it easier to reason about security. The Caja team's goal
> is to make module functions completely capability neutral, so that
> modules cannot be a vector through which vulnerability can propagate.
> The only way to be vulnerable to a system of modules is to explicitly
> white-list certain capabilities to its sandbox, through the "env"
> variable.
I think that require.env could be left out of your proposal to this
group. That doesn't prevent it to exist in other environments. I can't
think of a need for that to exist for serverjs standardized API. It
only causes confusion about what should be a module and what should be
on the env object. For example, where should a File object go? In a
module or request.env.File. It is not at all clear. The fact that
require.env could be left out and won't harm the goals of this group
indicates it should be left out.
Peter
Peter Michaux
>
>
>
> Peter Michaux wrote:
>> [snip]
>> The only real difference is in the need for explicit hiding or
>> explicit exporting.
>>
>>
>>
> If I understand properly the question is how to differentiate between
> hidden variables and exported ones. Have we considered differentiating
> bases simply on whether variables are scoped? Scoped variables would be
> hidden and those written as free variables (parent-level scope from the
> module view) would be exported. For example:
> module:
> var myPrivateFunction = function(){ ... }
> myExportedFunction = function(){ ... }
>
> using the module:
> require("module").myExportedFunction()
> require("module").myPrivateFunction -> undefined
This means forgetting "var" causes a variable to be exported. That
doesn't seem good.
Peter
Kris Kowal
> hidden variables and exported ones. Have we considered differentiating
> bases simply on whether variables are scoped? Scoped variables would be
> hidden and those written as free variables (parent-level scope from the
> module view) would be exported. For example:
> module:
> var myPrivateFunction = function(){ ... }
> myExportedFunction = function(){ ... }
Is there a way to capture the exports in pure JavaScript? If I'm not
mistaken, this would take advantage of the rule that free variable
assignment is attributed to the global object, which we can't supplant
on the client. If there's no way to do this in both environments,
interoperability might be needlessly diminished. I do like that this
proposal implicitly binds the name in lexical scope in addition to
exports.
Kris Kowal
Mario Valente
> Kris Kowal:
>> Peter:
>>> Perhaps you can write up a proposal for the wiki? There would be no
>>> need to describe all the various environments in which the code would
>>> be able to run etc. Just the minimal subset of your grand proposal
>>> that would matter for the writing of the serverjs standard library.
>>>
>>> https://developer.mozilla.org/ServerJS/Modules
>> Sure. I'll let you guys know when it's up.
>>
>
Trying to understand the "modules" issue (I would just copy
what Python does a be done with it...) I found this reference
which might come in handy
A comparison of module constructs in programming languages
http://www.oberon2005.ru/paper/fc1991-01e.pdf
-- MV
Patrick Mueller
I remember implementing this exact behavior in one of my experiments
(over a year ago) with Rhino. Made a lot of sense to me at the time.
But it's confusing. People will think, looking at the code, that
those are two different foo's, or if they do know they're the same,
make think this == exports.
Peter Michaux
> On Tue, Feb 3, 2009 at 3:02 PM, Kris Kowal <cowber...@gmail.com> wrote:
>> A module receives an "exports" object, synonymous with "this"
>> in the top scope of the module, that it may add its exported API
>> to as it executes.
>
> I don't find the argument of a transitional coding style for binding
> exports with "this" very compelling. I don't think it is necessary to
> have both and I wouldn't want to read code that augments to "this" as
> its way of exporting. We know that "this" is a problematic part of
> JavaScript. I'd rather avoid it and avoid any problems that might be
> associated that no one has though of yet. The designers of ECMAScript
> were smart folks and did not foresee the many problems with "this".
> I'd rather avoid repeating that history.
For example...
ES3 can cause a leak if the programmer is mistaken about what the
following means
Foo.prototype.bar = function(a) {
function baz(a) {
this.a = a;
};
baz(a);
};
I believe the actual behavior, where "this" is the global object, is
considered a spec bug and is being "corrected" in ES3.1. Personally, I
think it is not a spec bug because baz is called without a receiver
and so of course "this" is the global object. Anyway it is true and
the situation is a subtle one causing unfortunate program errors due
to misunderstanding. When the misunderstanding results in a program
bug there is a leak.
Now related to the module proposal and the fact that we must target
pre ES3.1 implementations, if "this" is bound to "exports" in the
above example then the value of "a" is exported. This is an unintended
leak due to programmer error but certainly doesn't seem to fit into
the requirements of securing the module and helping programmers
maintain their encapsulation. This opens a hole due to confusion about
the subtle behavior.
If on the other hand the programmer wanted "a" exported in the above
situation and "this" is not bound to "exports" then he would need to
write "exports.a". There would be no confusion at all of his
intention.
The main point is "this" is confusing in JavaScript. I think binding
"this" to "exports" is a recipe for trouble and that is unfortunate
since doing so is not necessary.
Peter
Kris Kowal
"this == exports" at a later date if it becomes more generally
apparent that they are in fact necessary for security and incremental
migration respectively. Ihab and I, for the time being, are in
agreement that if all loaders support this strict subset of the
proposal, it will be a win for everyone in the long run.
Kris Kowal
Peter Michaux
>
> I've pared down the Secure Module spec. We can add "require.env" and
> "this == exports"
I still see this == exports in the proposal and require.env is
mentioned at the bottom.
Since the proposal is for servers project in particular, is it ok with
you if others edit your proposal when defining the module identifier
domain is discussed?
Peter
Kris Kowal
> Since the proposal is for servers project in particular, is it ok with
> you if others edit your proposal when defining the module identifier
> domain is discussed?
I've fixed the noted missing edits. Naturally, the wiki's all yours:
https://developer.mozilla.org/ServerJS/Modules/Securable_Modules
Kris Kowal
Peter Michaux
After discussion on IRC this seemed to be non-essential for writing
standard libraries and so has been removed. This requirement also
complicates pure JavaScript implementations by requiring the use of
"with" which is non-strict in ES3.1. An implementation is free to add
this feature but standard libraries don't need it.
Peter
Kris Kowal
> standard libraries and so has been removed. This requirement also
> complicates pure JavaScript implementations by requiring the use of
> "with" which is non-strict in ES3.1. An implementation is free to add
> this feature but standard libraries don't need it.
To be clear, while I'm personally torn on the issue, we're in
agreement. But with the revised specification, it must be pervasively
understood that while a module loader might be free to add a "with
(exports)" block in the module factory function, any module that takes
advantage of it would no longer be interoperable. This means that any
module wanting to have their cake and eat it, must use this syntax:
var foo = exports.foo = …;
Or, explicate "exports.foo" in all internal uses.
I consider the revised specification sufficient, unless there's a riot
about whether this syntax is acceptable.
Kris Kowal
Peter Michaux
We are trying to reduce the proposal to the essentials to achieve the
goal of the group: actual sharable code. The "exports on the scope
chain" is controversial and not essential so why bother upsetting
those who do not like it.
Peter
Peter Michaux
On Tue, Feb 3, 2009 at 3:02 PM, Kris Kowal <cowber...@gmail.com> wrote:
Chris Zumbrunn pointed out in IRC that the third point may be
unnecessarily restrictive in what an implementation might decide to
export from a module. The third point is currently
"A module is guaranteed that "var" and "function" declarations are not
communicated implicitly to other modules."
To write standard libraries that would work in an environment with the
above guarantee, only the following spec is required
"An implementation may restrict the exports object as the only means
of exporting and so standard libraries must use the exports object as
the sole means of exporting."
The wording could be better but the "may" and "must" are carefully chosen.
Is that still restrictive enough to attain compatibility with your
goals? If so I will update the proposal.
Peter
Kris Kowal
> of exporting and so standard libraries must use the exports object as
> the sole means of exporting."
>
> The wording could be better but the "may" and "must" are carefully chosen.
>
> Is that still restrictive enough to attain compatibility with your
> goals? If so I will update the proposal.
>
> Peter
Yes. This is compatible with the goal of interoperability and
incremental migration.
Kris
Peter Michaux
I've added some sample code there to explore what sharing with the
browser might look like.
The wiki is moving so please just comment on the list and save edits for later.
Peter