Concerns about the proposed module system(s)
Tom Robinson
understand the desire to make JavaScript "safer", but I feel like
we're shoehorning in something that should be a language feature.
I'm in favor of a simpler load system that simply checks to see if a
file has been loaded previously, looks in search paths, and
essentially does a global eval or loads a binary lib (similar to
Ruby's "require" system)
I've discussed it a lot on IRC but I'll summarize my argument here.
1) The API:
This example was taken from another thread:
var greeten = require('greet-en').greet;
This is not user friendly at all. I've never encountered a module
system that requires you specify the name of the module, the name of
each object from the module you wish to import, and the name you want
to assign each object to.
The VAST majority of the time I just want to do:
require("File")
and have access to the File object.
Code should be properly namespaced to prevent collisions. We can
enforce it in the package management repository if we want.
2) Interoperability with the browser:
While it's possible to share code with the browser, you'll have to
write your module differently than the "normal" way, since there's no
way to automatically place all code from a file into a separate scope
in the browser, as is done on the server. That is, on the server you'd
write:
json_parse = function() { ... }
and it's automatically part of it's own module, whereas for the
browser you'd write the following in order to prevent global pollution:
JSON = {};
JSON.json_parse = function() { ... }
On the server, the first would be imported like:
var json_decode = require("json").json_decode;
While the second would have a useless extra level of namespacing:
var json_decode = require("json").JSON.json_decode;
I fear that half of the libraries will be written in the first style,
while half will be written in the second style. I realize you usually
have to look at API documentation anyway, but I'm a fan of
consistency, and this does not encourage consistency.
3) Interoperability with other module systems:
There's already a convention in JavaScript to namespace "modules"
simply using JavaScript objects. A global eval load system supports
that well.
I suspect many libraries that have their own module system (Dojo?
YUI?) would be incompatible with anything as complicated as the
proposed system.
And, as noted in the other thread, people are working on adding a
module system to the official ECMA spec. It would be a shame if we
came up with our own system that was incompatible with the official
one (if it's ever actually added to the spec).
4) Implementation difficulty:
Are we sure we can implement the proposed system in all the major
interpreters without too much trouble? ondras mentioned in IRC
yesterday that V8 doesn't support executing code with arbitrary
scopes, as is done in the example implementation on the Helma site .
There may be a workaround, I don't know.
A global eval load system can be implemented in pure JavaScript (given
a standard File API) and can be shared across all the interpreters. It
could even be used in the browser if you replace/abstract File with
XMLHttpRequest.
I'm interested in hearing if others agree, and if you don't, why not.
Thanks,
-Tom
Patrick Mueller
> I'm in favor of a simpler load system that simply checks to see if a
> file has been loaded previously, looks in search paths, and
> essentially does a global eval or loads a binary lib (similar to
> Ruby's "require" system)
I'm not against something like an include() function which operated
the same was as <script src=""> (ie, same as the C preprocessor's
#include statement). Over time, I expect people wouldn't really use
this much, at least for libraries, but it would certainly be useful
for existing bodies of JS code being used in browsers today.
> 1) The API:
>
> This example was taken from another thread:
>
> var greeten = require('greet-en').greet;
>
> This is not user friendly at all. I've never encountered a module
> system that requires you specify the name of the module, the name of
> each object from the module you wish to import, and the name you want
> to assign each object to.
>
> The VAST majority of the time I just want to do:
>
> require("File")
>
> and have access to the File object.
The code example was slanted for brevity. Here's how I would have
written the code "in real life".
var greet_lib = require('greet-en')
greet_lib.greet("hello")
greet_lib.greet("world")
I'd do it this way, because presumably these libraries contain more
than one function, so you don't really want to assign a local for each
function, but one per module, then dereference the function from the
module object. This is similar to common usage in Python (though
Python allows other behaviors also).
> 2) Interoperability with the browser:
I think we can get pretty close, actually. In the playing I've done,
you have to prefix variables in the module with var to prevent
infecting the global scope, but beyond that, I think you can do what
you need. Presumably we'll get this modularity added to the browser
somehow, over time.
> 3) Interoperability with other module systems:
Interesting question.
> 4) Implementation difficulty:
That's what we need to find out.
Patrick Mueller - http://muellerware.org/
Kris Kowal
Thanks for bringing up these concerns. I can address some of your
points.
On Feb 2, 6:37 pm, Tom Robinson <tlrobin...@gmail.com> wrote:
> This example was taken from another thread:
>
> var greeten = require('greet-en').greet;
>
> This is not user friendly at all. I've never encountered a module
> system that requires you specify the name of the module, the name of
> each object from the module you wish to import, and the name you want
> to assign each object to.
var {greet} = require("greet-en");
But, I do agree that imports like this should work in deeply legacy
JavaScript. This syntax is not in our current proposal, but what
would you think of this syntax:
include("greet-en", ["greet"]);
or
include("greet-en", {"greet": "greet"});
The trouble with this syntax is that, to support it, we need to add an
"imports" object, and a "with (imports)" block to the module function:
(function (require, include, imports, exports) { with (imports)
{ with (exports) {
With the addition of "with" blocks, the whole issue becomes much more
contentious, though I'm better prepared to discuss why this doesn't
cause the same problems "with" blocks are known for. However, we did
not include this option in our presentation to EC39 because it blows
static lexical analysis of free variables out of the water. It's a
hard trade-off.
>
> The VAST majority of the time I just want to do:
>
> require("File")
>
> and have access to the File object.
include("File"); // in deeply legacy supporting mode
from "File" import * // in future-ES
> Code should be properly namespaced to prevent collisions. We can
> enforce it in the package management repository if we want.
spaced file to give you peace of mind, especially when you're sharing
your name space with unknown, potentially malicious peers.
> 2) Interoperability with the browser:
the lexical scope in modern browsers. I'll get to how later, but we
have a prototype.
> 3) Interoperability with other module systems:
> YUI?) would be incompatible with anything as complicated as the
> proposed system.
defer to an external loader mechanism to be interoperable with Caja.
To that end, we're hoping to develop out our transitional syntax so
that Dojo and YUI could be refactored to not only work inside a module
loader, but also in global scripts.
> And, as noted in the other thread, people are working on adding a
> module system to the official ECMA spec. It would be a shame if we
> came up with our own system that was incompatible with the official
> one (if it's ever actually added to the spec).
be interoperable with the systems you're building.
> 4) Implementation difficulty:
> A global eval load system can be implemented in pure JavaScript (given
> a standard File API) and can be shared across all the interpreters. It
> could even be used in the browser if you replace/abstract File with
> XMLHttpRequest.
chain using closures (and if necessary, "with" blocks). This is what
we do in Chiron with modules.js http://code.google.com/p/chironjs/.
The system Ihab and I presented to ECMA would be supportable with a
loader in very old browsers.
Kris
Peter Michaux
>
> On Feb 2, 2009, at 9:37 PM, Tom Robinson wrote:
>
>> I'm in favor of a simpler load system that simply checks to see if a
>> file has been loaded previously, looks in search paths, and
>> essentially does a global eval or loads a binary lib (similar to
>> Ruby's "require" system)
>
> I'm not against something like an include() function which operated
> the same was as <script src=""> (ie, same as the C preprocessor's
> #include statement). Over time, I expect people wouldn't really use
> this much, at least for libraries, but it would certainly be useful
> for existing bodies of JS code being used in browsers today.
An include() that does a global eval cannot be *standard* if the
Helma/Pythonic proposal is accepted. The reason it cannot be standard
is that if it is standard then someone might use it in a library that
is "standard compliant" and the whole global clobbering issue returns.
That defeats the purpose of the Helma/Pythonic proposal.
An implementation could have an include() as an extension, of course,
but that might encourage code that is not sharable as the code would
not be standard compliant.
Peter
Peter Michaux
> On Feb 2, 6:37 pm, Tom Robinson <tlrobin...@gmail.com> wrote:
>> This example was taken from another thread:
>>
>> var greeten = require('greet-en').greet;
>>
>> This is not user friendly at all. I've never encountered a module
>> system that requires you specify the name of the module, the name of
>> each object from the module you wish to import, and the name you want
>> to assign each object to.
>
> Agreed. With Harmony syntax, this gets a bit better:
>
> var {greet} = require("greet-en");
Yes and I think that is admirably JavaScript-ish. The module is
clearly first-class as it is easy to see it being passed around.
> But, I do agree that imports like this should work in deeply legacy
> JavaScript. This syntax is not in our current proposal, but what
> would you think of this syntax:
>
> include("greet-en", ["greet"]);
> or
> include("greet-en", {"greet": "greet"});
If old JavaScript engines need support, just write the destructuring
out in full.
var mod = require('greet-en");
var greet = mod.greet;
Then apply some patients, browsers will catch up, and the code can be
reduced. This is the browser scripting world. Patients is part of the
system.
> from "File" import * // in future-ES
There is no need for the Python-like "from ... import ..." in
JavaScript. Although it doesn't cover the '*' case, the reuse of
(destructuring) assignment is sufficient and allows nested
destructuring which the above syntax doesn't.
>> Code should be properly namespaced to prevent collisions. We can
>> enforce it in the package management repository if we want.
>
> But there's nothing like actually having sovereignty in your own name
> spaced file to give you peace of mind, especially when you're sharing
> your name space with unknown, potentially malicious peers.
Or the monkey-patcher on the team.
>> 2) Interoperability with the browser:
>
> Summarily, it is possible to implement module loaders that manipulate
> the lexical scope in modern browsers. I'll get to how later, but we
> have a prototype.
>
>
>> 3) Interoperability with other module systems:
>
>> I suspect many libraries that have their own module system (Dojo?
>> YUI?) would be incompatible with anything as complicated as the
>> proposed system.
>
> Mark Miller pointed out last week that Dojo and YUI would have to
> defer to an external loader mechanism to be interoperable with Caja.
> To that end, we're hoping to develop out our transitional syntax so
> that Dojo and YUI could be refactored to not only work inside a module
> loader, but also in global scripts.
>
>> And, as noted in the other thread, people are working on adding a
>> module system to the official ECMA spec. It would be a shame if we
>> came up with our own system that was incompatible with the official
>> one (if it's ever actually added to the spec).
>
> That's us, and we're very interested in crafting a design that would
> be interoperable with the systems you're building.
Interoperable is a great goal.
Possibly a note of pragmatism: My guess is that it will be a very long
time (~10 years) before the fourth edition of ECMAScript is proposed,
finalized, approved, implemented, and ubiquitous in browsers. If there
is an ES3.2 it could even be longer. There is no way to know now if a
current module proposal will be standardized. We might not even be
programmers when it can be used in the browser. The programs we are
writing will likely be replaced by then. ECMAScript version will also
be opt-in as part of the script tag's type attribute value so current
code will continue working.
Although interoperability is a great goal but none of use have a
crystal ball. Macromedia thought it did and jumped the gun
implementing the original proposed ECMAScript with classes. That
became the ES4 proposal and was shot down partly due to the packages,
namespaces and units complexity.
Peter
Brian LeRoux
require('path.to.module.print');
print('hi there!);
...
I don't mind the pythonic syntax but it does smell like unnecessary
ceremony and repetition to 'protect' library authors from accidental
namespace collision which never happens in good practice.
Peter Michaux
>
> Precisely, what is the concern with:
>
> require('path.to.module.print');
That creates a local "print" variable but it is not shown as such.
Explicit is better, in many people's opinion.
What if there is a "print" variable in the module doing the requiring.
It gets clobbered by the above require.
Also is that a path/to.js module with a module.print property? Or is
that a path/to/module.js module with a "print" property? It is
ambiguous.
> print('hi there!);
Peter
Ross Boucher
proposed solutions are incredibly convoluted. I'd much prefer Tom's
suggestion of a simple require() function. The vast majority of
programmers want something simple, myself included.
If for no other reason that its by far the simplest thing to do, I
strongly recommend adopting the require() function which takes a file
path, and ensures only loading once. For people who really want the
ability to reload a file, either add an unload() method, or an
optional second parameter.
Benefits:
1) Simple. Familiar to anyone who has programmed in any C style
language.
2) Browser compatible. It's trivial to implement this method as a
synchronous (or async) XHR in the browser.
3) Easy to implement, very difficult to implement wrong.
Kris Kowal
On Feb 2, 8:53 pm, Ross Boucher <rbouc...@gmail.com> wrote:
> In exchange for all of this so called "security" of namepsacing, the
> proposed solutions are incredibly convoluted. I'd much prefer Tom's
> suggestion of a simple require() function. The vast majority of
> programmers want something simple, myself included.
JavaScript. The proposal Ihab and I are working on with the ECMA
group is to introduce a "hermetic eval" to the language or an "orphan"
method that would nullify the parent reference of a closure. In
addition to the "freeze" method already going into Harmony, we can use
these tools to make reasonable security assertions: what objects are
transitively accessible and mutable in evaluation contexts. It's our
hope that long before this is a ubiquitous feature in browser
JavaScript implementations, it will be emulated with Caja, and other
sandboxing solutions, with server side static analysis and proxied
script rewriting, as is necessary today for mutually suspicious
scripts like advertisements and user credential managers. That is to
say, converging on a way to write modules today could greatly enhance
interoperability between secure scripts, server side libraries, and
legacy client side libraries. Being ready for client side native
security sandboxing would be a nice side-effect.
Kris
Brian LeRoux
>
> That creates a local "print" variable but it is not shown as such.
> Explicit is better, in many people's opinion.
I can see that. I can also see how implicit conventions might be
beneficial too. Less code. Of course I could always implement a req
function that does things the way I like em so it really is not a big
concern.
> What if there is a "print" variable in the module doing the requiring.
> It gets clobbered by the above require.
Right, though I would like the design to assume developers understand
the rope they have is plenty useful and dangerous too. Certainly a
rogue could invoke require without thinking of doing something so
clever as:
function() {
require('module.print');
}();
or perhaps...
var SuperRad = function() {
require('module.print');
this.print = print;
};
SuperRad.prototype.choice = function(apples, oranges) {
this.print('bananas!')
};
...etc... coming from the Ruby world this is always mitigated by the
author correctly scoping their own libs. *shrug.
> Also is that a path/to.js module with a module.print property? Or is
> that a path/to/module.js module with a "print" property? It is
> ambiguous.
Had not considered this. The browser would say, 'wtf is this' in the
same voice benitio del toro used in fear in loathing. Agree w/ above
and I think explicitly including a js and having the author know 'the
right thing to do' library wise feels correct to me.
require('path/to/module.js'); // loads print into the current scope
Breton
> This is not user friendly at all. I've never encountered a module
> system that requires you specify the name of the module, the name of
> each object from the module you wish to import, and the name you want
> to assign each object to.
>
> The VAST majority of the time I just want to do:
>
> require("File")
>
feature of objective-J,
For the rest of us, proper namespacing is important, do not
underestimate it. The module system you propose is sinking the modern
web. It is not sustainable. Doesn't scale. It's better to trust an
indelible rule of the standard/implementation, than trust the good
behavior of thousands of individual developers.
You can enforce your namespacing on a repository, but you can't
enforce it on a module that I wrote myself, published outside of your
repository, and someone found it useful enough to include in their
project. It's a burden that's better to place on the computer than to
inflict on all of us. Unfortunately we don't all live in your perfect
world, where everyone is perfectly behaved.
As for usability, I do actually think that this is *more* usable,
because it's visible on my side what variables I'm getting/clobbring.
It's putting the user of the module in control, instead of the module
author. If I decide that I want to change over from using the
(printify) module, to using the (SFprintable) module, which happens to
have the same API, but implements new feature X Y Z, and improves
performance, I only have to change one line of code (the require), and
all the REFERENCES to that library stay the same throughout the rest
of my code. In a global eval situation, I'm left to the variable
naming whims of a perfect stranger, and by using their module, I'm
marrying my project to that module, indelibly. This is not good.
Mario Valente
> I'm still not convinced Pythonic modules is the way to go. I
> understand the desire to make JavaScript "safer", but I feel like
> we're shoehorning in something that should be a language feature.
>
> 1) The API:
>
> This example was taken from another thread:
>
> var greeten = require('greet-en').greet;
>
> This is not user friendly at all. I've never encountered a module
> system that requires you specify the name of the module, the name of
> each object from the module you wish to import, and the name you want
> to assign each object to.
>
> The VAST majority of the time I just want to do:
>
> require("File")
>
> and have access to the File object.
>
>
> I'm interested in hearing if others agree, and if you don't, why not.
>
I like the pytohnic modules I agree, but I also agree with you.
If we want to follow pythonic modules, the following should be
allowed:
var greetings = require('greetings').*
or
var greetings = require('greetings').all
or simply
var greetings = require('greetings')
This would be similar to the pythonic 'from module x import *'
You then could use greetings.greeten(); or to achieve
the original result do var greeten = greeting.greeten;
-- MV
George Moschovitis
I am wondering if the following scenario is possible:
main.js:
var File = require("lib.File");
var text = File.read("mytext.txt");
--
lib/File.js:
var File = {
read: function(path) {
},
..
}
ie using just:
var File = require("lib.File");
instead of
var File = require("lib.File").File;
for this common case (ie a var with the same name as the script file)
Hannes Wallnoefer
> I'm still not convinced Pythonic modules is the way to go. I
> understand the desire to make JavaScript "safer", but I feel like
> we're shoehorning in something that should be a language feature.
>
> I'm in favor of a simpler load system that simply checks to see if a
> file has been loaded previously, looks in search paths, and
> essentially does a global eval or loads a binary lib (similar to
> Ruby's "require" system)
>
> I've discussed it a lot on IRC but I'll summarize my argument here.
>
>
> 1) The API:
>
> This example was taken from another thread:
>
> var greeten = require('greet-en').greet;
>
> This is not user friendly at all. I've never encountered a module
> system that requires you specify the name of the module, the name of
> each object from the module you wish to import, and the name you want
> to assign each object to.
>
> The VAST majority of the time I just want to do:
>
> require("File")
>
> and have access to the File object.
have include() and import(), which build on top of require() to
provide more comfortable loading options, the first by copying
individual properties from the loaded to the loading module, the
second by setting up a property with the loaded module as a whole.
The reason we dropped it was to only have the minimal required
functionality in the standard. include() and import() can easily be
implemented on top of require() and can therefore be packed with the
framework or application on top of the standard API.
Here's how Helma NG implements this:
http://github.com/hns/helma-ng/blob/89a30cf8b6f83e8f180a6d87d45d30f57fdeefde/modules/global.js
> Code should be properly namespaced to prevent collisions. We can
> enforce it in the package management repository if we want.
>
>
> 2) Interoperability with the browser:
>
> While it's possible to share code with the browser, you'll have to
> write your module differently than the "normal" way, since there's no
> way to automatically place all code from a file into a separate scope
> in the browser, as is done on the server. That is, on the server you'd
> write:
>
> json_parse = function() { ... }
>
> and it's automatically part of it's own module, whereas for the
> browser you'd write the following in order to prevent global pollution:
>
> JSON = {};
> JSON.json_parse = function() { ... }
>
> On the server, the first would be imported like:
>
> var json_decode = require("json").json_decode;
>
> While the second would have a useless extra level of namespacing:
>
> var json_decode = require("json").JSON.json_decode;
>
> I fear that half of the libraries will be written in the first style,
> while half will be written in the second style. I realize you usually
> have to look at API documentation anyway, but I'm a fan of
> consistency, and this does not encourage consistency.
For example, Helma NG comes with the standard JSON and json2 modules -
why would you reimplement that in a slightly different way, just
because you can?
http://github.com/hns/helma-ng/blob/89a30cf8b6f83e8f180a6d87d45d30f57fdeefde/modules/core/json2.js
And you use them just like you would on the client - except you have
to load them first:
include('core.json2');
x = JSON.parse(y);
y = JSON.stringify(x);
So what you really have is the option to write code either way you
want. My guess is that in 99% of cases you know right from the start
whether a module is server-only, client-only or both, depending on
what it is going to do, and what host object or functionality it
requires. And if you really ever discover you need to make a piece of
code namespace-safe because you want to run it on the client, it's not
_that_ big a deal.
>
> 3) Interoperability with other module systems:
>
> There's already a convention in JavaScript to namespace "modules"
> simply using JavaScript objects. A global eval load system supports
> that well.
>
> I suspect many libraries that have their own module system (Dojo?
> YUI?) would be incompatible with anything as complicated as the
> proposed system.
>
> And, as noted in the other thread, people are working on adding a
> module system to the official ECMA spec. It would be a shame if we
> came up with our own system that was incompatible with the official
> one (if it's ever actually added to the spec).
standardize on a module system on the server-side, it will clash with
other module systems that have been built already for client-side use,
if only because they use the same global names such as 'require'.
If, on the other hand, ECMAScript were to get its own proper module
system with syntactic sugar and all, I'd be delighted and the first to
drop everything else, provided it is at least on par with what we
have, feature wise.
> 4) Implementation difficulty:
>
> Are we sure we can implement the proposed system in all the major
> interpreters without too much trouble? ondras mentioned in IRC
> yesterday that V8 doesn't support executing code with arbitrary
> scopes, as is done in the example implementation on the Helma site .
> There may be a workaround, I don't know.
shortcoming in the V8 API, which seems to be tailored exactly for use
in Chrome and nothing else. But I understand it's a showstopper until
there's a way around it.
Hannes
Peter Michaux
>> 4) Implementation difficulty:
>>
>> Are we sure we can implement the proposed system in all the major
>> interpreters without too much trouble? ondras mentioned in IRC
>> yesterday that V8 doesn't support executing code with arbitrary
>> scopes, as is done in the example implementation on the Helma site .
>> There may be a workaround, I don't know.
>
> Yes, ondras told be about that on IRC today. IMO this is a real
> shortcoming in the V8 API, which seems to be tailored exactly for use
> in Chrome and nothing else. But I understand it's a showstopper until
> there's a way around it.
I don't think that is a show stopper. It is not necessary to use V8.
Peter
Ondrej Zara
>>> interpreters without too much trouble? ondras mentioned in IRC
>>> yesterday that V8 doesn't support executing code with arbitrary
>>> scopes, as is done in the example implementation on the Helma site .
>>> There may be a workaround, I don't know.
>>
>> Yes, ondras told be about that on IRC today. IMO this is a real
>> shortcoming in the V8 API, which seems to be tailored exactly for use
>> in Chrome and nothing else. But I understand it's a showstopper until
>> there's a way around it.
same or better level than me) to confirm this, but my opinion after
several discussions in V8 mailing list is that Pythonic loading can
NOT be emulated in V8. I am able to get close, but it still has flaws.
>
> I don't think that is a show stopper. It is not necessary to use V8.
>
them) and freeze the module-related stuff. At least until we can know
V8 team's general attitude regarding this problematics.
Ondrej
> Peter
>
> >
>
Wes Garland
IMHO this shows that we should focus on other issues (we have lots of
them) and freeze the module-related stuff. At least until we can know
V8 team's general attitude regarding this problematics.
Ondrej -- what happens if you wrap the entire module in a closure which returns an object? That should work as it doesn't require any implementation specific stuff (it's closer to browser JS) and isn't too far from the proposed system. You could write modules like this:
function()
{
return {
method1: function() { do this; };
method 2: function() { do that; };
}
}
That might also to solve a nagging problem I've been having -- what syntax do we want to use for modules which only provide a class?
var messageClass = require("message");
var message = new messageClass.constructor();
...is simply terrible.
Wes
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
Ondrej Zara
>> them) and freeze the module-related stuff. At least until we can know
>> V8 team's general attitude regarding this problematics.
>
> Ondrej -- what happens if you wrap the entire module in a closure which
> returns an object? That should work as it doesn't require any
> implementation specific stuff (it's closer to browser JS) and isn't too far
> from the proposed system. You could write modules like this:
>
> function()
> {
> return {
> method1: function() { do this; };
> method 2: function() { do that; };
> }
> }
>
missing "var" :-(
> That might also to solve a nagging problem I've been having -- what syntax
> do we want to use for modules which only provide a class?
>
> var messageClass = require("message");
> var message = new messageClass.constructor();
>
var Message = require("message").Message;
var message = new Message();
Ondrej
Kevin Dangoor
> IMHO this shows that we should focus on other issues (we have lots of
> them) and freeze the module-related stuff. At least until we can know
> V8 team's general attitude regarding this problematics.
Is it possible to do a pure JS implementation that does not provide
"missing var" protection but does at least conform to the basic
interface? I would prefer having an imperfect solution running on v8
than nothing at all.
Kevin
--
Kevin Dangoor
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com
Patrick Mueller
> On Tue, Feb 3, 2009 at 8:47 AM, Ondrej Zara <ondre...@gmail.com>
> wrote:
>> IMHO this shows that we should focus on other issues (we have lots of
>> them) and freeze the module-related stuff. At least until we can know
>> V8 team's general attitude regarding this problematics.
>
> Is it possible to do a pure JS implementation that does not provide
> "missing var" protection but does at least conform to the basic
> interface? I would prefer having an imperfect solution running on v8
> than nothing at all.
I've been playing with "module" stuff a bit in JS, and this was the
trade-off that I took as well. Play some games, wrapping wad of JS
with a function closure/etc, which can then be eval()'d, but doesn't
provide protection from folks who forget the "var". Basically, it
works if you write your code correctly. Compared to if you use Rhino
and actually CAN provide the protection.
A trade-off I'm willing to live with for now.
My assumption is, if this "module" concept has legs, then we'll have
this capability natively in all the JS VM's eventually that do the
right thing.
Ondrej Zara
>> IMHO this shows that we should focus on other issues (we have lots of
>> them) and freeze the module-related stuff. At least until we can know
>> V8 team's general attitude regarding this problematics.
>
> Is it possible to do a pure JS implementation that does not provide
> "missing var" protection but does at least conform to the basic
> interface? I would prefer having an imperfect solution running on v8
> than nothing at all.
level, as it directly accesses filesystem. Therefore, it cannot be
realized as "pure JS" - at least not without a functional File()
object :)
From what I have learned in last few days, V8's realization would have
(at least) these flaws:
- changes to built-in structures (Array.prototype.XXX) are not visible
to modules,
- data structures defined in module A fail all "instanceof" checks
performed in any other non-A module (including the global context).
I am not sure if this list is exhaustive, though.
Ondrej
Ross Boucher
On Feb 3, 2009, at 12:55 AM, Breton wrote:
> it's really important to you, you could just implement that as a
> feature of objective-J,
It already is. But it's not as if we invented the idea ourselves. This
is what php has done forever. This is, essentially, what we already
have in JavaScript. This is almost certainly the most used way to
import code that there is.
> For the rest of us, proper namespacing is important, do not
> underestimate it. The module system you propose is sinking the modern
> web. It is not sustainable. Doesn't scale. It's better to trust an
> indelible rule of the standard/implementation, than trust the good
> behavior of thousands of individual developers.
Sinking the modern web? The web seems to be doing just fine with
global variables. Honestly, I can't think of a single instance in the
last 6 years I've been doing this where I ran into a serious namespace
collision. I'm not saying it doesn't happen, but I am suggesting its
not nearly as common as say. And jQuery, almost certainly the most
popular JavaScript framework, makes a point of being well behaved with
global variables. Are the two related? Maybe, maybe not. But to claim
that the modern web is sinking seems silly.
> If I decide that I want to change over from using the
> (printify) module, to using the (SFprintable) module, which happens to
> have the same API, but implements new feature X Y Z, and improves
> performance, I only have to change one line of code (the require), and
> all the REFERENCES to that library stay the same throughout the rest
> of my code.
This sounds like a huge red herring to me. How often are two libraries
entirely API compatible but use different method names? And if you're
really concerned, you can always implement a shim around any library.
In all honesty, I don't necessarily think a module system is a bad
idea. I just think its a terrible starting point. We're already seeing
reports that the proposed module system is not possible in V8. If the
goal of this discussion is to come up with something that we can
actually convince environments to implement, the more complex it
becomes the less likely we are to succeed.
Hannes Wallnoefer
> On Tue, Feb 3, 2009 at 8:47 AM, Ondrej Zara <ondre...@gmail.com> wrote:
>> IMHO this shows that we should focus on other issues (we have lots of
>> them) and freeze the module-related stuff. At least until we can know
>> V8 team's general attitude regarding this problematics.
>
> Is it possible to do a pure JS implementation that does not provide
> "missing var" protection but does at least conform to the basic
> interface? I would prefer having an imperfect solution running on v8
> than nothing at all.
sophisticated: evaluate the module within a closure on the module
scope, getting public and private variables for free in the process.
With the pythonic modules approach, when you evaluate a module, the
scope equals the this-object, so like in a global script everything is
public, regardless of whether it is defined via this.foo, var foo, or
function foo.
When you run the code wrapped in a closure function, only this.foo is
public, while var foo and function foo are only visible within the
closure. For example with the following code, a is defined in scope, b
and c in a private scope visible to a, and d is defined in the global
scope.
var scope = {};
(function() {
// pseudo-module begins here
this.a = function() { b(); };
function b() { print(c, d); }
var c = "c";
d = "d";
// pseudo-module ends here
}).call(scope);
So yes, we do have an attractive option that can be done in plain JS,
provided we can read the module source code.
Hannes
ihab...@gmail.com
On Tue, Feb 3, 2009 at 2:51 AM, Hannes Wallnoefer <han...@gmail.com> wrote:
> Yes, ondras told be about that on IRC today. IMO this is a real
> shortcoming in the V8 API, which seems to be tailored exactly for use
> in Chrome and nothing else. But I understand it's a showstopper until
> there's a way around it.
1. As I understand it, you mean V8 provides only static global scope
per OS process?
[ This would make sense, per your comment, since Chrome uses OS
processes extensively as a means of isolation, but I want to make sure
I understand. ]
2. Could you provide some more details of the API deficiency in question?
Thank you!!!
Ihab
--
Ihab A.B. Awad, Palo Alto, CA
ihab...@gmail.com
On Tue, Feb 3, 2009 at 8:26 AM, <ihab...@gmail.com> wrote:
> 1. As I understand it, you mean V8 provides only ONE static global scope
> per OS process?
I missed the word "one".
Ondrej Zara
>> (printify) module, to using the (SFprintable) module, which happens to
>> have the same API, but implements new feature X Y Z, and improves
>> performance, I only have to change one line of code (the require), and
>> all the REFERENCES to that library stay the same throughout the rest
>> of my code.
>
> This sounds like a huge red herring to me. How often are two libraries
> entirely API compatible but use different method names? And if you're
> really concerned, you can always implement a shim around any library.
>
I completely support this. From what I know, different libraries with
similar functionality always offer different names for their classes /
objects.
Ondrej
Davey Waterson
> global variables. Honestly, I can't think of a single instance in the
> last 6 years I've been doing this where I ran into a serious namespace
> collision. I'm not saying it doesn't happen, but I am suggesting its
> not nearly as common as say. And jQuery, almost certainly the most
> popular JavaScript framework, makes a point of being well behaved with
> global variables. Are the two related? Maybe, maybe not. But to claim
> that the modern web is sinking seems silly.
Actually IIRC the whole issue became a problem because of the
collision between JQuery and prototype and the use of the $ object in
the global namespace. I think everyone just got antsy after that and
the rule became 'thou shall not pollute the global namespace', it's a
good rule, but I think we should allow for well defined exceptions to
it, and move on.
I dont think it is a big problem for the most part, as we are
intending to produce well behaved modules, which should be tested
against global namespace additions by doing a before and after
walkthrough of the global object in the test suites.
for example, i'm not too worried about require living there as I think
(like in the case of JSON) it will eventually be implemented natively
in the VMs. We get the opportunity to describe the feature. I kind of
see the module thing playing out that way and perhaps even some of the
other API's like File.
My worry about what I see in the current discussions is a kind of
overreach, I think we should just try to find as simple an
implementation as we can and backburner much of the esoteric
discussion. Using the python pattern ok for ideas, but as JS is not
python I'd be wary of any effort to implement that in JS (same goes
for Ruby and Php) although I betting even that'll be tricky.
ihab...@gmail.com
On Tue, Feb 3, 2009 at 8:26 AM, <ihab...@gmail.com> wrote:
> Hi Hannes and Ondras,
s/Ondras/Ondrej/ -- my apologies.
Ondrej Zara
>> Yes, ondras told be about that on IRC today. IMO this is a real
>> shortcoming in the V8 API, which seems to be tailored exactly for use
>> in Chrome and nothing else. But I understand it's a showstopper until
>> there's a way around it.
>
> 1. As I understand it, you mean V8 provides only static global scope
> per OS process?
>
However, these scopes are heavily separated and each of them gets its
own set of top-level objects. Once can freely "link" global objects
from scopes (one scope as a property of second scope), but that is
basically everything that could be done. For more detail, please see
this my question on v8-users list:
http://groups.google.com/group/v8-users/browse_thread/thread/7bc77a7db7cf0a27
> 2. Could you provide some more details of the API deficiency in question?
>
Ondrej
Ondrej Zara
>
> s/Ondras/Ondrej/ -- my apologies.
O.
ihab...@gmail.com
> No. One OS process can create as many global scopes as necessary.
> However, these scopes are heavily separated and each of them gets its
> own set of top-level objects. Once can freely "link" global objects
> from scopes (one scope as a property of second scope), but that is
> basically everything that could be done.
So it seems you can create a new global scope, ensure its "Array"
points to a shared one, then run your code in that. And it seems like,
at the end of the thread, you were happy. :)
What is the missing part?
Ondrej Zara
this (thanks for a good direction!), but even now, I see two places
which are not making me happy:
- the need to enumerate all global objects to be "masked" (by their
global versions). This enumerated overwriting seems to me more like a
hack
- I am not sure if I can "access" (fetch) the global Array from global
object (so I can assign it to module's local context), as these
globals are normally "invisible" (not listed when doing "for .. in").
Perhaps I have expressed myself incorrectly regarding "linking" of
global objects: I was not talking about Object, Array, Date, Function
etc, but about the "global namespace" objects.
I will spend some time experimenting with this masking tomorrow.
O.
Kris Zyp
precludes implementors from copying all the variables from the module
scope to the global scope (where they don't exist in the global scope)
as modules are evaluated. This is completely compatible with Helma
"require" and the disambiguation that it provides. Thus implementations
can allow the users to enjoy the benefits of both pythonically scoped
modules and globally eval'ed modules at the same time. All the code that
is written to depend on the "serverJS standard" require, and code that
is disciplined to explicitly introduce library code with local variables
will work just fine if an implementation copies variables from the
modules scope to the global scope on evaluation:
var libA = require("libA");
var libB = require("libB");
printerA.print("explicitly using libA's print function");
printerB.print("explicitly using libB's print function");
And all the code that is written for globally eval'ed code and that
wants the convenience of top-level functions with minimal effort will work:
print("I don't care who's implementation of print, just print something");
One thing to remember is that there is a wide variety of application
development scenarios out there. Large production libraries that may be
used across various applications should employ extensive discipline to
avoid name collisions. However, while it is easy to theorize about all
the possible name collisions that could happen, there also *a lot* of
applications out there that will be written by one person, in a
deterministic environment where no one is going to be writing anything
on top of it. Making things simple and easy for them should be
essential, and forcing them to learn and use this new discipline is
inappropriate and represents a disproportionate amount of effort for the
rare benefits (as many have attested to).
I am +1 for Helma require because I can implement it (satisfying the
requiring behavior and use cases) and still put the variables from
modules into the global scope, giving both ends of the spectrum what
they need. Implementations can choose to copy to the global or not,
depending on their use case (they could have it as a setting). Some may
want to use object-capability security, others may want a more
browser-like environment (like Jaxer). Allowing for a variety of servers
to and their targets is valuable. It's an elegant API, IMO. Or at least
a good compromise.
Thanks,
Kris
Wes Garland
I just don't see jumping hoops to avoid programmer incompetence as a huge issue.
Tom Robinson
On Feb 3, 2009, at 6:57 AM, Kevin Dangoor wrote:
>
> On Tue, Feb 3, 2009 at 8:47 AM, Ondrej Zara <ondre...@gmail.com>
> wrote:
>> IMHO this shows that we should focus on other issues (we have lots of
>> them) and freeze the module-related stuff. At least until we can know
>> V8 team's general attitude regarding this problematics.
>
> Is it possible to do a pure JS implementation that does not provide
> "missing var" protection but does at least conform to the basic
> interface? I would prefer having an imperfect solution running on v8
> than nothing at all.
>
> Kevin
If I understand correctly, a pure JS implementation without "missing
var" protection *is* possible with Kris (Kowal) and Ihab's proposal,
but not the Pythonic / Helma modules. This is primarily because you
explicitly specify which things are exported by adding them to the
"export" variable.
(An alternative to the "export" object in Kris/Ihab's proposal is
using "this" as the export object, as Hannes mentioned. But I think
they're essentially equivalent, it's just a matter of if you want to
use "this" or "export")
Since the module can be wrapped in a function when loaded, if you
"var" everything when using the Kris/Ihab module system the code can
be reused on pure JS implementations (in the browser or on the server)
without polluting the global namespace, but if you use it with a
"real" implementation you get the global pollution protection too.
Seems like a pretty good tradeoff to me. Sort of a graceful
degradation. The standard library will certainly be scrutinized enough
to ensure there aren't any missing vars.
In the Pythonic/Helma modules anything that's added to the module's
scope is implicitly "exported". IMO this is a huge drawback. As
mentioned in another message, there's no way to "scrape" these
implicitly exported properties in pure JS implementations, and they
would pollute the global scope if you loaded them on the client.
Also, if I understand correctly, this means anything you import into a
module also gets *automatically exported* even if there's no reason
for it to be. This makes the "include" helper function completely
useless, since you're now (recursively) adding every property of every
included module to your module's scope, negating the purpose of the
module system. Or am I missing something?
This could be mitigated by wrapping the module in a function, and
using "var" for things you don't want to export, but it feels like a
hack to me.
I still think a complicated module system is beyond the scope of this
project, but Kris/Ihab's proposal addresses many of my original
concerns. It sounds like they've thought about this problem a *lot*. I
might be willing to support their proposal, but not the Pythonic
modules proposal. At least not without some changes.
-Tom
Peter Michaux
> I still think a complicated module system is beyond the scope of this
> project,
The goal of the group is to enable writing sharable code. A module
loading system is central to that goal because sharable code may load
other sharable code. That is, the module loading system will appear
directly in sharable code.
Peter
Ondrej Zara
"prerequisite" for any other work - or do you think it is?
Ondrej
>
> Peter
>
> >
>
Peter Michaux
It is not prerequisite to discussions about APIs. It is definitely
prerequisite to creating actual sharable library files which is what
we all really want in the end...actual code we can download and use.
Peter
Ondrej Zara
> points to a shared one, then run your code in that. And it seems like,
> at the end of the thread, you were happy. :)
>
> What is the missing part?
>
If I create a global scope for loaded module, that module won't be
able to access anything defined in _the_ global scope. This is not
exactly a problem since we want to separate modules, but it is not
compatible with Hannes's implementation on Rhino (his implementation
allows modules to see stuff in global scope).
O.
ihab...@gmail.com
> Just found one :/
> If I create a global scope for loaded module, that module won't be
> able to access anything defined in _the_ global scope. This is not
> exactly a problem since we want to separate modules, but it is not
> compatible with Hannes's implementation on Rhino (his implementation
> allows modules to see stuff in global scope).
I see. Yeah, so far, Kris and I have imagined that we would sequester
away any concept of "the" global scope in favor of hygiene.
Davey Waterson
avoid over engineering, just the features we need to get the others
api's to be loadable and play well together. (I do realize that is a
pretty ambitious goal in itself)
Kris Kowal
> I completely support this. From what I know, different libraries with
> similar functionality always offer different names for their classes /
> objects.
>
> Ondrej
example, "md5", "md4", "crc32", "sha", and "sha256" provide a "hash"
export. All "base64", "base32", "utf8", and "utf16" all support
"encode" and "decode" exports, so are interchangeable. "http",
"json", and "jsonp" all support a "request" method that agree on the
same parameters.
Kris
Hannes Wallnoefer
>
> In the Pythonic/Helma modules anything that's added to the module's
> scope is implicitly "exported". IMO this is a huge drawback. As
> mentioned in another message, there's no way to "scrape" these
> implicitly exported properties in pure JS implementations, and they
> would pollute the global scope if you loaded them on the client.
ordinary scripts is a deliberate decision. The (function() {}).apply
(this) closure pattern is widely known by now, so module authors that
need private variables are expected to apply it (I do it all the
time). Again, the idea is to have the least amount of "magic" going on
behind the scenes, and provide what is essentially a standard
JavaScript environment, just without the need to entrench yourself
into some namespace tower.
Besides, modules that expect to be wrapped into a closure won't be
doing much better on the client side without a matching loader.
> Also, if I understand correctly, this means anything you import into a
> module also gets *automatically exported* even if there's no reason
> for it to be. This makes the "include" helper function completely
> useless, since you're now (recursively) adding every property of every
> included module to your module's scope, negating the purpose of the
> module system. Or am I missing something?
the export() function to explicitly tell include what to copy. It's
really useful to have those listed explicitely. As an example, look at
the helma.unittest module:
http://github.com/hns/helma-ng/blob/89a30cf8b6f83e8f180a6d87d45d30f57fdeefde/modules/helma/unittest.js
> This could be mitigated by wrapping the module in a function, and
> using "var" for things you don't want to export, but it feels like a
> hack to me.
version, isn't it?
> I still think a complicated module system is beyond the scope of this
> project, but Kris/Ihab's proposal addresses many of my original
> concerns. It sounds like they've thought about this problem a *lot*. I
> might be willing to support their proposal, but not the Pythonic
> modules proposal. At least not without some changes.
Peter said, without that, any common API will be quite useless. I want
to run be able to run the Helma NG framework on other JS runtimes, and
run other frameworks on the Helma NG runtime. For that I need a
compatible module system and a compatible API.
Hannes
> -Tom
Ondrej Zara
of classes implementing the same interface - but they share method
names.
So you do (I don't know your code, so this example is presumably crippled):
"md5.hash()", "md4.hash()", "http.request()", "json.request()" etc...
Similarly, having classes MySQL() and PostgreSQL() which both have
method "query" presents no conflict...
O.
>
> Kris
> >
>
Kris Kowal
> I see. Yeah, so far, Kris and I have imagined that we would sequester
> away any concept of "the" global scope in favor of hygiene.
>
> Ihab
systems that provide security assertions. There inevitably will be
modules that are intended to only work in particular platforms.
There's a lot of discussion about whether a module system can or
should meet security in the middle, but this is only really necessary
to make it easier to be interoperable with security sandboxes (to
notice flaws like free variables aka missing var bugs). With Pythonic
or our ECMA proposal, a lot of modules can be shared without any
issues because they only import other modules, and provide exports
that only perform computation. The only time interoperability becomes
a problem is when the environments differ. For example, our proposal
for transitional module loaders does not require the loader to mask
the global scope, it merely states that modules that use the global
scope would not work in the future, or with server side rewriters like
Caja. There's a lot of room for compromise in these cases, where some
modules are simply less conformant than others and thus cannot be
interoperable.
Kris