On Mon, Aug 25, 2008 at 4:10 AM, Hannes Wallnoefer <hann...@gmail.com> wrote: > Could you elaborate some more on the difference between the module > scope and the module-this object, and how you implement the anonymous > function closure?
One notion that Chiron's modules.js preserves is of strong parallelism between the body of a module and the body of a prototype's constructor function. In a constructor function, inner functions, vars, and lets are effectively private because they are in the closure, and the closure is distinct from the context object. So, in a module (which usually maps 1 module for 1 file), the "module" object is analogous to the "this" context object, and the "moduleScope" is analogous to the closure. The "module" object is public, and the "moduleScope" is private.
This offers us an opportunity to avoid some problems that I encounter in Python occasionally when I'm playing dirty with the "from module import *" semantics. In Python, everything you import is implicitly an export. So, in Chiron, the "include" function copies items from the foreign module into the "moduleScope". Anything you want to publish, you have to put on the "module", or "this".
However, unlike in a constructor function, I also put the "module" object on the scope chain. This is a convenience that decreases the amount of dereferencing in Chiron code, but may have disadvantages if you're doing static analysis and compression. I'd be loathe to go back to my code and add a bunch of "module." dereferences for things that are conceptually global, like "forEach" which I define in a module, but I can.
So here's a layout of each of the objects in the scope chain that I think we can converge upon or around:
I think our specification should be exhaustive about what "must" be in the "builtins" object, but not necessarily what "must not". Simply, modules should not presume that anything else will be there. Since in a browser implementation, there's no way to mask out the "window" scope, we should not require it. To that end, it would be acceptable to simply use the native "global" in Rhino-based implementations as well, as long as we communicate what variables we guarantee to be available in cross-platform modules.
A word on persistence: "builtins" is global scope and the same object is available in any module. Every "module" has its own "moduleScope" and anonymous closure. To that end, whether functions like "include" and "log" are in "builtins" and "moduleScope" could be left as an exercise to the implementor. In Chiron, it's desirable to have them in "moduleScope" because it lets me "bind" a unique version of the "include" function to each "module", permitting me to divine the "moduleUrl" from "this.moduleScope" inside the definitive "include" function.
> In Helma NG, we have a simpler setup - there's just the module scope > as top level scope which has the "traditional" JS global object as its > prototype, and one module importing another don't share anything > except that common global object. This means that everything defined > in a module is public, i.e. visible to and accessible from the > importing module. The only way to implement a private variable scope > would have been to wrap a function declaration around the module code > at compile time, like this pseudo-code:
> We decided against this because a) it can be done by module writers if > private variables are required, and b) security and code isolation is > not too much a concern for Helma's main usage scenario (server side > apps by trusted authors).
And client-side security is impossible without server-side treatment and client-side checks like what the Caja folks are doing. There are other reasons for the anonymous function, which is implemented (almost) exactly as you posited to do in Helma NG. I already mentioned that it helps avoid the "imports become exports" problem. It also irons out some issues that crop up when you're using an object as both a context object and a member of the scope chain. Particularly, browsers that adhere to the latest JS specification put "var" declarations in the top most function-block-scope, not necessarily the top-most scope. This means that if you're inside of a function (as you inevitably are if you're using a JS-implemented module loader safely), and this function contains a "with" block with the object that's supposed to be the "module", when you use "var", those declarations are added in the closure, not the module. However, there are browsers that behave differently. So, the solution is to always use an anonymous enclosure inside of a "with" block; that makes it behave consistently. I envy environments like Helma NG and xjs where you have the opportunity to fiddle with the scope chain without resorting to such evil.
In summary, it's not possible to consistently reconstruct an environment like the global environment in a browser from native JavaScript. You cannot have the topmost member of the scope chain also be the context object. This isn't really a major problem in my experience: merely an opportunity to construct a more elegant environment.
> I'm not sure I understand this right, but in Helma NG we have an > "import x, y from z":
> Using this, methodX and methodY become properties of the importing > module, and when called "this" is bound to the importing module rather > than the one they're defined in. But maybe I misunderstood the > foreignModuleBind semantics.
I really like "importFromModule", and it is almost exactly like "include". I would enjoy implementing this. We could both go to the drawing board and do something more future-compatible and use destructuring. Here's a proposal:
require(moduleUrl): returns the foreign "module" object require(moduleUrl, structure): returns an object using the given structure to map names from the origin module to names in the returned object. include(moduleUrl): copies the all items from the foreign module into the "moduleScope" include(moduleUrl, structure): performs a require(moduleUrl, structure) and copies the items from the returned object into the "moduleScope" publish(object) copies items from the given object into the "module" publish(object, structure) uses the structure to rename items from the given object and copies those items to the "module"
The ES-body has been discussing destructuring. I think it's something like argument packing and unpacking in Python. Like:
I really like the variadic syntax you already support though. I also wouldn't want to do both; there's too much room for complex interactions. So, let me know what you think.
Also, I _could_ transform URL's to more Java-like and Python-like module.name's, but I'd rather not. There semantic space with URL's is much more expressive and while I don't plan to take advantage of their expressiveness for standard things, I expect there are situations where they will be handy. To that end, I propose the following site-package layout:
URL's beginning with a "." are resolved relative to the current "moduleUrl" using urlJoin. Other module's are resolved relative to the "moduleRootUrl" using urlJoin.
There are other issues about how to organize scripts, but they are more relevant to package management. I'll say now that I think there should be a well-managed set of modules that should be consistent across platforms, even if some of them are not available in some environments. For site-packages, I think they should be in directories corresponding to their origin domain name.
> So "include" evaluates the module in a separate scope and copies > properties into the calling scope? If that is so, it would be largely > equivalent to importFromModule("module", "*") in Helma NG.
Yeah, that's it exactly.
> Our importModule() returns the module scope, but additionally sets it > up as a property in the calling scope that defaults to the module name > but can be overriden using the second argument.
> Actually I started thinking that this is overkill and maybe we should > only return the module and not set it up in the calling scope, as it's > easy to do this on the importing side and would make implementation of > importModule simpler.
> var bar = importModule("foo.bar");
I'm not really a fan of recreating deeply nested module trees, but it would be onerous to go without them, I can accommodate them in Chiron. They also aren't conducive to URL mappings.
> I think the implementation details of how code is looked up and loaded > should be left to the environment, but it is important to settle for a > common import syntax and semantics that are part of the language.
I agree completely. I also think that we agree about what kinds of scope control are necessary to ensure portability. We just need to converge on the names of functions and their particular semantics.
> I'm just starting to look deeper into Chiron. The module > implementation is in modules.js, right?
<matthias.plat...@gmail.com> wrote: > So we need some common denominator for basic loading/importing > functionality to load rhino/javascript code. I couldn't find any docs > about your require() function, do you have any pointers. What is > require doing different from load()
> So maybe we (helma-ng, xjs, other rhino based environments using a > common package installing machanism) could agree on a common basic > load / import / require function, which just loads some .js code into > the current namespace. If i still would need to implement helma-ng > modules i would just need to write a wrapper module:
Absolutely. Let's make a spec.
> We would end up with modules/packages that can be loaded via a basic > load() statement, and other (helma-ng) specific modules, which just > work in helma-ng. The package installing mechanism should know about > this difference.
This would be a good compromise. I find that some modules can actually be written in such a fashion that they would work in either a global environment or a module environment, albeit with a lot of the ordinary cruft that is unnecessary in a module environment. With the solution I propose, a module like jQuery can be included as a module, as long as you replace two occurrences of the word "window" with "this". In a global browser load, "this" is the "window", so it's a non-issue. In Chiron's module loader, "this" is the module, so the impact is reduced to module scope.
> And some more words on helma-ng module loading mechanism: > I very much like the loadModule mechanism from helma-ng. Helma 1.x has > just one global environment / namespace. Especially with modules this > became a problem. I disagree with you, that an application environment > doesn't need to target those problems, but i fully understand your > point that in many cases you won't need this extra magic that > loadModule provides.
On Mon, Aug 25, 2008 at 7:08 PM, Kris Kowal <cowbertvon...@gmail.com> wrote: > One notion that Chiron's modules.js preserves is of strong parallelism > between the body of a module and the body of a prototype's constructor > function. In a constructor function, inner functions, vars, and lets > are effectively private because they are in the closure, and the > closure is distinct from the context object. So, in a module (which > usually maps 1 module for 1 file
"usually"? What is the syntax for two modules in one file and how does the importing module (say one of two modules in another file) work?
> "usually"? What is the syntax for two modules in one file and how does > the importing module (say one of two modules in another file) work?
> Peter
One of the use cases for Chiron modules involves bundling to optimize page load times and minimize HTTP requests. That's why the moduleScope contains a "register" function that allows you to set the module object or a module factory function for a given URL. A bundle script writes up the module semantic accouterments around each module's text, then "registers" these module factory functions. When a module next "requires" that module, "modules.js" calls the module factory instead of doing its usual script fetching. The build process reads a list of files that should be bundled with their dependencies and replaces the script with a file that contains all of its dependencies and then its own text at the end. Then the build script goes on to pack/minify/compress/gzip the resultant file.
Or, drawn out here, this would register a module factory with the original module's text elided. The order is unimportant and "modules.js" still assures that they are loaded in the same order they would be as multiple files.
register("module1.js", function () { with (this.moduleScope) { with (this) { (function () { … }).call(this); } } });
register("module2.js", function () { with (this.moduleScope) { with (this) { (function () { … }).call(this); } } });
On Mon, Aug 25, 2008 at 9:31 PM, Kris Kowal <cowbertvon...@gmail.com> wrote: > I envision this would be a handy bit of data to include in your > package description file, come to think of it.
Packages will have a list of dependencies in their meta-data file.
> Packages will have a list of dependencies in their meta-data file.
> Peter
That is distinct though, right? Isn't a package a collection of modules (sometimes only one) that the package manager downloads over the module root? By that reckoning inter-module dependencies are distinct from inter-package dependencies and serve different purposes. An inter-package dependency notifies the package manager that other packages must be downloaded into the module root from a repository at install-time, and an inter-module dependency notifies the module system that a module must be loaded at run-time. Inter-module dependencies are expressed through calls to importFromModule(), load(), require(), or include(), but does not necessarily mean that all of those modules can be concatenated for a single HTTP request.
On Mon, Aug 25, 2008 at 10:08 PM, Kris Kowal <cowbertvon...@gmail.com> wrote:
>> Packages will have a list of dependencies in their meta-data file.
>> Peter
> That is distinct though, right? Isn't a package a collection of > modules (sometimes only one) that the package manager downloads over > the module root?
I'm not sure why you write "over the module root". I don't know what "over" means.
I'll just assume the "module root" is something like "/usr/local/lib/xjs" where there are files like "File.js" or Assert.js" installed which can be loaded at runtime in a JavaScript program with "require('File')". I think it might be better to call the the files in "/usr/local/lib/xjs" by the name "library files" as there may not be a 1:1 file:language-module relationship.
> By that reckoning inter-module dependencies are > distinct from inter-package dependencies and serve different purposes.
Definitely different purposes.
> An inter-package dependency notifies the package manager that other > packages must be downloaded into the module root from a repository at > install-time,
Yes.
> and an inter-module dependency notifies the module > system that a module must be loaded at run-time.
Yes.
> Inter-module > dependencies are expressed through calls to importFromModule(), > load(), require(), or include(),
Yes.
> but does not necessarily mean that > all of those modules can be concatenated for a single HTTP request.
I don't know. That depends on language-specific details of whether or not a two language-level modules can exist in a single source file. This are details outside a package management system.
--------
The piece of the puzzle that interests me in sharing is the package management client and server software. I'm writing it anyway and thought if I talked with the Helma folks they might be able to use it too if it was sufficiently generic. It might be handy for module.js if it does not merge with Helma.
There is no concept of language-level modules in a package management client. A package management client is just worried about declared package dependencies in the package meta-data, copying files, deleting files, upgrading package versions, etc. As far as the package management client tools go the packages could be just full of plain text files for a novel (e.g. chapter 2 package depends on chapter 1 package.)
I've made a draft specification for interchangeable module loaders. This specification diverges a little bit from my current module loader implementation, mostly to entertain the idea of using the ES-Harmony destructuring idiom. I assume that there are changes that you guys would insist upon before converging on an implementation. I am committed to revising the module loader and refactoring my standard library according to whatever specification we converge upon.
If you guys have a Wiki where you want to play with these ideas, please send me a link. My Chiron Trac wiki is under siege until I get around to setting up Akismet.
On Sep 4, 12:53 pm, "Kris Kowal" <cowbertvon...@gmail.com> wrote:
> I've made a draft specification for interchangeable module loaders.
> This specification diverges a little bit from my current module loader
> implementation, mostly to entertain the idea of using the ES-Harmony
> destructuring idiom. I assume that there are changes that you guys
> would insist upon before converging on an implementation. I am
> committed to revising the module loader and refactoring my standard
> library according to whatever specification we converge upon.
Looks very nice, and pretty close to what we are doing. I'm still
amazed that this is possible on the browser using XmlHTTPRequest
(haven't done a lot of browser work obviously). I have to take some
time to look at your proposal and modules.js in detail, but I think we
could find common ground. Helma NG is only a few months old and there
aren't any major applications yet, so we're still quite flexible.
> If you guys have a Wiki where you want to play with these ideas,
> please send me a link. My Chiron Trac wiki is under siege until I get
> around to setting up Akismet.
On Thu, Sep 4, 2008 at 7:00 AM, Hannes Wallnoefer <hann...@gmail.com> wrote: > We have a home grown wiki at <http://dev.helma.org/ng/>. Feel free to > peruse and add to it.
Thanks Kris. Really interesting stuff. I find some of the ideas in
your proposal quite ingenious, but there are some bits I don't see the
benefit of.
In Helma NG we currently have importModule(moduleName, [importAs]) and
imortFromModule(moduleName, property, ...) which roughly correspond to
your require() and include(). Let's talk about naming first. I do have
a problem with our slightly bulky names, and I find your naming choice
quite good. What I don't like is the fact that two functions that
actually perform similar tasks have completely different names. I'm
not completely convinced here. What do others think?
The next thing is that importModule() currently sets the module object
in the calling scope using the module name or the optional second
"importAs" argument, without the need to assign it to a variable. I've
come to see that this approach is problematic, as it's actually less
readable than simply assigning the module object to a variable. And
you can't import a module without "polluting" the local module
namespace.
One thing I really like is the destructuring functionality where you
pass an array to select or an object to select and rename module
imports. Currently we are only able to select which properties to
import with importFromModule() but not rename them.
include("helma.file", {"File": "JSFile"}) to import helma.file.File as
JSFile is really nice and simple.
There are some things I don't fully understand. One is the exact
distinction between the module and the moduleScope. Is the moduleScope
the scope on which the module is originally evaluated and the module
the object the exported properties are copied to? If so, is the
moduleScope created only once and the module for each import/require?
I also take it that the moduleScope is reachable from the module
object via the moduleScope property.
In Helma NG we have a simpler setup - just what you call the builtins
layer and the module scope. Also, the builtins scope (or shared global
as we call it) is not the module scope's parent scope chain but in its
prototype chain. This makes it virtually impossible to pollute the
shared global object, even if you use variables without var
declaration etc.
On the other side of the scope chain, I can see that having an
anonymous scope for private variables is a nice thing. We thought
about doing this in Helma NG at one point by evaluating modules as
function closures like you do in modules.js. However we felt that this
was too much "magic behind the scenes", and that module authors could
easily do that on their own if they want it. Now I think I understand
you have to do it that way in order to get the magic to work in the
browser environment, but I at least prefer to not have any private
scope magic by default.
One final item I'd like to bring up is the foreignModuleBind()
functionality. I think I understand what it does, but I'm not really
sure what the use case would be. Can you give an example, or point me
to some code?
Oh, one more thing: while the module path and relational URL stuff
pretty much reflects what Helma NG does, like python or dojo we do use
"abstract" module names with the file extension removed and path
separators replaced with dots, e.g. "helma/file.js" becomes
"helma.file". While it's a minor detail, I've really come to like this
and would hesitate to give that up.
Hannes
On Sep 4, 8:49 pm, "Kris Kowal" <cowbertvon...@gmail.com> wrote:
> On Thu, Sep 4, 2008 at 7:00 AM, Hannes Wallnoefer <hann...@gmail.com> wrote:
> > We have a home grown wiki at <http://dev.helma.org/ng/>. Feel free to
> > peruse and add to it.
On Fri, Sep 5, 2008 at 2:02 PM, Hannes Wallnoefer <hann...@gmail.com> wrote: > What I don't like is the fact that two functions that > actually perform similar tasks have completely different names. I'm > not completely convinced here. What do others think?
This has always bothered me. "include" and "require" are bad names. I got together with Phil Holland, a former and brilliant co-worker when I was at Apple, to thoroughly review our options. In a sense my goal for the names came from the tripod of truth: coherence, correspondence, and pragmatism. They needed to look good together, have meanings that implied their function, and be brief enough to encourage use. We filled a white-board with names from precedented to outlandish including: "use", "import", "include", "require", "fetch", "load", "acquire", "procure", "adopt", "assume", "embrace", "usurp", &c.
use: could mean either include or require import: reserved keyword include: tends to imply incorporating names into yourself, so coherent and correspondent. require: only implies dependency. does not imply acquisition. fetch: only implies getting the module, but also implies that the module isn't cached. "fetch" would seem to be the part of "load" before "eval". load: already meaningful in rhino for "fetch and eval in global scope". acquire: good alternative to require, but "require" has precedent in languages like PHP adopt: Phil really liked this alternative to "include". it effectively implies copying the elements of another module. However, there is no precedent.
embrace, assume, and usurp are all in the same boat as adopt, also with no precedent except generally more absurd. procure is in the same boat as acquire.
I look at them as a pair: require: I am dependent on this module. (it would also be nice if it meant "I would like to get the module", but it doesn't) include: I want to incorporate this module into myself. (I feel this makes up the difference for the previous caveat since it subsumes the implication of require and more.)
> The next thing is that importModule() currently sets the module object > in the calling scope using the module name or the optional second > "importAs" argument, without the need to assign it to a variable. I've > come to see that this approach is problematic, as it's actually less > readable than simply assigning the module object to a variable. And > you can't import a module without "polluting" the local module > namespace.
I like how the "require" semantics solve this problem.
var module = require(moduleUrl); // import module var A = require(moduleUrl).a; // from module import a as A
Also, since these are in the anonymous closure, they don't implicitly export. Explicitly exporting them isn't much trouble either:
this.module = require(moduleUrl); // import/export module this.A = require(moduleUrl).a // from module import/export a as A
Also, since all of these objects are in scope, code that uses these imported values doesn't need to change if the author changes their mind about whether they want to export.
> There are some things I don't fully understand. One is the exact > distinction between the module and the moduleScope. Is the moduleScope > the scope on which the module is originally evaluated and the module > the object the exported properties are copied to? If so, is the > moduleScope created only once and the module for each import/require?
Both the module and the moduleScope are on the scope chain during the evaluation of a module. Anything on the module object is "public"; anything on the moduleScope object is "private". Both are created only once for each module file.
> I also take it that the moduleScope is reachable from the module > object via the moduleScope property.
I probably should explicate in the specification that it's important that the "include" function deliberately not copy the foreign module's moduleScope property into the current module's moduleScope. In my implementation, that could break the "require" function since it's dependent on "module.moduleScope.moduleUrl" to resolve the foreign module's URL.
> In Helma NG we have a simpler setup - just what you call the builtins > layer and the module scope. Also, the builtins scope (or shared global > as we call it) is not the module scope's parent scope chain but in its > prototype chain. This makes it virtually impossible to pollute the > shared global object, even if you use variables without var > declaration etc.
I really like this. We could reword the specification such that the scope chain I specified may be implemented with prototypical inheritance or scope-chain manipulation. You could use the same technique to construct a longer prototype chain.
var builtins = new Global(); this.setupModuleContext = function (moduleUrl) { function moduleScope () { this.include = bind(include, this); this.require = bind(require, this); … }; moduleScope.prototype = builtins; function Module () { this.moduleScope = moduleScope; moduleScope.module = this; }; module.prototype = moduleScope; var module = new Module(moduleUrl); return module;
};
You would still need to put an enclosure around the program to give you the anonymous innermost scope, I think.
> On the other side of the scope chain, I can see that having an > anonymous scope for private variables is a nice thing. We thought > about doing this in Helma NG at one point by evaluating modules as > function closures like you do in modules.js. However we felt that this > was too much "magic behind the scenes", and that module authors could > easily do that on their own if they want it. Now I think I understand > you have to do it that way in order to get the magic to work in the > browser environment, but I at least prefer to not have any private > scope magic by default.
It's my experience that any JavaScript developer that knows about enclosures _always_ wants one in every module, and if they don't need local variables, removing it is a late optimization. To this end, I always write my code inside enclosures that would work outside the enclosure if I don't end up using private variables. It boils down to boilerplate that makes my module files work the way I would expect them to in another language. I don't think of this as magic but rather a feature that beginner programmers take for granted, or would like to.
In any case, I'm pretty sure I wouldn't be able to make the client module loader work without some scope magic. In particular, I can't make var and function declarations implicitly export: that scope will always be inaccessible to me in at least one browser.
> One final item I'd like to bring up is the foreignModuleBind() > functionality. I think I understand what it does, but I'm not really > sure what the use case would be. Can you give an example, or point me > to some code?
I use foreignModuleBind in several places in Chiron. It's useful for any function that you want to resolve a URL relative to its calling context or inspect it's calling module. This is unfortunately not functionality that can be tacked on in a module; it has to have a hook in the module loader; I kept it out as long as I could. In particular:
The dir() method in base.js uses foreignModuleBind because, like the Python analog, when called with no arguments, it provides a list of all of the names that are in the current module and moduleScope.
I use it in base.js#type, base.sj#method, http.js#resolve, http.js#relative (which may become urllib.js#urlJoin and urllib.js#urlSplit), http.js#request, and test.js#assert. The assert method uses it to note what module the assertion was called in when it logs the result to a console.
> Oh, one more thing: while the module path and relational URL stuff > pretty much reflects what Helma NG does, like python or dojo we do use > "abstract" module names with the file extension removed and path > separators replaced with dots, e.g. "helma/file.js" becomes > "helma.file". While it's a minor detail, I've really come to like this > and would hesitate to give that up.
There are a couple reasons I haven't done this yet. It takes additional code to translate (and code is money on the client-side). Also, in a client environment, modules can conceivable be hosted by CGI scripts. I don't personally do this, but is a use-case I've seen. Some people like to do their bundling, compression, and caching on demand, and it's often easiest to just give the script a .php, .cgi, .py, .jsp, or whatever extension in some environments.
However, if it's a deal-breaker, I'd like to fully implement Python's "." prefix notation for module relative module's as well. I pretty much get this behavior for free (well, reusing urlJoin) when I use URL's though.
Summarily, these are mostly just counter arguments. I'm going to defer to you guys for all of the final value judgments. In particular, I will accept whatever names you propose for "require" and "include", whether you chose to standardize foreignModuleBind, the prototypical scope chain option, and whether you chose to go with abstract module names or URL's. The only thing that would be tricky for me is if you decided to make "var" and "function" declarations implicit exports; I'd need some help and an epiphany for that.
And, you should seriously consider this option despite my reservations because it has very real advantages. Using strictly anonymous functions makes stack traces
...
For what it's worth, this use of include and require will confuse the heck out of PHP users. In PHP both verbs mean, interpolate and execute the entire contents of this file right here at this line. The only difference is that if the file is missing include only warns but require throws a fatal error. I don't know how much you should weigh that confusion, just FYI.
On Fri, Sep 5, 2008 at 8:30 PM, Joshua Paine <midnightmons...@gmail.com> wrote: > For what it's worth, this use of include and require will confuse the > heck out of PHP users. In PHP both verbs mean, interpolate and execute > the entire contents of this file right here at this line. The only > difference is that if the file is missing include only warns but require > throws a fatal error. I don't know how much you should weigh that > confusion, just FYI.
Oh…yeah. I've been slowly forgetting PHP. The whole "require_once" thing, in particular, I've been *trying* to forget. I'm sure that this kind of cross-language nomenclature adoption problem is what pushed the current design choice toward multi-word alternatives. I hope it doesn't come to that.
Does anyone have any feelings about renaming "log" to "print"? I think I would like to do that in my implementation.
Also, Hannes, I presume you're the primary decision maker for this issue. I'm not very familiar with your team and its structure. Could you name some of the people you would like to hear feedback from? I'm not in any particular hurry, but that would help me gauge how close we are to locking down a spec so I can go refactor my library. I personally am only waiting to hear from Ihab on the Google Caja team who brought up the issue of modules on the ES3.1 discussion list. A modicum of reusability in Google Gadgets would be pretty exciting.
On Fri, Sep 5, 2008 at 2:02 PM, Hannes Wallnoefer <hann...@gmail.com> wrote:
[snip]
> The next thing is that importModule() currently sets the module object > in the calling scope using the module name or the optional second > "importAs" argument, without the need to assign it to a variable. I've > come to see that this approach is problematic, as it's actually less > readable than simply assigning the module object to a variable. And > you can't import a module without "polluting" the local module > namespace.
I agree. I like the explicit naming of the module object being imported in the module doing the importing.
> One thing I really like is the destructuring functionality where you > pass an array to select or an object to select and rename module > imports. Currently we are only able to select which properties to > import with importFromModule() but not rename them. > include("helma.file", {"File": "JSFile"}) to import helma.file.File as > JSFile is really nice and simple.
I like the syntax that was suggested on the es-discuss list
var mod = import(someModuleOrUrl);
and with the JavaScript 1.7 destructuring it is natural to import only some properties from the module.
var {a: a} = import(someModuleOrUrl);
It is not at all clear what "someModuleOrUrl" will mean yet for ECMAScript.
[snip]
> In Helma NG we have a simpler setup - just what you call the builtins > layer and the module scope. Also, the builtins scope (or shared global > as we call it)
I think "builtins" is confusing and "shared global" is much better; however, "shared global" seems slightly redundant as globals are shared. In the ES4 proposal the global object (the regular ES3 global object we all know as "this" in the global scope or "window" in the browser) was to be available through the global variable named "global" which was a really good idea, I thought, as it being available as "window" only makes sense in the browser. I hope "global" makes it into ES-Harmony.
[snip]
> One final item I'd like to bring up is the foreignModuleBind() > functionality. I think I understand what it does, but I'm not really > sure what the use case would be. Can you give an example, or point me > to some code?
foreignModuleBind makes me nervous. It seems similar to "with" and dynamic scoping, while possibly useful at times, are both very dangerous and can create code which is very difficult to understand but living without them is not difficult.
> Oh, one more thing: while the module path and relational URL stuff > pretty much reflects what Helma NG does, like python or dojo we do use > "abstract" module names with the file extension removed and path > separators replaced with dots, e.g. "helma/file.js" becomes > "helma.file". While it's a minor detail, I've really come to like this > and would hesitate to give that up.
In the client-side world it is important that "module" does not equal "file" as that creates many expensive HTTP requests.
On Fri, Sep 5, 2008 at 3:32 PM, Kris Kowal <cowbertvon...@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 2:02 PM, Hannes Wallnoefer <hann...@gmail.com> wrote: >> What I don't like is the fact that two functions that >> actually perform similar tasks have completely different names. I'm >> not completely convinced here. What do others think?
> This has always bothered me. "include" and "require" are bad names. > I got together with Phil Holland, a former and brilliant co-worker > when I was at Apple, to thoroughly review our options. In a sense my > goal for the names came from the tripod of truth: coherence, > correspondence, and pragmatism. They needed to look good together, > have meanings that implied their function, and be brief enough to > encourage use. We filled a white-board with names from precedented to > outlandish including: "use", "import", "include", "require", "fetch", > "load", "acquire", "procure", "adopt", "assume", "embrace", "usurp", > &c.
> use: could mean either include or require > import: reserved keyword > include: tends to imply incorporating names into yourself, so coherent > and correspondent. > require: only implies dependency. does not imply acquisition. > fetch: only implies getting the module, but also implies that the > module isn't cached. "fetch" would seem to be the part of "load" > before "eval". > load: already meaningful in rhino for "fetch and eval in global scope". > acquire: good alternative to require, but "require" has precedent in > languages like PHP > adopt: Phil really liked this alternative to "include". it > effectively implies copying the elements of another module. However, > there is no precedent.
> embrace, assume, and usurp are all in the same boat as adopt, also > with no precedent except generally more absurd. > procure is in the same boat as acquire.
> I look at them as a pair: > require: I am dependent on this module. (it would also be nice if it > meant "I would like to get the module", but it doesn't)
I think it does mean "get the module". It almost seems to demand it by the fact it is a requirement. Requirements are not optional, after all.
> include: I want to incorporate this module into myself. (I feel this > makes up the difference for the previous caveat since it subsumes the > implication of require and more.)
I think these are good justifications for the names "require" and "include". At least good enough. Some of the other candidates you list above are good too. In the end, no one word is going to describe such a complex thing as setting up a scope change and prototypes etc. Short is more important than a full description as the words would be used relatively frequently. It takes very little time to learn "cons", "car", and "cdr" and they almost mean nothing at all to the language semantics (they came from hardware terms, I believe.)
>> There are some things I don't fully understand. One is the exact >> distinction between the module and the moduleScope. Is the moduleScope >> the scope on which the module is originally evaluated and the module >> the object the exported properties are copied to? If so, is the >> moduleScope created only once and the module for each import/require?
> Both the module and the moduleScope are on the scope chain during the > evaluation of a module. Anything on the module object is "public"; > anything on the moduleScope object is "private". Both are created > only once for each module file.
>> I also take it that the moduleScope is reachable from the module >> object via the moduleScope property.
> I probably should explicate in the specification that it's important > that the "include" function deliberately not copy the foreign module's > moduleScope property into the current module's moduleScope. In my > implementation, that could break the "require" function since it's > dependent on "module.moduleScope.moduleUrl" to resolve the foreign > module's URL.
I find the use of "module" and "moduleScope" together confusing.
>> In Helma NG we have a simpler setup - just what you call the builtins >> layer and the module scope. Also, the builtins scope (or shared global >> as we call it) is not the module scope's parent scope chain but in its >> prototype chain. This makes it virtually impossible to pollute the >> shared global object, even if you use variables without var >> declaration etc.
> I really like this. We could reword the specification such that the > scope chain I specified may be implemented with prototypical > inheritance or scope-chain manipulation.
I don't think this should be optional. It should be one way or the other or both or none but not maybe like this or maybe like that.
[snip]
> It's my experience that any JavaScript developer that knows about > enclosures
Sorry for this nit pick but here goes...specs are about nit picking anyway...
In the wiki page you wrote up, the language seems to be just a little different than the "standard" language or use of language for discussing JavaScript that it really slowed down my reading.
For example,
"Modules MUST NOT write items to the builtins object"
What does "write items to" mean? All three words are not the normal language for object manipulation and lead to ambiguity. I think you mean something more along the lines of "Modules MUST NOT create new properties, modify the values of existing properties, or delete existing properties of the builtins object." This could be shortened to "Modules MUST NOT mutate the builtins object."
I only mention this because I felt myself stumbling along quite a bit as I translated the writing to the language I normally read when discussing JavaScript.
In this email "enclosures" made me wonder if you were talking about something slightly different than "closures" which is the usual word. With all this terminology flying around the appearance of different connotations is cause for pause.
That is the end of the nit pick.
> _always_ wants one in every module, and if they don't need > local variables, removing it is a late optimization. To this end, I > always write my code inside enclosures that would work outside the > enclosure if I don't end up using private variables. It boils down to > boilerplate that makes my module files work the way I would expect > them to in another language. I don't think of this as magic but > rather a feature that beginner programmers take for granted, or would > like to.
So you explicitly write the closure around your modules or your loader adds it automatically at load time?
[snip]
Although I personally don't see the desperate need for a module system or even what it fixes, I know that others do and that there will be more discussions in the es-discuss list about this topic so I'm trying to follow along here and there.
I think it would be good if the module.js/Helma NG spec starts by explaining exactly the problem the module system is solving.
For example, modules don't prevent namespace collisions. Instead they move where the namespace collisions occur. Instead of variable names colliding it is module names that can collide. In a sense it is the module names that are all in the same namespace. If file==module and modules are loaded by url then this isn't really a problem because urls are supposed to be unique; however, if multiple modules can be concatenated into a single file, which they need to be, then the modules names can easily collide when they register themselves with the module loader.
Another example, modules don't prevent a programmer from modifying the global object. All he has to write is setTimeout("require=function(){}", 100) or dynamically insert a script element and he can execute code in the global scope and cause a lot of trouble.
So I think a precise description of the need being successfully filled would be a good idea.
On Fri, Sep 5, 2008 at 8:30 PM, Joshua Paine <midnightmons...@gmail.com> wrote:
> For what it's worth, this use of include and require will confuse the > heck out of PHP users. In PHP both verbs mean, interpolate and execute > the entire contents of this file right here at this line. The only > difference is that if the file is missing include only warns but require > throws a fatal error. I don't know how much you should weigh that > confusion, just FYI.
I wouldn't worry at all about the PHP use of these words.
On Fri, Sep 5, 2008 at 11:34 PM, Peter Michaux <petermich...@gmail.com> wrote: > I find the use of "module" and "moduleScope" together confusing.
I'm just going to rattle off some ideas:
public, private exports, imports module, local
If we go with "local", the variable that was once known as "builtins" could be "global". Alternately they could be "globals", and "locals"; I have no idea which pair is most natural.
> Sorry for this nit pick but here goes...specs are about nit picking anyway...
And wikis are for fixing. I really have no idea how to write clearly; I tend to use obscure grammar, punctuation and vocabulary, and you've been following the JS standards body much more closely than I have lately. If you wouldn't mind putting your lawyer hat on and rewriting some prose, I think it would help a lot.
> existing properties of the builtins object." This could be shortened > to "Modules MUST NOT mutate the builtins object."
Or "modify".
> In this email "enclosures" made me wonder if you were talking about > something slightly different than "closures" which is the usual word. > With all this terminology flying around the appearance of different > connotations is cause for pause.
You caught me. I've been plugging this word for a while, writing it in public places and hoping that people would start passing it along as if it were common knowledge. My ultimate goal, I must say, is to find it defined in John Resig's new book without having to tell him about it. I plugged it in an arsicle a year ago:
Enclosure is subtly more than a closure. It, in fact, differs by exactly two characters: (). (get it!?) "enclsoure" is what i've been calling a closure that exists when you create an anonymous function and call it, just to enclose some local variables. I would use Christian Heilmann's term, Module Pattern, but that term encompasses a great deal more boilerplate code and practices than merely an enclosure.
To clarify my previous message, when I craft normal JavaScript modules to be included with <script> tags, I think programmers should always use an enclosure. It never hurts to have an enclosure and having an enclosure unifies and greatly simplifies the meaning of "var" statements in the topmost scope of a file and inside a function. The performance degradation and stack consumption is nominal. It forces people to explicitly add variables to the "window" or "globals", or use "this" instead of nothing or "var". Since they're using "this" to mean "shared scope", porting them to the module system is trivial, where shared scope simply drops to the "module". So, if you're always using an enclosure, it might as well be implicitly added by the module loader.
If you think it would be more clear to use a different word or phrase, I don't presume to own the wiki page. I just think that "enclosure" is enough like "closure" that most people will double-take once and infer its meaning, saving me the pain of writing "an anonymous function immediately called to produce a unique and isolated function block scope" repeatedly ;-)
On Sat, Sep 6, 2008 at 12:25 AM, Kris Kowal <cowbertvon...@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 11:34 PM, Peter Michaux <petermich...@gmail.com> wrote: >> I find the use of "module" and "moduleScope" together confusing.
> I'm just going to rattle off some ideas:
> public, private > exports, imports > module, local
> If we go with "local", the variable that was once known as "builtins" > could be "global". Alternately they could be "globals", and "locals"; > I have no idea which pair is most natural.
These are all better than "moduleScope" and the choice would be based on exactly what the module and moduleScope objects do in relation to existing uses of these words. I'm not totally clear on the nuances of these objects.
>> Sorry for this nit pick but here goes...specs are about nit picking anyway...
> And wikis are for fixing. I really have no idea how to write clearly; > I tend to use obscure grammar, punctuation and vocabulary, and you've > been following the JS standards body much more closely than I have > lately. If you wouldn't mind putting your lawyer hat on and rewriting > some prose, I think it would help a lot.
I think I would need to understand things a lot better than I do now. For example, had I seen "enclosure" I would have automatically changed it to "closure" and lost some of your intended meaning.
> I just think that "enclosure" > is enough like "closure" that most people will double-take once and > infer its meaning, saving me the pain of writing "an anonymous > function immediately called to produce a unique and isolated function > block scope" repeatedly ;-)
This is what I was thinking it must be but was still unsure and had to ask causing even more discussion. I think it is worth defining all non-standard terminology (i.e. terms not used in the ES3 spec.)
I've made the following changes to the spec draft:
* global instead of builtins * local instead of moduleScope * memoize instead of cache (wouldn't want you guys to think it should get culled when it grew) * print instead of log * properties instead of items (I had hoped to beg a distinction between the two, but such doesn't exist clearly in JavaScript) * access, modify, and replace instead of read, write, and overwrite
I also got rid of a couple phrases with "in the transitive closure of". I really like the mathematical precision of the phrase, but "accessible through references from" is probably good enough for our purposes.
> On Fri, Sep 5, 2008 at 2:02 PM, Hannes Wallnoefer <hann...@gmail.com> wrote:
>> One thing I really like is the destructuring functionality where you >> pass an array to select or an object to select and rename module >> imports. Currently we are only able to select which properties to >> import with importFromModule() but not rename them. >> include("helma.file", {"File": "JSFile"}) to import helma.file.File as >> JSFile is really nice and simple.
> I like the syntax that was suggested on the es-discuss list
> var mod = import(someModuleOrUrl);
> and with the JavaScript 1.7 destructuring it is natural to import only > some properties from the module.
> var {a: a} = import(someModuleOrUrl);
> It is not at all clear what "someModuleOrUrl" will mean yet for ECMAScript.
I like this too. And I just discovered that there is a destructuring assignment shorthand feature [1] in Firefox/JS 1.8 that lets you simplify this to:
So it looks like we could really have import, import-from, import-from-as functionality all in one simple function, just using a, {a} or {a: b} on the left hand side.
Of course include or import-all functionality can't be modeled using destructuring assignment. But I'm starting to think that maybe we should decompose that into two steps: first importing the module and then augmenting the local scope with it.
var m = importModule(someModule); augment(this, m);
What do you think? Of course the requirement on JS 1.8 or at least 1.7 puts this in the remote future for cross-browser implementations. Rhino currently supports 1.7, but I think the step to 1.8 won't be that big a step.
On Sat, Sep 6, 2008 at 12:29 PM, Hannes Wallnoefer <hann...@gmail.com> wrote: > Of course include or import-all functionality can't be modeled using > destructuring assignment. But I'm starting to think that maybe we > should decompose that into two steps: first importing the module and > then augmenting the local scope with it.
> var m = importModule(someModule); > augment(this, m);
It's a good idea, but "include" is by far the most common use-case in every module I've written. It catches all situations except when there's a name-conflict among dependency modules. Perhaps with the two-argument forms of require and include, there won't be as much need for an "from module import *" pattern, but if we went this route, I think we would see a lot of code like:
var base = import("base"); adopt(base); var browser = import("browser"); adopt(browser); var events = import("events"); adopt(events); var environment = import("environment"); adopt(environment);
In all of these cases, it is superfluous to bind a variable in local scope too.
I think that we would need either de-structuring assignment before we got away from the need for a "require and adopt" function call, and we won't have that universally on the client-side for a few more years at least.
> So it looks like we could really have import, import-from, > import-from-as functionality all in one simple function, just using a, > {a} or {a: b} on the left hand side.
I think this is really exciting too. We can create an easy migration path for this feature.
import module var module = require("module"); // current javascript with our module loaders var module = import("module"); // future javascript with the native module loader
from module import a var a = require("module").a // current javascript with our module loaders include("module", ["a"]); // plus the de-structuring feature we've slated for the include and require functions include("module", {a}); // future javascript with self-mapping object shorthand var {a} = require("module"); // shifted to the de-structuring form var {a} = import("module"); // shifted to the native module loader
from module import a, b include("module", ["a", "b"]); // current javascript with our module loaders include("module", {"a", "b"}); // plus the self-mapping object shorthand include("module", {a, b}); // at any point, we can remove the extraneous quotes, but we don't have to var {a, b} = require("module"); // plus the de-structuring feature var {a, b} = import("module"); // shifting to the future, native module loader
from module import a as A var A = require("module").a; include("module", {'a': 'A'}); var {'a': 'A'} = require("module"); var {'a': 'A'} = import("module");
from module import * include("module"); // current javascript with our module loaders importAll("module"); // not sure how or whether they'll do this in future javascript
from module import a as A, b as B include("module", {"a": "A", "b": "B"}); var {"a": "A", "b": "B"} = require("module"); var {"a": "A", "b": "B"} = import("module");
In all of these cases, we could create really simple refactoring tools to automatically migrate code. The most complex transformation is between the common include(module, structure) form and the var strucure = import(module) form. That's childsplay to automate.
On Sat, Sep 6, 2008 at 12:29 PM, Hannes Wallnoefer <hann...@gmail.com> wrote:
[snip]
> I like this too. And I just discovered that there is a destructuring > assignment shorthand feature [1] in Firefox/JS 1.8 that lets you > simplify this to:
This handy short cut was mentioned in the es-discuss list too.
> So it looks like we could really have import, import-from, > import-from-as functionality all in one simple function, just using a, > {a} or {a: b} on the left hand side.
I like how succinct it is and that is it is just a natural use of a feature that is in JavaScript and hopefully will be in ECMAScript. There is nothing new to learn.
> Of course include or import-all functionality can't be modeled using > destructuring assignment. But I'm starting to think that maybe we > should decompose that into two steps: first importing the module and > then augmenting the local scope with it.
> var m = importModule(someModule); > augment(this, m);
Depends how "this" is going to work if it is both in the prototype and scope chain or not.
I'm far more interested in being able to write things like
var mod = import(someModule); var {a: a} = import(someModule); var {a, b} = import(someModule);
It is reasonably easy to write a cross browser version of the above three lines. Destructuring is just sugar.
var mod = import(someModule);
var mod = import(someModule); var a = mod.a; var b = mod.b; mod = null;
var a = import(someModule).a;
Sure this could be bulky in some cases but the semantics are identical which is the most important part.
On Sat, Sep 6, 2008 at 6:34 PM, Peter Michaux <petermich...@gmail.com> wrote:
> On Sat, Sep 6, 2008 at 12:29 PM, Hannes Wallnoefer <hann...@gmail.com> wrote:
> [snip]
>> I like this too. And I just discovered that there is a destructuring >> assignment shorthand feature [1] in Firefox/JS 1.8 that lets you >> simplify this to:
> This handy short cut was mentioned in the es-discuss list too.
>> So it looks like we could really have import, import-from, >> import-from-as functionality all in one simple function, just using a, >> {a} or {a: b} on the left hand side.
> I like how succinct it is and that is it is just a natural use of a > feature that is in JavaScript and hopefully will be in ECMAScript. > There is nothing new to learn.
>> Of course include or import-all functionality can't be modeled using >> destructuring assignment. But I'm starting to think that maybe we >> should decompose that into two steps: first importing the module and >> then augmenting the local scope with it.
>> var m = importModule(someModule); >> augment(this, m);
> Depends how "this" is going to work if it is both in the prototype and > scope chain or not.
> I'm far more interested in being able to write things like
> var mod = import(someModule); > var {a: a} = import(someModule); > var {a, b} = import(someModule);
> It is reasonably easy to write a cross browser version of the above > three lines. Destructuring is just sugar.
> var mod = import(someModule);
> var mod = import(someModule); > var a = mod.a; > var b = mod.b; > mod = null;
> var a = import(someModule).a;
> Sure this could be bulky in some cases but the semantics are identical > which is the most important part.