This is essentially the same in purpose as:
http://wiki.commonjs.org/wiki/Modules/Transport/B
Or generally, the http://wiki.commonjs.org/wiki/Modules/Transport effort.
If you would like, please make a concrete Transport/C proposal. My
only tip at this point is that the module factory function should
receive require, exports, and module in a single object as the first
and only argument of the module factory, since this will make it
possible to transport DSL's that use additional free variables.
Kris Kowal
--
You received this message because you are subscribed to the Google Groups "CommonJS" group.
To post to this group, send email to comm...@googlegroups.com.
To unsubscribe from this group, send email to commonjs+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commonjs?hl=en.
Yeah, Transport/B is designed to meet these requirements. Of course,
it's somewhat incomplete. There are a few naming choices to nail it
down, but that pretty much is it.
Kris Kowal
This we need to be cautious about. I do not think that we should
require or even encourage a system where any module has the authority
to add a module factory to a running module system from within that
system. Module registration could be run-time, but it should be done
outside the module system, preferably before any modules get loaded by
the module system. We don't want to run into problems where modules
are not executed because they are not available yet, or races to
register a module, where the last module to be registered at the time
that the module is required wins.
Would anyone be against solidifying the naming issues in Transport/B
and putting it up for a hand show? I think it would make a good basis
for further development. Particularly, we might want to create an
extension to the spec for the URL API for module transport retrieval
to satisfy Wes's requirement 3 (which is orthogonal but cohesive to
Transport/B), and eventually an extension for creating modules from
Strings to satisfy one of Irakli's requirements, which could be easily
added to Transport/B in another revision.
Kris Kowal
I just made a Transport/C:
http://wiki.commonjs.org/wiki/Modules/Transport/C
Main difference with Transport B:
* use positional arguments, this allows for less hand-coded typing, so
it could be hand-coded by front end developers. It also less
indentation since the factory function is two levels shallower than
Transport B.
* Makes the "require", "exports" and "module" explicit dependencies if
they are used in the factory function, and allows dependencies to be
passed as named arguments to the factory function. "require",
"exports" and "module" should be optional since not all modules need
them, and explicitly specifying them as dependencies and function
arguments makes it easier to transform existing CommonJS modules to
this format.
* Avoids deepDependencies, not clear of its value. Better to leave it
off to make the spec simpler.
Transport/C is something I could support in RunJS very quickly, so we
could have a good async, script-tag based implementation for the
browser fairly quickly. While I still favor allowing a return value
from the factory function to define the module's exported value, that
is specified as an optional part of Transport/C. If that needed to be
cut to get Transport/C approved, I can live with that, but will likely
support it in my implementation.
James
Thanks, I like that this melds our two proposals. I will give this
some diligent thought.
Some thoughts:
* require.def is fine by me.
* the dependency array names the arguments and binds them to other
dependency. I like that you've made "require" and "exports"
dependencies that are injected by the module factory caller. I have
to think about what kinds of patterns this would lead us toward. One
concern with the approach though is that it seems that it conflates
"dependency" and "module object". Is the intent that "require" and
"exports" would somehow be modules? Or, does this system suppose that
all modules are "makers" in the "e-maker" [1] sense into which the
dependee must inject the transitive dependencies of the depended
module? How are modules/dependencies instantiated with this system?
Kris Kowal
[1] http://wiki.erights.org/w/index.php?title=Emaker
[2] http://wiki.ecmascript.org/doku.php?id=strawman:modules_emaker_style
[3] http://wiki.ecmascript.org/doku.php?id=strawman:modules_primordials
The factory function in Transport/C is different from the "makers in
e-maker", in the sense that Transport/C functions are only called
once, by the loader environment, to define the module, not for each
require() done for that dependency. I assumed this is also how
CommonJS modules worked today, since the spec only specifies free
variables allowed are require, module and exports. However, the module
spec does not mention allowing other free variables. Maybe it was
written that way to mean they either can or cannot be there, but I
read it as there would not be other free variables.
I view the factory functions as scripts in the browser that had a bit
of an enclosed environment, and allowed dependencies to be loaded
async, but the factory functions would then be evaluated like scripts
loaded in the browser, once, in the right dependency order.
Conflating dependency and module object:
How I understood the current CommonJS module spec, dependencies and
modules are the same? Doing a require("alpha") means that I depend on
alpha existing and the return value for the require call is the
"alpha" module, defined by what alpha.js exports. Maybe these are
different, but I read the "exports is always an object" discussion as
meaning that we would not see code like require("alpha")("some",
"args"). Maybe I misunderstood?
> Is the intent that "require" and "exports" would somehow be modules?
Based on what I wrote above on dependency vs module object, I am not
sure it matters if they are modules. What tradeoffs do you see if they
are or are not?
> How are modules/dependencies instantiated with this system?
Taking this example:
TODO_X("alpha", ["require", "exports", "beta"], function (require,
exports, beta) {
exports.verb = function() {
return beta.verb();
//Or:
return require("beta").verb();
}
});
"alpha" says it depends on "require", "exports" and "beta". The loader
knows what require and exports map to, and makes sure to load "beta".
When it calls "alpha"'s factory function, it passes require and
exports to the function, and the value for the beta argument is
equivalent of doing a require("beta") and passing that as the third
arg to the factory function.
This assumes as I said above, that the loader already called "beta"'s
factory function and cached that export value. require("beta") then in
this case is just giving back that cached export value.
If beta is a circular dependency, then the third arg to "alpha"'s
factory function will be undefined. "alpha"'s factory function is free
to call require("beta") at some later point inside the contents of the
factory function and may get back the exported value for "beta".
James
Alright, I think that these are notionally compatible, there are only
possible problems in the technical minutae.
"exports", "require", and "module" need to be unique for each module.
It's my understanding that every argument named for the module factory
is a singleton module. That constitutes a contradiction.
If:
function (require) {
var exports = require("exports");
}
is equivalent to:
function (require, exports) {
}
Then "exports" would have to be a singleton module object, not the
exports object of this module. There are similar problems for the
"module" module and "require" module.
I like the direction that this is going, but I don't think that the
notions of scope injection can be the same as "singleton module
injection". I do think that "require", "exports", and "module" could
be injected in the manner you propose, but there is a paradoxical loop
here if "require", "exports", and "module" are also modules
themselves. They would have to be special-cased for this to work.
Kris Kowal
I am fine with saying that "require", "exports" and "module" are
special-cased, in that require("exports"), require("require") and
require("module") are not allowed (can throw). I think this is fine to
say since those are also called out as explicit free variables in the
module spec too.
James
I converted the RunJS code to use the Transport/C syntax:
http://github.com/jrburke/runjs/tree/require
I also have a basic conversion script that converts normal CommonJS
modules to Transport/C:
http://github.com/jrburke/runjs/blob/require/build/convert/convertCommonJs.js
I ran it over the narwhal/lib files and the narwhal/tests files, and
created a test file here:
http://github.com/jrburke/runjs/blob/require/build/convert/narwhal-test.html
If you want to reproduce, follow the instructions in that test page
HTML. All the tests do not pass, but I believe it is because the test
is out of sync with the uri module. For instance the test checks that
a parsed url.fragment is null, but I cannot see where the uri module
references "fragment" at all in its source, so url.fragment returns
undefined instead of null.
There are some things with converting "./" and "../" paths, but I
think it works out. You can see in the test page where some modules
needed to have very explicit path mappings to have it all work out --
cannot really do a search path sort of thing in the browser, so really
have to configure one location for each module.
But the important thing is that the modules loaded and were called in
correct dependency order, and the test was able to run, and execute
the uri module's code. I had do to a bit of work to set up some
modules so the tests would run in the browser, (those files are in
build/convert/commonjs), but as far as a module loader, it seems to
work. I tried it in Firefox and Webkit. It does not work in IE because
at least one of the modules use __defineGetter__.
So I am encouraged by the results, and I am happy to use this instead
of the RunJS syntax and evangelizing it for use in browser-based
toolkits. I can switch my work project, Mozilla Messaging's Raindrop,
to this loader too. Note that this require.js supports returning a
value from the factory function, to allow things like function
constructors as module values, and it is has a plugin architecture to
allow things like i18n and text file dependencies. More details in the
README:
http://github.com/jrburke/runjs/blob/require/README.md
So if this group can reach agreement on this transport syntax, and are
OK with this implementation, then I am happy to take it further and
push it more for browser toolkits.
James
To be clear, I think that this is great. RunJS appears to have some
real momentum in some circles. I however can't stand behind
Transport/C because I think that in the very long run, it is a
mistake. It mashes two very distinct approaches to module loading
together in a way that is *almost* perfect, but not quite right.
Conflating the module name space and the module scope would be an
error. The thing I like the most about Transport/C is that it
resembles the E-Maker approach to module loading, which I think we
should explicitly support in the long term future. E-Makers are
lower-level than modules and permit the *caller* to inject free
variables into the scope of the function. An E-Maker could easily
compile to a form where the free variables are explicitly named in an
array and positionally injected into the function (the E-Maker caller
convention would be to pass an object with corresponding keys). In a
sense, the E-Maker approach would combine with the RunJS approach by
having the entity instantiating a Maker function pass the module memo
explicitly. The essential difference is that the object is *only* a
scope injection object, not necessarily the module memo. I think it's
close.
See:
http://wiki.ecmascript.org/doku.php?id=strawman:modules_primordials
http://wiki.ecmascript.org/doku.php?id=strawman:modules_emaker_style
Kris Kowal
On Tue, Jan 26, 2010 at 11:55 PM, Kris Kowal <cowber...@gmail.com> wrote:
> On Tue, Jan 26, 2010 at 9:57 AM, James Burke <jrb...@gmail.com> wrote:
>> I converted the RunJS code to use the Transport/C syntax:
>> http://github.com/jrburke/runjs/tree/require
>
> To be clear, I think that this is great. RunJS appears to have some
> real momentum in some circles. I however can't stand behind
> Transport/C because I think that in the very long run, it is a
> mistake. It mashes two very distinct approaches to module loading
> together in a way that is *almost* perfect, but not quite right.
Do you believe the current CommonJS module format will support the
E-Maker approach? At first read I do not think it does, but I am still
digesting E-Makers.
My goal was to try to get something that would work similar to the
existing behavior of the CommonJS module format, but something that
worked in the browser via script injection.
> Conflating the module name space and the module scope would be an
> error.
"name space" to me is unclear in this context. Could you expand on
this a bit more vs module scope?
> The thing I like the most about Transport/C is that it
> resembles the E-Maker approach to module loading, which I think we
> should explicitly support in the long term future. E-Makers are
> lower-level than modules and permit the *caller* to inject free
> variables into the scope of the function. An E-Maker could easily
> compile to a form where the free variables are explicitly named in an
> array and positionally injected into the function (the E-Maker caller
> convention would be to pass an object with corresponding keys). In a
> sense, the E-Maker approach would combine with the RunJS approach by
> having the entity instantiating a Maker function pass the module memo
> explicitly. The essential difference is that the object is *only* a
> scope injection object, not necessarily the module memo. I think it's
> close.
Could you expand what you mean by module memo? Sorry, I am still
learning all the lingo.
How would you change Transport/C to be more in line with E-Makers?
For the example, there are many use cases where there are multiple
dependencies.
Would it be better if the example were
var dependencies = ['beta','zeta','theta'];
TODO_X("alpha", ["require", "exports", dependencies], function
(require, exports, dependents) {
exports.verb = function() {
return beta.verb();
//Or:
return require("beta").verb();
}
});
where TODO_X is passed an array of module ids and calls the callback
with a dependents
object which looks like {beta:beta,zeta:zeta,theta:theta}
Kam
On Jan 27, 9:50 am, James Burke <jrbu...@gmail.com> wrote:
> Thanks for the feedback! This is useful.
>
> On Tue, Jan 26, 2010 at 11:55 PM, Kris Kowal <cowbertvon...@gmail.com> wrote:
You can get that same effect now by using require() inside the function:
TODO_X("alpha", ["require", "exports", "beta", "zeta", "theta"],
function(require, exports) {
exports.verb = function() {
return require("beta").verb();
}
});
This also works well because you do not have to work out a way to
convert module names, which can contain front slashes, into some
property on a dependents object. If you just use the module name for
the property name, then you are just trading some square brackets for
parentheses, and a function call. Does not seem worth introducing more
moving parts, more things to specify in a spec.
James
Answering myself, I apologize. I believe Kris Kowal is probably in
transit and probably cannot respond at the moment, but I think I may
have understood the difference between Transport/C and e-maker style
that he thinks may be a problem in the long run. I'll try to
demonstrate via an example:
In emaker style, a "util/point" may be coded as such in something that
looked like Transport/C:
TODO_X("util/point", function(x, y) {
return {
getX: function() { return x; },
getY: function() { return y; },
setX: function(x_) { if (x_ > 0) { x = x_; } },
setY: function(y_) { if (y_ > 0) { y = y_; } }
};
}
and the function(x,y) would be called for each call to "util/point":
var point = require("util/point")(5, 10);
In the Transport/C, non-emaker style it would look like so:
TODO_X("util/point", function() {
return function(x, y) {
return {
getX: function() { return x; },
getY: function() { return y; },
setX: function(x_) { if (x_ > 0) { x = x_; } },
setY: function(y_) { if (y_ > 0) { y = y_; } }
};
};
}
In Transport/C the function associated with "util/point" is only
called once, to set up the module for util/point, in this case the
module is defined by returning a function that accepts the x and y
values (this assumes a return value is a valid export value -- if you
do not like that, assume exports is passed to the function and a
exports.makePoint is defined with that function. The use of
Transport/C's util/point would be the same:
var point = require("util/point")(5, 10);
(or var point = require("util/point").makePoint(5, 10); if you prefer
exports style)
So, emaker, the function associated with util/point is called for
every require call, where Transport/C only calls that function once.
Kris, please correct me if I am wrong.
If that is the concern, then I do believe that normal CommonJS modules
written today will have the same problem as Transport/C -- they are
assuming that they are only evaluated once and their exported value is
reference from then on?
In other words, regular CommonJS modules would need to change if
emaker style was supported by some future ECMAScript spec, but only if
multiple evaluation of the module text was seemed as hazardous. At the
very least, it just seems like the Transport/C functions may be called
more than once, but that change could be made in the loader
implementation. I can see Transport/C still working with emaker
support?
If I got this right, I do not think that concern should block
Transport/C, as Transport/C would be affected by the spec the same way
normal CommonJS modules would be affected. It also assumes a strawman
proposal for a future ECMAScript makes it through the spec process and
makes it through unchanged.
If anyone else has issues with Transport/C it would be good to know
soon. I would like to try to get it approved otherwise.
> In emaker style, a "util/point" may be coded as such in something that
> looked like Transport/C:
>
> TODO_X("util/point", function(x, y) {
> return {
> getX: function() { return x; },
> getY: function() { return y; },
> setX: function(x_) { if (x_ > 0) { x = x_; } },
> setY: function(y_) { if (y_ > 0) { y = y_; } }
> };
>
> }
> and the function(x,y) would be called for each call to "util/point":
>
> var point = require("util/point")(5, 10);
>
I like the eMaker idea (the example above is really just a factory,
btw, although I guess there are other possiblities).
However, as the eMaker page (http://wiki.ecmascript.org/doku.php?
id=strawman:modules_emaker_style) alludes to in the "Accessible shared
static module objects" section, the syntax above has problems when you
want to have shared constant data (the sine/cosine example), shared
variable data (ex: a registry module), or even I think just when you
want to define a class (or a hierarchy of classes):
> TODO_X("util/circle", function(x, y, rad) {
> return new Circle(x, y, rad);
> ...
(how do I define the Circle class once, rather than every time a
circle is instantiated?)
It seems like the eMaker proposal is depending on additions to
ECMAScript to make stuff like that possible. Did I miss something?
> In the Transport/C, non-emaker style it would look like so:
>
> TODO_X("util/point", function() {
> return function(x, y) {
> return {
> getX: function() { return x; },
> getY: function() { return y; },
> setX: function(x_) { if (x_ > 0) { x = x_; } },
> setY: function(y_) { if (y_ > 0) { y = y_; } }
> };
> };
>
> }
>
> In Transport/C the function associated with "util/point" is only
> called once, to set up themodulefor util/point, in this case themoduleis defined by returning a function that accepts the x and y
> values (this assumes a return value is a valid export value -- if you
> do not like that, assume exports is passed to the function and a
> exports.makePoint is defined with that function. The use of
> Transport/C's util/point would be the same:
That syntax makes more sense to me since it allows the eMaker style of
programming but also allows things like shared data.
While that may work when the function callback is called in the
browser, it does not hold up if that function callback later does some
sort of setTimeout sort of thing, or later-reference to one of those
free variables:
function(){
exports.foo = function() {
if (module.id == "something") {
exports.bar = "something";
}
}
}
In this case, foo will get added to the correct export, but later, if
something calls foo(), the module and exports free variables are
likely to be some other values (the last module's function callback
values, not necessarily the ones needed for foo).
In the larger context, related to e-maker and the primodials strawmen,
I do not understand the usefulness of a generic free variable system
-- the free variables like the ones used for require module loading,
qunit test functions or things like jake tasks assume your module has
opted in to those free variables.
For require module loading, that seems like a base construct, so that
can be handled as a default environment setup. Although it gives
preference to the require style of module definitions.
However, for other free variable type of injection, like qunit and
jake tasks, it seems like my module would at least have to do a
require("qunit") or require("jake") to indicate that I want to
interact with its free variables. If I have done that already, then I
do not see much gain over just saying var q = require("qunit"), then
use things like q.assert().
If a require() is not needed to get qunit's free variables, then that
just sounds like using globals. Seems like specifying how to add and
access global properties needs to be defined vs a system for free
variable injection. At least in the browser, this is straightforward.
So, I am skeptical that working out a way to inject free variables is
generally useful vs. just specifying a way to do globals, or just
encouraging namespacing via var q = require("qunit'), q.assert(). A
generic free variable injection system still has the possibility for
namespace collisions (compared to globals), and does not make the code
any easier to understand/read, to figure out when looking at a code
block where a variable comes from.
What is the case for free variable injection in the general case? It
just seems like a way to make the typing on an author slightly easier
(although not easier if there are globals), but still complicates code
structure, when the code is looked at by humans.
James
I have updated the Transport/C proposal to have more specific
information, TODO_X is now specified as require.def() and some other
parts have been cleaned up. I also went with describing the function
as a "module definition function" instead of a "factory function" to
avoid possible misunderstandings with any overlap with e-maker
modules.
As for the feedback so far on the proposal, I believe the module
definition function is affected the same way as normal CommonJS
modules as far as an e-maker spec/factory behavior. As for wanting a
generic way to specify free variables for other modules outside the
basic module spec, I do not see a way to do this myself, and I have
not seen a description of how it might work for a transport format
that needs function wrappers and works in the browser.
I have converted RunJS to RequireJS:
http://github.com/jrburke/requirejs
and it implements the Transport/C proposal. I am using it now in a
real project, Mozilla's Raindrop.
I am not sure how the process should work now in light of the
"ratification" thread, but I would like to move Transport/C to the
next phase. I will be pushing RequireJS for use by other browser-based
toolkits.
James
James Burke wrote:
> If that is the concern, then I do believe that normal CommonJS modules
> written today will have the same problem as Transport/C -- they are
> assuming that they are only evaluated once and their exported value is
> reference from then on?
The way I (being far from expert on the subject) interpreted the
previous rounds of specs, notably
http://docs.google.com/View?docid=dfgxb7gk_34gpk37z9v
was that modules were constructed once per sandbox.
I haven't fully digested the specs at the ecmascript wiki yet, but
it seems they are opening up for both construct-once and -always.
Best regards
Mike