SecureableModules question

17 views
Skip to first unread message

Robert Cerny

unread,
Feb 8, 2009, 3:26:01 AM2/8/09
to serv...@googlegroups.com
Would it be possible to replace the exports object by a provide
function which is conceived the same way as require? The code would
look like the following snippets. I think this is better readable,
because at the first glance i see what a module has to offer. If not,
which property of the system would be lost?

Thanks in advance and best regards,

Robert


--- signature.js

provide("signature", signature);

function signature(func, returnType) {
// Implementation of signature
}
signature(signature, "undefined", "function", "any", "any");

--- end


--- interceptor.js

provide("intercept", intercept);
provide("Interceptor", Interceptor);

var signature = require("signature").signature;

function intercept(object, func, interceptor) {
// Implementation of intercept
}
signature(intercept, "undefined", "function", Interceptor);

function Interceptor() {
// implementation of Interceptor
}
signature(Interceptor, Interceptor);

--- end

ihab...@gmail.com

unread,
Feb 8, 2009, 10:52:10 AM2/8/09
to serv...@googlegroups.com
On Sun, Feb 8, 2009 at 12:26 AM, Robert Cerny <robert...@gmail.com> wrote:
> Would it be possible to replace the exports object by a provide
> function which is conceived the same way as require? ...

>
> provide("signature", signature);
>
> function signature(func, returnType) {
> // Implementation of signature
> }
> signature(signature, "undefined", "function", "any", "any");

I take it you are taking advantage of the hoisting of named function
initialization. This is a good idea. However, it is a special case of
a general module that may do:

var foo = exports.foo = 500;

(function() {
var bar = exports.bar = function() { ... };
})();

var k;
for (k in someObj) { exports[k] = otherObj[k]; }

Conversely, with the "exports.something" syntax, you could still do:

exports.foo = foo;
exports.bar = bar;

function foo() { ... }
function bar() { ... }

Does that work?

Ihab

--
Ihab A.B. Awad, Palo Alto, CA

Robert Cerny

unread,
Feb 8, 2009, 12:43:51 PM2/8/09
to serv...@googlegroups.com
On Feb 8, 2009, at 4:52 PM, ihab...@gmail.com wrote:

> Conversely, with the "exports.something" syntax, you could still do:
>
> exports.foo = foo;
> exports.bar = bar;
>
> function foo() { ... }
> function bar() { ... }
>
> Does that work?

That would work, too. And that is something i will still do, if i
cannot convince you guys, because i really like the idea of
secureable modules. Here are some of the arguments for using a
function:

1. i do not want to put my fate in the hands of the assignment
operator (the implementation cannot be changed (or only with a lot of
work))
2. same 1. in other words: a method can always be intercepted or the
implementation changed, the semantics and implementation of the
assignment is fixed
3. one could add a third optional parameter to the function "provide"
which determines the scope and consequently who is allowed to use the
provided product (by default the product is provided for anyone), e.g.
provide("somename", somefunction, "protected");
or
provide("somename", somefunction, "module1", "module2");
4. it is easy to explain to users coming from any programming
language: "Listen, you have two functions one for providing
"services" and one for requiring them!" In a little bit this would be
the most natural thing for people, not even requiring to know of what
is going on in the background (it took me a while to figure that out,
i did not get a lot of sleep last night :-)
5. the tempering with require.install on the wiki page could be
replaced by different implementations of "provide"

Here is some more (rather silly ones):
* if require is a function then why is exports not?
* the signature looks cleaner, both parameters are functions
* require could (with some compromise) be replaced by an object
called "imports", then it would be symmetrical the other way

To be honest i do not even think it is necessary to come to an
agreement, although i would very much like to! I think the most
important thing is a shared understanding of "require". Whether one
uses the exports approach or the provide approach could be abstracted
away.

Anyway apart from all my objections: this is a fabulous idea!

What i still haven't figured out is one little thing: how does my
module get attached in the module namespace?

Kind regards,

Robert

ihab...@gmail.com

unread,
Feb 8, 2009, 10:07:17 PM2/8/09
to serv...@googlegroups.com
Hi Robert,

On Sun, Feb 8, 2009 at 9:43 AM, Robert Cerny <robert...@gmail.com> wrote:
> Here are some of the arguments for using a function:

Ah so in your mind it is the use of a function vs. an assignment that
is the main distinction. Ok.

> 1. i do not want to put my fate in the hands of the assignment
> operator (the implementation cannot be changed (or only with a lot of
> work))

This is a very good point. E.g., in general, for understandability of
a modular system, once a module has exported symbol "foo", we do not
want it to modify the value of that symbol. So we always intended this
to be illegal:

exports.foo = function () { ... };
exports.foo = function () { ... };

Using a function instead of an assignment does indeed give us a place
to stand to enforce this and other restrictions.

> 2. same 1. in other words: a method can always be intercepted or the
> implementation changed, the semantics and implementation of the
> assignment is fixed

Right.

> 3. one could add a third optional parameter to the function "provide"
> which determines the scope and consequently who is allowed to use the
> provided product (by default the product is provided for anyone), e.g.
> provide("somename", somefunction, "protected");
> or
> provide("somename", somefunction, "module1", "module2");

Ok -- I'm not sure I would use *that* security mechanism exactly, but
in general, your point is well taken. Another example could be
providing an expression with metadata about the exported symbol, like:

provide("somename", function() { ... }, "docstring");

> 4. it is easy to explain to users coming from any programming
> language: "Listen, you have two functions one for providing
> "services" and one for requiring them!" In a little bit this would be
> the most natural thing for people, not even requiring to know of what
> is going on in the background (it took me a while to figure that out,
> i did not get a lot of sleep last night :-)

That is a good point, though I must stress that there are three things:

a. Requiring services (loading modules)
b. Using environment symbols (the "require.env" stuff)
c. Providing services (exporting symbols to other modules)

It is outside of the scope of this thread why I consider (b) so
importantly distinct from (a), but it has to do with how to build a
securable system. If we believe our systems will mostly use (a) and
(c), then you are correct.

> 5. the tempering with require.install on the wiki page could be
> replaced by different implementations of "provide"

I don't understand this but reasons 1 thru 4 seemed mighty good to me so far.

> Here is some more (rather silly ones):
> * if require is a function then why is exports not?

Right. What I would have said up till now is that it's because require
is a hook into the loader and sandbox mechanism and has lots of stuff
going on under the covers. But your points 1 and 2 above deal with
that.

> * the signature looks cleaner, both parameters are functions

Cool.

> * require could (with some compromise) be replaced by an object
> called "imports", then it would be symmetrical the other way

Not as I understand it so far -- again, I'd be happy to elaborate.

> What i still haven't figured out is one little thing: how does my
> module get attached in the module namespace?

Let me restate to make sure I understand. You made your module and put
it somewhere. Someone loads:

require("com.cerny.robert.Useful");

Your question is: how does the name "com.cerny.robert.Useful" get
mapped to wherever it is you stashed your module. This is the job of
the Loader in our design. Having thus partitioned the problem, we have
not yet solved it ;) but there are many solutions possible. It seems
serverjs is going towards a PATH-like solution, which sounds
reasonable. I have more ideas about this, though, which I'd be happy
to expand on in a separate thread.

Robert Cerny

unread,
Feb 9, 2009, 3:23:56 AM2/9/09
to serv...@googlegroups.com
Hi Ihab,

On Feb 9, 2009, at 4:07 AM, ihab...@gmail.com wrote:

> This is a very good point. E.g., in general, for understandability of
> a modular system, once a module has exported symbol "foo", we do not
> want it to modify the value of that symbol. So we always intended this
> to be illegal:

Require could clone[1] the module before it returns it. Then the
assignment is not illegal, but it has no effect on others.

>> 3. one could add a third optional parameter to the function "provide"
>> which determines the scope and consequently who is allowed to use the
>> provided product (by default the product is provided for anyone),
>> e.g.
>> provide("somename", somefunction, "protected");
>> or
>> provide("somename", somefunction, "module1", "module2");
>
> Ok -- I'm not sure I would use *that* security mechanism exactly, but
> in general, your point is well taken. Another example could be

For sake of simplicity, i would also leave it out. But library
authors will ask for that, because once you provide something, you
are stuck with it. If i want some privacy, e.g. because i do not want
to expose all implementation detail, but just a very tiny bit, i only
have the option of putting everything in one script, which might mean
duplicating code. Not having some form of friendship between modules,
makes API design more complicated than in should be, and it is
already complicated enough. But i can live very well with "everything
you provide is public". It is just harder for library users to
distinguish products and byproducts.

> a. Requiring services (loading modules)
> b. Using environment symbols (the "require.env" stuff)
> c. Providing services (exporting symbols to other modules)
>
> It is outside of the scope of this thread why I consider (b) so
> importantly distinct from (a), but it has to do with how to build a
> securable system. If we believe our systems will mostly use (a) and
> (c), then you are correct.

I could not find anything about require.env. Can you point me to some
explanation? Is it possible to do it with another function passed in:

var a = env("somesymbolname");

>> What i still haven't figured out is one little thing: how does my
>> module get attached in the module namespace?
>
> Let me restate to make sure I understand. You made your module and put
> it somewhere. Someone loads:
>
> require("com.cerny.robert.Useful");
>
> Your question is: how does the name "com.cerny.robert.Useful" get
> mapped to wherever it is you stashed your module. This is the job of
> the Loader in our design. Having thus partitioned the problem, we have
> not yet solved it ;) but there are many solutions possible. It seems
> serverjs is going towards a PATH-like solution, which sounds
> reasonable. I have more ideas about this, though, which I'd be happy
> to expand on in a separate thread.

My question is: how do i declare where my module is located? I would
expect a module declaration as the very first line of my module
(similar to a java package declaration):

module("cerny.interception");

This would make it 4 functions that one needs:
require, provide, module, and env

Kind regards,

Robert

Kris Kowal

unread,
Feb 9, 2009, 5:12:50 AM2/9/09
to serv...@googlegroups.com
CAST (in order of appearance)

On Mon, Feb 9, 2009 at 12:23 AM, Robert Cerny <robert...@gmail.com> wrote:
> On Feb 9, 2009, at 4:07 AM, ihab...@gmail.com wrote:

> Require could clone[1] the module before it returns it. Then the
> assignment is not illegal, but it has no effect on others.

This is, in fact, often necessary for other optional loader features
like a "curry the id of the module in which this function is called"
decorator. It's a good idea for require to return a shallow copy in
general, or at least specify that a module loader might do so. The
only downside is that people using "this" in module "methods" (which
you shouldn't do) would be surprised to find that the imported module
has unique state for every module it's required in. So, we should
probably explicate that conformant modules do not use "this" in their
exported functions. They can close on "exports" for shared data
instead.

> I could not find anything about require.env. Can you point me to some
> explanation? Is it possible to do it with another function passed in:
>
> var a = env("somesymbolname");

There isn't much reason for or against using a function instead of an
object tree for the environment. The only reason for hanging "env"
off of "require" was to avoid adding another implicit "special" name
to the module scope, but I would be okay personally with having an
"env" or "environment" variable, analogous to "global", "imports", or
"exports". The reason there's no information on the "env" object as
yet is that we haven't gotten to the point where it's generally
accepted. However, it will eventually be necessary to acquire
capabilities from outside a group of modules' sandbox, and since we
forbid free variables, some mechanism like "env" will be necessary.
One alternative is to conflate the name spaces of modules and
capabilities and just use "require" to get injected dependencies, but
that's considered unclean by the security folks, who like to
conceptually separate inert module factory functions and objects that
communicate power and vulnerability.

>>> What i still haven't figured out is one little thing: how does my
>>> module get attached in the module namespace?

The module namespace is a mapping that the "require" function in each
module from the same sandbox closes on. The sandbox adds your module
exports object to that mapping and then calls your module factory
function with itself and your exports the first time another module
requires yours. We haven't introduced the concept of a sandbox for
the purpose of interoperable scripts yet since it's one of many ways
to implement a module loader, and it's fairly easy to bootstrap a
sandbox inside a module to load de-privileged modules. We would
however like to make sure that a "require.loader" with a particular
interface does get added to the specification at some point so secure
sandboxes can reuse the host module system's module factory functions
without having to fetch them. All of this stuff is in Ihab's and my
proposal to TC39, which you can see here:

http://docs.google.com/Doc?id=dfgxb7gk_34gpk37z9v&hl=en

Or the presentation here:

http://docs.google.com/Presentation?docid=dcd8d5dk_0cs639jg8&hl=en

> My question is: how do i declare where my module is located? I would
> expect a module declaration as the very first line of my module
> (similar to a java package declaration):
>
> module("cerny.interception");

A declaration method ("module" in this example) is not necessary or
desirable. The loader is required to establish a mapping between
module identifiers and files, albeit through a catalog, file paths,
URL's, or whatever the implementation requires. By this means, the
identifier of a module is the canonical identifier through which it
was found. Autonomy is not a desirable feature for a module because
a.) it can lie and b.) it can be wrong. Not requiring a module to
explicate its identifier makes a module tree easier to move in its
containing module identifier name space.

Great points though, really. I'm personally content with the
"exports" object and the convention that a "provide" utility is simple
to implement in its terms (I'd use a "provide" infer the name from the
Function.name to reduce some verbosity) The initial reason for going
with the "exports.foo = " notation is to provide a simple, textual
migration path to the eventual "export foo = " syntax that we are
promoting for a future version of the language. It's also easy to use
a bulk object update with the "exports" object if, for example, a
single module is a façade for more discrete modules.

Kris Kowal

Robert Cerny

unread,
Feb 9, 2009, 6:27:40 AM2/9/09
to serv...@googlegroups.com
Hi Kris,

On Feb 9, 2009, at 11:12 AM, Kris Kowal wrote:

>> I could not find anything about require.env. Can you point me to some
>> explanation? Is it possible to do it with another function passed in:
>>
>> var a = env("somesymbolname");
>
> There isn't much reason for or against using a function instead of an
> object tree for the environment.

Yes, there is. The same one as with the case for providing products
with a function. The implementation of a function can be changed,
e.g i can quickly add a line into the body of env, which logs every
call to a certain file.

>> My question is: how do i declare where my module is located? I would
>> expect a module declaration as the very first line of my module
>> (similar to a java package declaration):
>>
>> module("cerny.interception");
>
> A declaration method ("module" in this example) is not necessary or
> desirable. The loader is required to establish a mapping between
> module identifiers and files, albeit through a catalog, file paths,
> URL's, or whatever the implementation requires. By this means, the
> identifier of a module is the canonical identifier through which it
> was found. Autonomy is not a desirable feature for a module because
> a.) it can lie and b.) it can be wrong. Not requiring a module to
> explicate its identifier makes a module tree easier to move in its
> containing module identifier name space.

That sounds great. But what if:

vendor module A requires module B as "module1"
vendor module C requires module D as "module1"

my application requires module A and module C.

Would that still work? And if so why?

> Great points though, really. I'm personally content with the
> "exports" object and the convention that a "provide" utility is simple
> to implement in its terms (I'd use a "provide" infer the name from the
> Function.name to reduce some verbosity)

Yes, good idea. A second optional parameter is still necessary
though, because i might export something else than a function or one
function under multiple names.

Anyway, i think secureable modules is a great idea. A shared
understanding of "require" is more essential to them than the way
that the modules provide their products, which is defined in the
context of require. A convention would be very good though.

Best,

Robert

Kevin Dangoor

unread,
Feb 9, 2009, 9:46:55 AM2/9/09
to serv...@googlegroups.com
On Sun, Feb 8, 2009 at 3:26 AM, Robert Cerny <robert...@gmail.com> wrote:
>
> Would it be possible to replace the exports object by a provide
> function which is conceived the same way as require? The code would
> look like the following snippets. I think this is better readable,
> because at the first glance i see what a module has to offer. If not,
> which property of the system would be lost?

FWIW, Python uses

__exports__ = ["foo", "bar", "baz"]

which I think is more readable, if the goal is being able to see at a
glance what a module has to offer. (mind you "__" is not particularly
pretty...)

It is unfortunate that JS doesn't offer a way to change the assignment
behavior of an object, but so it goes. That said, if the behavior of

exports.foo = 1

needed to be changed, that could be changed in the loader mechanism
before the object is returned to the caller.

If a future ES may very well be doing something like

export foo = 1

that's a fairly compelling reason to use the existing proposed syntax.

Kevin


--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

Kris Kowal

unread,
Feb 9, 2009, 4:12:18 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 3:27 AM, Robert Cerny <robert...@gmail.com> wrote:
> On Feb 9, 2009, at 11:12 AM, Kris Kowal wrote:
> vendor module A requires module B as "module1"
> vendor module C requires module D as "module1"
>
> my application requires module A and module C.
>
> Would that still work? And if so why?

The "as" clause binds the exported value in local scope, and the
"from" clause is bound in the module identifier name space.

Python:
module A: import B as module1
module C: import D as module1
Transitional (our proposal):
module A: var module1 = require("B");
module C: var module1 = require("D");
Future ES (our TC39 proposal):
module A: import "B" as module1
module C: import "D" as module1

In all of these cases "module1" is scoped in the module and thus not
shared between the modules, giving each module "sovereignty" over its
local/private name space and orthogonally separating the "module
identifier name space".

Question for the group at large: what are your reactions to having an
"environment" function (instead of the as-yet-not-ratified
"require.env" object tree), and what are your reactions to using
"provide(foo) | provide(name, foo)" instead of "exports.foo =". Would
it be sufficient for "provide" to be implemented in "user-space"? If
so it would be desirable to have it in a module, and if that, it would
be desirable for a function to be able to request to the module system
that it receive, through a curry, the calling module's mutable exports
object. Also, bearing in mind that would solidify the point that
modules in a sandbox would be vulnerable to each other and the only
way to get security isolation would be to instantiate a depriviledged
sandbox with a constrained environment, which is okay since the inert
module factory functions can be shared.

Kris Kowal

Peter Michaux

unread,
Feb 9, 2009, 5:08:16 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 1:12 PM, Kris Kowal <cowber...@gmail.com> wrote:

> Question for the group at large: what are your reactions to having an
> "environment" function (instead of the as-yet-not-ratified
> "require.env" object tree),

I don't think we need the concept of require.env here. For our
purposes here, what is the use case that can not be covered by
requiring a module?

Waldemar Horwat noted the same in the following email to the es-discuss group.

https://mail.mozilla.org/pipermail/es-discuss/2009-January/008711.html

"- Why have different mechanisms for getting definitions from the
environment and from imported modules? Why not make everything into a
module? Part of the argument for separating them is that modules are
compute-only and cannot do i/o, but we may also have prepopulated
modules that do have i/o capabilities. The argument against
separating them is that it makes versioning of things in the
environment hard; implementations may wish to use external modules to
replace environment functionality on older browsers that don't have
the functionality built-in to the environment. Currently this is done
via monkey-patching, which is not possible here."

It seems to me that everything can be a module. Certainly I've made
File available as a module with Rhino. It was no problem at all.


> and what are your reactions to using
> "provide(foo)

That wouldn't be enough information. There is no name associated with
the exported object. An module doing the importing would not have
anything to call that object to get a reference to it.

| provide(name, foo)" instead of "exports.foo =".

I prefer "exports.foo =" because it gives a way for modules to
communicate easily by mutating the exports object. The module
exporting foo can also see the resulting change because "exports"
works as a pointer to a pointer. Some modules may freeze their exports
object before the module has finished executing, other modules might
not.

Peter

ihab...@gmail.com

unread,
Feb 9, 2009, 5:24:48 PM2/9/09
to serv...@googlegroups.com
Replying to the easy part first:

On Mon, Feb 9, 2009 at 2:08 PM, Peter Michaux <peterm...@gmail.com> wrote:
> I prefer "exports.foo =" because it gives a way for modules to
> communicate easily by mutating the exports object. The module
> exporting foo can also see the resulting change because "exports"
> works as a pointer to a pointer. Some modules may freeze their exports
> object before the module has finished executing, other modules might
> not.

Presumably, one could write "provide(...)" to do all these things too.
I'm not yet grokking the foundational difference you are pointing at.

Peter Michaux

unread,
Feb 9, 2009, 5:33:40 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 2:24 PM, <ihab...@gmail.com> wrote:
>
> Replying to the easy part first:
>
> On Mon, Feb 9, 2009 at 2:08 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> I prefer "exports.foo =" because it gives a way for modules to
>> communicate easily by mutating the exports object. The module
>> exporting foo can also see the resulting change because "exports"
>> works as a pointer to a pointer. Some modules may freeze their exports
>> object before the module has finished executing, other modules might
>> not.
>
> Presumably, one could write "provide(...)" to do all these things too.

It would be much more difficult than it is with the exports object.

> I'm not yet grokking the foundational difference you are pointing at.


// Module A

exports.foo = 1;

exports.getFoo = function() {
return exports.foo;
};

// Module B

var a = require('A');
a.foo = 2;

// Module C

var a = require('A');
a.foo = 3;

// Program

require('B')
require('C')
require('A').getFoo(); // 3

In the last step of the program, the reason getFoo can return 3 is
because Module A has a way to get at the object being changed in
modules B and C.

----------

For contrast consider the complexity of implementing the same variable
sharing with something like provide(). If provide is augmenting some
hidden "exports" object, how will module A actually get at the most
recently set value of foo that modules B and C have been setting? If
module A will continue to access through "exports.foo" then it makes
sense that it exports by assigning to that property. If other modules
are setting that foo value by assignment to that property then it
makes sense that module A should be able to do so also.

Peter

Kris Kowal

unread,
Feb 9, 2009, 5:40:37 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 2:33 PM, Peter Michaux <peterm...@gmail.com> wrote:
> exports.foo = 1;
>
> exports.getFoo = function() {
> return exports.foo;
> };

This is a very good point in favor of "exports" that I missed. Thanks, Peter.

Kris Kowal

Kris Kowal

unread,
Feb 9, 2009, 6:01:22 PM2/9/09
to serv...@googlegroups.com
[+list]

On Mon, Feb 9, 2009 at 1:20 PM, <ihab...@gmail.com> wrote:
> On Mon, Feb 9, 2009 at 1:12 PM, Kris Kowal <cowber...@gmail.com> wrote:

>> Question for the group at large: what are your reactions to having an
>> "environment" function (instead of the as-yet-not-ratified
>> "require.env" object tree), and what are your reactions to using
>> "provide(foo) | provide(name, foo)" instead of "exports.foo =".
>

Apologetically, my previous argument may have been unclear and muddied
by bad sentence structure and free association. Let me state again:

Options:
1. hang "provide" off of the arguments to a module factory function
2. write "provide" in any module you wish to use it instead of exports.
3. write "provide" in a module that any module can import if they
would prefer it instead of exports.
4. none of these. stick with "exports"

If 2 or 3, "provide" must close on the "exports" of the module in
which it's called.
If 3, the module system must alternately provide a means for the
exported "provide" function to close on the exports of the module into
which it is imported.
If 4, perl -pe 's/exports.([^ ]+) /export ($1) /' would be sufficient
to migrate compliant modules to proposed, future, native ES syntax.

Kris Kowal

ihab...@gmail.com

unread,
Feb 9, 2009, 6:08:36 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 2:40 PM, Kris Kowal <cowber...@gmail.com> wrote:
> This is a very good point in favor of "exports" that I missed.

True.

That all said, I have so far been assuming that the "exports" object
is [ to be considered ] frozen [ and that maybe even double
assignments to the same exported symbol are not allowed in order to
make reasoning about modules easier ].

A couple of constraints we threw around were:

1. The module system is free to make "shadow copies" of module
instances. In other words, in general, both within and across
different require-ing modules:

require('foo.js') !== require('foo.js')

2. The module system guarantees that modules cannot rewrite their
exported symbols. Again, both within and across require-ing modules:

require('foo.js').bar === require('foo.js').bar

This sort of follows the "improve understandability" thing but the
question is, *why* are these *necessary*? Maybe they are not. Here is
the logic:

#1 -> This is probably pretty useless. We could do without this extra
degree of freedom. But I'm worried about painting ourselves into a
corner.

#2 -> This is useful due to the desugaring of the "export" statement
we've discussed in ES Harmony. Specifically:

export foo = 3;
...
print(foo);

desugars to:

const foo = exports.foo = 3;
...
print(foo);

Now, if we let clients of the module change "exports.foo", this would
lead to pretty weird and unexpected results. The alternative would be
to simply desugar to:

exports.foo = 3;
...
print(exports.foo);

which would solve the problem. And a module wishing to protect its
exports could do:

Object.prototype.freeze(exports);

So I guess maybe there *is* no reason to cling to #1 and #2. Hm.

Thoughts?

Kris Kowal

unread,
Feb 9, 2009, 6:39:25 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 2:33 PM, Peter Michaux <peterm...@gmail.com> wrote:
> // Module A
>
> exports.foo = 1;
>
> exports.getFoo = function() {
> return exports.foo;
> };
>
> // Module B
>
> var a = require('A');
> a.foo = 2;

But this last line is decidedly not good. One of our stipulations in
"Security (2)" on
https://wiki.mozilla.org/ServerJS/Modules/SecurableModules is that the
object returned should not be tampered with and, if it is, it is not
guaranteed to do what you expect. For one, I think we really do need
these freedoms:

1. require(A) !== require(A)
2. require(A).X !== require(A).X

The first requirement enables a module loader to return a snapshot of
the exports of the requested module instead of its literal exports
object. For secure implementations, this means that the "require"
method may freeze the imports. The second requirement enables a
module loader to rebind or curry an exported function based on a
request. These are not things I expect that every implementation
would provide, but they are capabilities I think module loaders should
be permitted to provide.

Kris Kowal

Peter Michaux

unread,
Feb 9, 2009, 7:38:29 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 3:08 PM, <ihab...@gmail.com> wrote:
>
> On Mon, Feb 9, 2009 at 2:40 PM, Kris Kowal <cowber...@gmail.com> wrote:

> That all said, I have so far been assuming that the "exports" object
> is [ to be considered ] frozen [ and that maybe even double
> assignments to the same exported symbol are not allowed in order to
> make reasoning about modules easier ].

Double assignment not allowed? Seems like Erlang. If double assignment
is not allowed then I think assignment syntax may not be appropriate.
I would guess that almost everyone looking at the exports object in
the proposal would assume it is a JavaScript object (meaning the
Object as specified in ES) and that it is not a host object with its
own special assignment behaviors.

With regard to the exports object being frozen here is a circular
dependency example.

// Module A
exports.a0 = function(){};
var B = require('B');
exports.a1 = function(){};

// Module B
exports.b0 = function(){};
var A = require('A');
exports.b1 = function(){};

// Program
require('B');

Module B's exports object cannot be actually frozen when module A
requires module B. If it was frozen then exports.b1 could not be
assigned. I suppose there could be such a thing as unfreeze but there
isn't now and it would be nice (perhaps read "nice" as "necessary") if
the standard was implementable now. If module B wanted exports.b0
frozen it could use defineProperty from ES3.1 when exports.b0 is first
assigned.

JavaScript doesn't have any concept of permissions on an object: this
module can modify this object but that other module cannot modify the
same object. There are a couple possible ways to approach adding
permissions to the exports object.

If the object returned by require('B') is not module B's "exports"
object, then module A could not assign to module B's "exports" object.
However, this would require something like the "exports" object being
a host object that has a "onPropertySet" event. That event could be
used so that properties added, modified or removed in module B could
be seen on the object returned by "require('B')". This seems far too
complex and would lead to slow implementations.

Permissions could be added by using functions to get and set objects
exported. Module B would have the setter function. All modules
importing Module B would only receive the getter function. But this
doesn't achieve actual "safety".

// Module A
var a = function(){};
a.b = 1;
setExport("a", a);

// Module B
var getExport = require('A'); // returns the getter
getExport("a").b = 2;

Now module B hasn't the ability to mutate which objects are exported
by A but module B can mutate the properties of the objects exported by
A. In this case B has reached into A and modified a.b.


> A couple of constraints we threw around were:
>
> 1. The module system is free to make "shadow copies" of module
> instances. In other words, in general, both within and across
> different require-ing modules:
>
> require('foo.js') !== require('foo.js')

This does not seem consistent with the idea that a module is a
singleton. The singleton idea is easy to grasp.


> Now, if we let clients of the module change "exports.foo", this would
> lead to pretty weird and unexpected results. The alternative would be
> to simply desugar to:
>
> exports.foo = 3;
> ...
> print(exports.foo);
>
> which would solve the problem. And a module wishing to protect its
> exports could do:
>
> Object.prototype.freeze(exports);

If the module won't be used in some circular dependency system then it
would be fine to put the line above at the bottom of the file.
Otherwise then properties need to be frozen one by one, although that
does not prohibit other models from adding other properties (no
permissions system in JavaScript).

Peter

Peter Michaux

unread,
Feb 9, 2009, 7:45:51 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 3:39 PM, Kris Kowal <cowber...@gmail.com> wrote:
>
> On Mon, Feb 9, 2009 at 2:33 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> // Module A
>>
>> exports.foo = 1;
>>
>> exports.getFoo = function() {
>> return exports.foo;
>> };
>>
>> // Module B
>>
>> var a = require('A');
>> a.foo = 2;
>
> But this last line is decidedly not good. One of our stipulations in
> "Security (2)" on
> https://wiki.mozilla.org/ServerJS/Modules/SecurableModules is that the

The security section is not part of the main proposal for the purposes
of this group. It is simply a "to be compatible..." note.

> object returned should not be tampered with and, if it is, it is not
> guaranteed to do what you expect. For one, I think we really do need
> these freedoms:
>
> 1. require(A) !== require(A)

Not consistent with the idea a module is a singleton.

> 2. require(A).X !== require(A).X

What about the following? true or false?

require(A).x.y === require(A).x.y

> The first requirement enables a module loader to return a snapshot of
> the exports of the requested module instead of its literal exports
> object.

If it provides a snapshot then in the circular dependency issue, what
if more exports are added later. Things won't work as the programmer
expects.

> For secure implementations, this means that the "require"
> method may freeze the imports. The second requirement enables a
> module loader to rebind or curry

I don't think you mean "curry".

> an exported function based on a
> request. These are not things I expect that every implementation
> would provide, but they are capabilities I think module loaders should
> be permitted to provide.

I think things are getting too complicated both mentally to grasp and
implementationally.

Peter

Tom Robinson

unread,
Feb 9, 2009, 8:22:59 PM2/9/09
to serv...@googlegroups.com
On Feb 9, 2009, at 3:01 PM, Kris Kowal wrote:
>
> If 4, perl -pe 's/exports.([^ ]+) /export ($1) /' would be sufficient
> to migrate compliant modules to proposed, future, native ES syntax.

Maybe I shouldn't be doing this, but I've been inlining the exports
with an assignment to a local variable for convenience within the
module, i.e.

var Foo = exports.Foo = function() { ... }

which I suspect would would break...

var Foo = export Foo = function() { ... }

would it be better to do something like this:

var Foo = function() { ... }

...

exports.Foo = Foo

Or should we not worry about this?

Kris Kowal

unread,
Feb 9, 2009, 8:28:33 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 5:22 PM, Tom Robinson <tlrob...@gmail.com> wrote:
>
> Maybe I shouldn't be doing this, but I've been inlining the exports
> with an assignment to a local variable for convenience within the
> module, i.e.
>
> var Foo = exports.Foo = function() { ... }
>
> which I suspect would would break...
>
> var Foo = export Foo = function() { ... }
>

I would contend that yours is good style and ought to be encouraged
(unless we can get that implicit locally bound "exports").

> Or should we not worry about this?

Shhhhhh. We have at least two years to come up with a better regex ;-)

Or this is a good argument _for_ implicit locally bound exports (with
(exports) in the factory function), in which case I would like to
escalate this to a problem that must get as much attention as possible
toward that resolution ;-)

Kris

Kris Kowal

unread,
Feb 9, 2009, 8:43:52 PM2/9/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 4:38 PM, Peter Michaux <peterm...@gmail.com> wrote:
> On Mon, Feb 9, 2009 at 3:08 PM, <ihab...@gmail.com> wrote:
>> On Mon, Feb 9, 2009 at 2:40 PM, Kris Kowal <cowber...@gmail.com> wrote:

> Double assignment not allowed?

There currently are no clauses forbidding double assignment on
exports, and Ihab and I had an aside and now agree that there do not
need to be normative clauses in the specification banning double
assignment. It's dangerous, but sometimes desirable, especially for
lazy initializer functions.

> With regard to the exports object being frozen here is a circular
> dependency example.
>
> // Module A
> exports.a0 = function(){};
> var B = require('B');
> exports.a1 = function(){};
>
> // Module B
> exports.b0 = function(){};
> var A = require('A');
> exports.b1 = function(){};
>
> // Program
> require('B');
>
> Module B's exports object cannot be actually frozen when module A
> requires module B. If it was frozen then exports.b1 could not be
> assigned. I suppose there could be such a thing as unfreeze but there
> isn't now and it would be nice (perhaps read "nice" as "necessary") if
> the standard was implementable now. If module B wanted exports.b0
> frozen it could use defineProperty from ES3.1 when exports.b0 is first
> assigned.

To solve this problem, it is sufficient for a secure module loader to
copy the exports of the remote module, freeze that object, and return
it each time it is requested. This is why module loaders should be
free to return a copy of the remote object's exports, which means
that: require(A) is not necessarily the same object as require(A)
later. It is not the only good reason though. Some module loader
implementors (a.k.a., Kris and like-minded folks) would probably like
to allow a module to request that certain functions get curried with
the module id of the module in which they were called, or similar
features, with extensions that would not be required but permitted by
the specification, for which a module can feature-test like "if
(require.curryId) …". I support simplifying the specification as much
as possible, but these are features I would prefer not to forbid.

> JavaScript doesn't have any concept of permissions on an object: this
> module can modify this object but that other module cannot modify the
> same object. There are a couple possible ways to approach adding
> permissions to the exports object.

The Caja paper mentions an idea called "brands", where everyone who
has permission to call a function accepts a "sealed" object that
closes on a flag that can be used to test whether the requesting
module has legitimately received permission to call the function.
This is a "really cool" permission model that you can implement
entirely in user-space.

It's in the Caja paper at http://code.google.com/p/google-caja/ around page 13.

My conclusion is that the module system does not need to provide an
inherent mechanism for "private", "protected", "public" exports.
Brands are better and don't need specification.

>> 1. The module system is free to make "shadow copies" of module
>> instances. In other words, in general, both within and across
>> different require-ing modules:
>>
>> require('foo.js') !== require('foo.js')
>
> This does not seem consistent with the idea that a module is a
> singleton. The singleton idea is easy to grasp.

The "exports" objects _are_ a singleton. You might get a copy of the
exports instead of the exact object. The purpose of having a
singleton is not so that you can test them for object identity (not a
real world case) but so that you are guaranteed that there is exactly
one local scope for the module which can be used to share state, and
that cyclic dependencies won't cause infinite recursion.

Kris Kowal

Peter Michaux

unread,
Feb 10, 2009, 12:05:23 AM2/10/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 5:43 PM, Kris Kowal <cowber...@gmail.com> wrote:

> To solve this problem, it is sufficient for a secure module loader to
> copy the exports of the remote module,

That screws up "this" as "this" will refer to the copy object, not the
original exports object. It matters if one of the original export
objects should be mutated when calling one of the exported functions.

> freeze that object, and return
> it each time it is requested.

Each time it is requested, it may have more properties due to how the
circular dependencies work. This will be complex for the user of a
module to understand. "I required the module. Why doesn't it have all
the exported properties?"

Peter

Kris Kowal

unread,
Feb 10, 2009, 12:30:08 AM2/10/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 9:05 PM, Peter Michaux <peterm...@gmail.com> wrote:
> On Mon, Feb 9, 2009 at 5:43 PM, Kris Kowal <cowber...@gmail.com> wrote:

>> To solve this problem, it is sufficient for a secure module loader to
>> copy the exports of the remote module,

> That screws up "this" as "this" will refer to the copy object, not the
> original exports object. It matters if one of the original export
> objects should be mutated when calling one of the exported functions.

At this juncture, I don't think it's safe to assume that "this" is any
particular object in module scope or in the scope of an exported
function. It is also sufficient, readable, and deterministic to use
"exports" instead of "this" in both scopes. In an exported function,
the value of "this" depends on whether you call the exported function
as a method of the module or as a stand-alone function, and the module
user ought to have the right to chose.

Shall we place normative clauses banning use of "this" in these cases?

>> freeze that object, and return
>> it each time it is requested.

> Each time it is requested, it may have more properties due to how the
> circular dependencies work. This will be complex for the user of a
> module to understand. "I required the module. Why doesn't it have all
> the exported properties?"

This is orthogonal. If an imports object is partial, and you try to
bind its exports to locals, you'll meet the same confusion. In either
case, when you have mutually dependent modules, they must import each
other in a consistent order, and it's easiest if the latter modules
are required and bound at the bottom of the file.

Kris Kowal

Peter Michaux

unread,
Feb 10, 2009, 1:38:08 AM2/10/09
to serv...@googlegroups.com
On Mon, Feb 9, 2009 at 9:30 PM, Kris Kowal <cowber...@gmail.com> wrote:
>
> On Mon, Feb 9, 2009 at 9:05 PM, Peter Michaux <peterm...@gmail.com> wrote:
>> On Mon, Feb 9, 2009 at 5:43 PM, Kris Kowal <cowber...@gmail.com> wrote:
>
>>> To solve this problem, it is sufficient for a secure module loader to
>>> copy the exports of the remote module,
>
>> That screws up "this" as "this" will refer to the copy object, not the
>> original exports object. It matters if one of the original export
>> objects should be mutated when calling one of the exported functions.
>
> At this juncture, I don't think it's safe to assume that "this" is any
> particular object in module scope or in the scope of an exported
> function. It is also sufficient, readable, and deterministic to use
> "exports" instead of "this" in both scopes. In an exported function,
> the value of "this" depends on whether you call the exported function
> as a method of the module or as a stand-alone function, and the module
> user ought to have the right to chose.
>
> Shall we place normative clauses banning use of "this" in these cases?

I'm not sure. I'm not happy about the direction assignment to the
exports object is heading. The "exports" object is becoming so much
less like a normal JavaScript object and more like a funky host
object. It is almost as odd as assigning to the cookies object (more
like a += operation) which has to be the least intuitive API in the
browser. If the intuitive meaning of assignment to the exports object
is not what assignment to the exports object does, then assignment is
the wrong API as it is confusing how it works. Programmers will need
to be trained about how this strange new exports host object works and
even then it will challenge their intuition at times when they just
slip into seeing it as assignment to a property.

If the "exports" object is not just a plain old object, I'm going to
reconsider a function like

export("foo", foo);

as that shows foo is being sent off into the abyss somewhere and I
cannot assume anything like I will with property assignment. I think
my intuition about how this export() function would work is closer to
how you and Ihab see the "export" keyword working 5-10 years from now
if you are successful. It can intuitively be a set-once for a given
key. It can intuitively be clear that only the owner module can affect
its exports (i.e. no other module can fiddle with the object returned
by "require".)

I'm also thinking that if the exports object has such strange behavior
with circular dependencies that it would be better to change "require"
to take multiple arguments or return a function to get exports.
something like one (or actually all) of these

var a = require('Foo", 'a');

var [a, b, c] = require("Foo", 'a', 'b', 'c');

var [a, b, c] = require("Foo", ['a', 'b', 'c']);

var foogetter = require('Foo');
var b = foogetter(b);
var a = require('Foo')(a);

Getting by a function clears up the problem where the object returned
by "require" might be a copy and/or snapshot of the module's "exports"
object with properties not yet defined missing. The fact that I'd need
to take a snapshot just before using any exported properties of
another module, to ensure it has finally been exported, is
troublesome. See below.

>>> freeze that object, and return
>>> it each time it is requested.
>
>> Each time it is requested, it may have more properties due to how the
>> circular dependencies work. This will be complex for the user of a
>> module to understand. "I required the module. Why doesn't it have all
>> the exported properties?"
>
> This is orthogonal. If an imports object is partial, and you try to
> bind its exports to locals, you'll meet the same confusion.

Binding to locals is not what I was thinking. The following code
doesn't care when the foo property is bound to module A's export
object.

var A = require('A')

var bar = function() {
return A.foo();
}

As long as foo is bound to exports in A before it is called, all is
well. This is not true with a snapshot where foo may not have been
present when I required A, but would be present if I did the following

var bar = function() {
return require('A').foo();
};

I wouldn't want to see this style of programming become popular as
these snapshots would be expensive.

Peter

Robert Cerny

unread,
Feb 10, 2009, 4:53:52 AM2/10/09
to serv...@googlegroups.com
On Feb 10, 2009, at 7:38 AM, Peter Michaux wrote:

> If the "exports" object is not just a plain old object, I'm going to
> reconsider a function like
>
> export("foo", foo);

"export" is a reserved JavaScript keyword, thus my call for "provide".

But my main point is the following: The author wants to provide some
product of his module for reuse, so that upon a call of require to
the module, the product can be reassigned to a local variable and
subsequently used. That is what he wants to do. How this actually
works is not important to him. That is why a function is preferable
over an assignment to some object, which has very specific
interpretation which little room for variation. Additionally it gives
the implementors freedom to find their own solutions to all the
problems that have been discussed in this thread. And it is easier to
specify and communicate.

Robert

Robert Cerny

unread,
Feb 10, 2009, 7:50:44 AM2/10/09
to serv...@googlegroups.com
On Feb 9, 2009, at 10:12 PM, Kris Kowal wrote:

> On Mon, Feb 9, 2009 at 3:27 AM, Robert Cerny
> <robert...@gmail.com> wrote:
>> vendor module A requires module B as "module1"
>> vendor module C requires module D as "module1"
>>
>> my application requires module A and module C.
>>
>> Would that still work? And if so why?
>
> The "as" clause binds the exported value in local scope, and the
> "from" clause is bound in the module identifier name space.
>
> Python:
> module A: import B as module1
> module C: import D as module1
> Transitional (our proposal):
> module A: var module1 = require("B");
> module C: var module1 = require("D");
> Future ES (our TC39 proposal):
> module A: import "B" as module1
> module C: import "D" as module1
>
> In all of these cases "module1" is scoped in the module and thus not
> shared between the modules, giving each module "sovereignty" over its
> local/private name space and orthogonally separating the "module
> identifier name space".

I did not make myself clear. Here is another attempt:

The proposed concept as i understand it:
1. there is no fully qualified, canonical identifier under which a
module is referred to when required
2. every module can choose by which identifier it refers to another
module, depending on the filesystem at development time

Then, how is it possible to disambiguate if two modules from
different vendors use the same identifier for two different modules?

Your Python example as i meant it:
module A: import module1 as whatever1
module C: import module1 as whatever2

Creator of module A and creator of module C do not mean the same
module when they write module1.

Kris, i think you addressed something in another thread, where the
python community had problems using relative identifiers.

Robert

Kevin Dangoor

unread,
Feb 10, 2009, 10:53:29 AM2/10/09
to serv...@googlegroups.com
On Tue, Feb 10, 2009 at 7:50 AM, Robert Cerny <robert...@gmail.com> wrote:
> Then, how is it possible to disambiguate if two modules from
> different vendors use the same identifier for two different modules?
>
> Your Python example as i meant it:
> module A: import module1 as whatever1
> module C: import module1 as whatever2
>
> Creator of module A and creator of module C do not mean the same
> module when they write module1.

This has not generally been an issue in Python. Every now and then,
with a very generic name, people have released tarballs with the same
top-level package name. But, this has been rare among thousands of
packages registered at pypi.python.org

> Kris, i think you addressed something in another thread, where the
> python community had problems using relative identifiers.

Likely a different issue.

my_package /
__init__.py
bar.py
foo.py

foo /
__init__.py


With the above files on the PYTHONPATH, the issue was in
my_package/bar.py saying:

import foo

Python supported relative imports, so it was not obvious which "foo"
would be loaded. Python has since added syntax for explicit relative
imports vs. top level imports.

ihab...@gmail.com

unread,
Feb 10, 2009, 11:21:56 AM2/10/09
to serv...@googlegroups.com
Responding to just a small portion, by way of clarification --

On Mon, Feb 9, 2009 at 10:38 PM, Peter Michaux <peterm...@gmail.com> wrote:
> Binding to locals is not what I was thinking. The following code
> doesn't care when the foo property is bound to module A's export
> object.
>
> var A = require('A')
>
> var bar = function() {
> return A.foo();
> }

It's worth noting that the "when" problem *only* arises in the case of
a circular dependency among modules. Kris and I have worried about
that because we are trying to find a general solution, and because
these are the sorts of corner cases that ECMA committees like to pick
apart. :)

The 99.99% common case is this: If a module has not yet been
instantiated, the instantiation starts *and completes* within the
require() call that first caused the module to be instantiated, and
the result of that require() is the fully instantiated module.

The reason the circular dependency case becomes funky is that we want
to be both correct and flexible. Correctness implies a well-defined
specification so stuff works predictably. Flexibility implies that we
provide programmers in *both* the require()-ing and require()-ed
modules a place to stand to rearrange stuff so that the circularity
gets resolved in a way that makes everyone happy. Very importantly, we
have to provide enough hooks so that a programmer who controls only
*one* side of the link that caused the circularity can do enough
rearrangement to resolve any obvious problems.

All this, and yes, we *still* want the simple 99.99% case to work
without surprises. :)

Reply all
Reply to author
Forward
0 new messages